data handling issue

[curwen]

Hi all,
I'd like to have you opinion about the following performance/design
issue:

a logging system is generating a lot of records every our,(about
500k/day)
I need to store them in a huge Oracle RAC system
a Java web application connects to the data and executes queries on
them, mainly on last entered set of records
queries on informations stored more then 3 months ago are quite rare

as first guess I managed to store them in a table indexed by a
primary key, let's say in the table 'LOGS'
the problem is that table is going to grows very big,

have you got 'best practice' ideas to handle the situation in a better
way?

I thought about moving data priodically in a 'LOGS_ARCHIVE' table ,
for example using a Oracle job
is it necessary?
PS I can't use Oracle standard file archiving support, I need data to
stay on tables

thanks in advance
jc

[Jim Kennedy]

Re: data handling issue


"curwen" <josephcurwen@d espammed.comwro te in message
news:cc68edc4.0 402250553.18a9d abf@posting.goo gle.com...

Hi all,
I'd like to have you opinion about the following performance/design
issue:
>
a logging system is generating a lot of records every our,(about
500k/day)
I need to store them in a huge Oracle RAC system
a Java web application connects to the data and executes queries on
them, mainly on last entered set of records
queries on informations stored more then 3 months ago are quite rare
>
as first guess I managed to store them in a table indexed by a
primary key, let's say in the table 'LOGS'
the problem is that table is going to grows very big,
>
have you got 'best practice' ideas to handle the situation in a better
way?
>
I thought about moving data priodically in a 'LOGS_ARCHIVE' table ,
for example using a Oracle job
is it necessary?
PS I can't use Oracle standard file archiving support, I need data to
stay on tables
>
thanks in advance
jc

I always find it strange that we see requests like this. Sort of along the
lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some
knowledge and expertise on Oracle? It just strikes me as very odd.
Jim

[Ed prochak]

Re: data handling issue

"Jim Kennedy" <kennedy-downwithspammer sfamily@attbi.n etwrote in message news:<%Y3%b.560 84$Xp.266401@at tbi_s54>...
[]

I always find it strange that we see requests like this. Sort of along the
lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some
knowledge and expertise on Oracle? It just strikes me as very odd.
Jim

My guess is the manager gets the cheapest programmer available and
keeps asking him to take over responsibilitie s of the more
knowledgable folks as they are laid off. Of course then the manager
hears about outsourcing to India at even lower rates and the pattern
begins again. 8^)

But seriously, I'm as puzzled as you are.
Ed

[curwen]

Re: data handling issue

jc

I always find it strange that we see requests like this. Sort of along the
lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have some
knowledge and expertise on Oracle? It just strikes me as very odd.
Jim

My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you

plus, I'll be much more professional answering in Usenet

thanks anyway
jc

[curwen]

Re: data handling issue

ed.prochak@magi cinterface.com (Ed prochak) wrote in message news:<4b5394b2. 0402261042.7986 17d9@posting.go ogle.com>...

"Jim Kennedy" <kennedy-downwithspammer sfamily@attbi.n etwrote in message
>
My guess is the manager gets the cheapest programmer available and
keeps asking him to take over responsibilitie s of the more
knowledgable folks as they are laid off. Of course then the manager
hears about outsourcing to India at even lower rates and the pattern
begins again. 8^)
>
But seriously, I'm as puzzled as you are.
Ed

My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you

plus, I'll be much more professional answering in Usenet

thanks anyway
jc

[Jim Kennedy]

Re: data handling issue


"curwen" <josephcurwen@d espammed.comwro te in message
news:cc68edc4.0 402270027.4c59b 7c9@posting.goo gle.com...

jc

I always find it strange that we see requests like this. Sort of along

the

lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have

some

knowledge and expertise on Oracle? It just strikes me as very odd.
Jim

>
>
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
>
plus, I'll be much more professional answering in Usenet
>
thanks anyway
jc

I am sure you are a hard working good person. My comment has nothing to do
with your personality, but rather why people do this. (more the fault of
the manager from not getting experienced people for a task that needs an
experienced person) It is sort of like I've been hired to design a Boeing
747 and I have experience building paper airplanes. There is quite a gulf
there in needed experience.
Jim

[curwen]

Re: data handling issue

Jim,
I do have some experience, maybe this is the most complex project I
have dealt with, but we have enough time and resources to improve the
skills if required..
anyway the project is not so mission critical, even if a lot of data
are expected, and from a general point of view the factors are quite
calibrated

anyway we used range partitioning to handle the issue

best regards
jc


"Jim Kennedy" <kennedy-downwithspammer sfamily@attbi.n etwrote in message news:<HgI%b.298 33$AL.531756@at tbi_s03>...

"curwen" <josephcurwen@d espammed.comwro te in message
news:cc68edc4.0 402270027.4c59b 7c9@posting.goo gle.com...

jc
I always find it strange that we see requests like this. Sort of along

the

lines of "I am building this huge Oracle database and I don't know much
about Oracle but..." Wouldn't one get people on the project who have

some

knowledge and expertise on Oracle? It just strikes me as very odd.
Jim

My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you

plus, I'll be much more professional answering in Usenet

thanks anyway
jc

I am sure you are a hard working good person. My comment has nothing to do
with your personality, but rather why people do this. (more the fault of
the manager from not getting experienced people for a task that needs an
experienced person) It is sort of like I've been hired to design a Boeing
747 and I have experience building paper airplanes. There is quite a gulf
there in needed experience.
Jim

[Ed prochak]

Re: data handling issue

josephcurwen@de spammed.com (curwen) wrote in message news:<cc68edc4. 0402270027.3b77 55b9@posting.go ogle.com>...

ed.prochak@magi cinterface.com (Ed prochak) wrote in message news:<4b5394b2. 0402261042.7986 17d9@posting.go ogle.com>...

"Jim Kennedy" <kennedy-downwithspammer sfamily@attbi.n etwrote in message

My guess is the manager gets the cheapest programmer available and
keeps asking him to take over responsibilitie s of the more
knowledgable folks as they are laid off. Of course then the manager
hears about outsourcing to India at even lower rates and the pattern
begins again. 8^)

But seriously, I'm as puzzled as you are.
Ed

>
My experience in computer science is limited, I only have a First
Class Masters degree in Electronics Engineering, but I'm confident
with hard working and studying I can become 'good' as you
>
plus, I'll be much more professional answering in Usenet
>
thanks anyway
jc

The point is that there is little to offer in way of answering your
original question. And the little bit of information you did provide
sounds so much like the many
just-out-of-college-web_guru-now-dba-on-Oracle posts that we have to
wonder. Now with your additional information, I really wonder why the
manager hired you when there are experienced Oracle folks around. Try
rereading my post, I was making a comment about the hiring manager,
not you.

But since you brough it up. If you are the DBA, yet you cannot find
out how to add space to the databasebase as needed, or what partioning
of tables can get you, or any of a dozen other DBA tasks I have to
wonder.

You never mentioned ORACLE version, hosting platform. You only hint
about size (500k records per DAY is not necessarily BIG).


(Boy I've been in a grouchy mood recently.)

Finally, FYI, this group is not really supported. The official groups
are
comp.databases. oracle.server
comp.databases. oracle.misc
comp.databases. oracle.tools
comp.databases. oracle.marketpl ace

Some servers still carry this group, but you'll get wider support
posting in the appropriate one of those four.

Sorry you took offense. Rephrase your question and you'll get a better
response.

good luck.
Ed