speed: file vs. DB

[Olaf Noehring]

Hi

I am the administrator of TSEP - The Search Engine Project (www.tsep.info).
We developer are unclear about the speed of two different methods:
Is it faster to
a) write to a file
or
b) update a record in a database

Olaf

--

http://www.team-noehring.de

[Tony Marston]

Re: speed: file vs. DB

Writing is not the issue, reading is. If you have a text file containing
100,000 records how quickly can you read just one particular record?
Database tables have keys, so it is very easy to read a record with a
particular key value. Text files don't have keys, so you cannot go straight
to a particular record. You have to read serially through the whole file
until you happen to find the record you want.

If you know so little about databases then what makes you think your little
project will be of use to anyone?

--
Tony Marston

http://www.tonymarston.net

"Olaf Noehring" <olaf0000@tea m-noehring.de(000 0=year)> wrote in message
news:Xns968A700 8FBAADteamnoehr ing@195.14.215. 228...[color=blue]
> Hi
>
> I am the administrator of TSEP - The Search Engine Project
> (www.tsep.info).
> We developer are unclear about the speed of two different methods:
> Is it faster to
> a) write to a file
> or
> b) update a record in a database
>
> Olaf
>
> --
> http://www.team-noehring.de[/color]

[Erwin Moller]

Re: speed: file vs. DB

0000=year wrote:
[color=blue]
> Hi
>
> I am the administrator of TSEP - The Search Engine Project
> (www.tsep.info). We developer are unclear about the speed of two different
> methods: Is it faster to
> a) write to a file
> or
> b) update a record in a database
>
> Olaf
>[/color]

Hi Olaf,

Do you only want to write some data away, or do you want to retrieve the
data later?
If you want to retrieve it later: use a database.
A database can use indexing, making retrieval of data much faster.

If you just want to write it away: I do not know. I guess filewriting speed
depends on filesystem you use, HD-speed, fragmentation, and the number of
files already on the filesystem.

My guess is your best bet is a database for most situations.

Regards,
Erwin Moller

[Olaf Noehring]

Re: speed: file vs. DB

Hi

to add some more information what we need this for. Sorry, that I forgot
to throw in this information right away:

We need to write only about 5 lines to the file - or 1 record to the DB.
It's only needed to tell the indexer (of TSEP) which reads the contents of
the files that another instance of the indexer is already running. So, the
first instance started will write to the file / db record when it has been
started and that it is running at time X (every 30 seconds)
The reading (from the file/record) is only done if a instance of the
indexer is started and there IS a record already.

Olaf
[color=blue]
> I am the administrator of TSEP - The Search Engine Project
> (www.tsep.info). We developer are unclear about the speed of two
> different methods: Is it faster to
> a) write to a file
> or
> b) update a record in a database
>
> Olaf
>[/color]



--

http://www.team-noehring.de

[Erwin Moller]

Re: speed: file vs. DB

0000=year wrote:
[color=blue]
> Hi
>
> to add some more information what we need this for. Sorry, that I forgot
> to throw in this information right away:
>
> We need to write only about 5 lines to the file - or 1 record to the DB.
> It's only needed to tell the indexer (of TSEP) which reads the contents of
> the files that another instance of the indexer is already running. So, the
> first instance started will write to the file / db record when it has been
> started and that it is running at time X (every 30 seconds)
> The reading (from the file/record) is only done if a instance of the
> indexer is started and there IS a record already.
>
> Olaf[/color]

Use a database. :-)
You ARE accessing the data (files) again, so you better switch to a real
solution (database) right away.

Just think about it: You have 5.000.000 files in a directory on a filesystem
like NTFS. You need a file with the name xyzwtq.txt
How to find it?
The system has to scan through all filenames stored and compare it to your
filename.
If you have some smart indexing going on, this will seriously reduce the
seekspeed.

Maybe there are some fancy filesystems out there that can compete with a
database by using some smart indexing, I do not know to be honest.

Regards,
Erwin Moller

[Bert Melis]

Re: speed: file vs. DB

Olaf Noehring wrote:[color=blue]
> Hi
>
> I am the administrator of TSEP - The Search Engine Project (www.tsep.info).
> We developer are unclear about the speed of two different methods:
> Is it faster to
> a) write to a file
> or
> b) update a record in a database
>
> Olaf
>[/color]
Designers of databases have done all the very fast coding for you, why
shouldn't you use it?

[Chung Leong]

Re: speed: file vs. DB

In that case I would use a file. Using a database seems overkill.

[SOR]

Re: speed: file vs. DB

<comp.lang.ph p , Tony Marston , tony@NOSPAM.dem on.co.uk>
<dadlnb$t5j$1$8 30fa795@news.de mon.co.uk>
<Tue, 5 Jul 2005 11:00:10 +0100>
[color=blue]
> Writing is not the issue, reading is. If you have a text file containing
> 100,000 records how quickly can you read just one particular record?
> Database tables have keys, so it is very easy to read a record with a
> particular key value. Text files don't have keys, so you cannot go straight
> to a particular record. You have to read serially through the whole file
> until you happen to find the record you want.
>[/color]

Not always , As a newish user I done www.sparesorrepair.co.uk using
flatfiles and each advert is saved on its own and called/displayed by
the advert number .

(obviously not saying its the best way to do it)