speed: file vs. DB

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Olaf Noehring

    speed: file vs. DB

    Hi

    I am the administrator of TSEP - The Search Engine Project (www.tsep.info).
    We developer are unclear about the speed of two different methods:
    Is it faster to
    a) write to a file
    or
    b) update a record in a database

    Olaf

    --
    Datenbank-Projekt bietet IT Services rund um Softwareentwicklung mit dem Schwerpunkt Datenbanken aber auch Beratung und Coaching. Hierzu gehören Datenbankentwurf, Datenbankentwicklung, Datenbankpflege, Datenbankoptimierung und Datenbankadministration; Programmierung, Entwicklung und Weiterentwicklung von offline (MS Access) und online Anwendungen; Softwarewartung, Softwarebetreuung, Softwarepflege, Softwareerweiterung, Softwareauswahl und Softwarewechsel aber auch Datenübernahme und Datentransformation. Beratung, Schulungen und Coaching on-Site und virtuell sowie Fernhilfe.

  • Tony Marston

    #2
    Re: speed: file vs. DB

    Writing is not the issue, reading is. If you have a text file containing
    100,000 records how quickly can you read just one particular record?
    Database tables have keys, so it is very easy to read a record with a
    particular key value. Text files don't have keys, so you cannot go straight
    to a particular record. You have to read serially through the whole file
    until you happen to find the record you want.

    If you know so little about databases then what makes you think your little
    project will be of use to anyone?

    --
    Tony Marston

    This is Tony Marston's web site, containing personal information plus pages devoted to the Uniface 4GL development language, XML and XSL, PHP and MySQL, and a bit of COBOL




    "Olaf Noehring" <olaf0000@tea m-noehring.de(000 0=year)> wrote in message
    news:Xns968A700 8FBAADteamnoehr ing@195.14.215. 228...[color=blue]
    > Hi
    >
    > I am the administrator of TSEP - The Search Engine Project
    > (www.tsep.info).
    > We developer are unclear about the speed of two different methods:
    > Is it faster to
    > a) write to a file
    > or
    > b) update a record in a database
    >
    > Olaf
    >
    > --
    > http://www.team-noehring.de[/color]


    Comment

    • Erwin Moller

      #3
      Re: speed: file vs. DB

      0000=year wrote:
      [color=blue]
      > Hi
      >
      > I am the administrator of TSEP - The Search Engine Project
      > (www.tsep.info). We developer are unclear about the speed of two different
      > methods: Is it faster to
      > a) write to a file
      > or
      > b) update a record in a database
      >
      > Olaf
      >[/color]

      Hi Olaf,

      Do you only want to write some data away, or do you want to retrieve the
      data later?
      If you want to retrieve it later: use a database.
      A database can use indexing, making retrieval of data much faster.

      If you just want to write it away: I do not know. I guess filewriting speed
      depends on filesystem you use, HD-speed, fragmentation, and the number of
      files already on the filesystem.

      My guess is your best bet is a database for most situations.

      Regards,
      Erwin Moller

      Comment

      • Olaf Noehring

        #4
        Re: speed: file vs. DB

        Hi

        to add some more information what we need this for. Sorry, that I forgot
        to throw in this information right away:

        We need to write only about 5 lines to the file - or 1 record to the DB.
        It's only needed to tell the indexer (of TSEP) which reads the contents of
        the files that another instance of the indexer is already running. So, the
        first instance started will write to the file / db record when it has been
        started and that it is running at time X (every 30 seconds)
        The reading (from the file/record) is only done if a instance of the
        indexer is started and there IS a record already.

        Olaf
        [color=blue]
        > I am the administrator of TSEP - The Search Engine Project
        > (www.tsep.info). We developer are unclear about the speed of two
        > different methods: Is it faster to
        > a) write to a file
        > or
        > b) update a record in a database
        >
        > Olaf
        >[/color]



        --
        Datenbank-Projekt bietet IT Services rund um Softwareentwicklung mit dem Schwerpunkt Datenbanken aber auch Beratung und Coaching. Hierzu gehören Datenbankentwurf, Datenbankentwicklung, Datenbankpflege, Datenbankoptimierung und Datenbankadministration; Programmierung, Entwicklung und Weiterentwicklung von offline (MS Access) und online Anwendungen; Softwarewartung, Softwarebetreuung, Softwarepflege, Softwareerweiterung, Softwareauswahl und Softwarewechsel aber auch Datenübernahme und Datentransformation. Beratung, Schulungen und Coaching on-Site und virtuell sowie Fernhilfe.

        Comment

        • Erwin Moller

          #5
          Re: speed: file vs. DB

          0000=year wrote:
          [color=blue]
          > Hi
          >
          > to add some more information what we need this for. Sorry, that I forgot
          > to throw in this information right away:
          >
          > We need to write only about 5 lines to the file - or 1 record to the DB.
          > It's only needed to tell the indexer (of TSEP) which reads the contents of
          > the files that another instance of the indexer is already running. So, the
          > first instance started will write to the file / db record when it has been
          > started and that it is running at time X (every 30 seconds)
          > The reading (from the file/record) is only done if a instance of the
          > indexer is started and there IS a record already.
          >
          > Olaf[/color]

          Use a database. :-)
          You ARE accessing the data (files) again, so you better switch to a real
          solution (database) right away.

          Just think about it: You have 5.000.000 files in a directory on a filesystem
          like NTFS. You need a file with the name xyzwtq.txt
          How to find it?
          The system has to scan through all filenames stored and compare it to your
          filename.
          If you have some smart indexing going on, this will seriously reduce the
          seekspeed.

          Maybe there are some fancy filesystems out there that can compete with a
          database by using some smart indexing, I do not know to be honest.

          Regards,
          Erwin Moller

          Comment

          • Bert Melis

            #6
            Re: speed: file vs. DB

            Olaf Noehring wrote:[color=blue]
            > Hi
            >
            > I am the administrator of TSEP - The Search Engine Project (www.tsep.info).
            > We developer are unclear about the speed of two different methods:
            > Is it faster to
            > a) write to a file
            > or
            > b) update a record in a database
            >
            > Olaf
            >[/color]
            Designers of databases have done all the very fast coding for you, why
            shouldn't you use it?

            Comment

            • Chung Leong

              #7
              Re: speed: file vs. DB

              In that case I would use a file. Using a database seems overkill.

              Comment

              • SOR

                #8
                Re: speed: file vs. DB

                <comp.lang.ph p , Tony Marston , tony@NOSPAM.dem on.co.uk>
                <dadlnb$t5j$1$8 30fa795@news.de mon.co.uk>
                <Tue, 5 Jul 2005 11:00:10 +0100>
                [color=blue]
                > Writing is not the issue, reading is. If you have a text file containing
                > 100,000 records how quickly can you read just one particular record?
                > Database tables have keys, so it is very easy to read a record with a
                > particular key value. Text files don't have keys, so you cannot go straight
                > to a particular record. You have to read serially through the whole file
                > until you happen to find the record you want.
                >[/color]

                Not always , As a newish user I done www.sparesorrepair.co.uk using
                flatfiles and each advert is saved on its own and called/displayed by
                the advert number .

                (obviously not saying its the best way to do it)

                Comment

                Working...