Speed of data access in arrays

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Steven James Samuel Stapleton

    Speed of data access in arrays

    Will calling ksort() on an array speed up it's access?

    For example, I have the array $file_index, which is accessed by a key (the
    entry id) and has two sub elements in a one dimensional array (line number
    of the start of the entry in the file, and the byte offset of the start of
    the entry).

    so, I might have an array that can be represented thus:

    {
    2 => {1, 100),
    4 => (30, 556),
    10 => (376, 2637),
    ...
    }

    These usually have thousands of entries, so the arrays are quite large, and
    anything I can do to speed up the data access will be big.

    One file, contains under 10MB of data in under 2000 entries, the next
    contains about 35MB of data in around 8000 entries, and the final contains
    about 125MB of data in 20000+ entries, and I'm noticing significant lag on
    the latter, so I was hoping to find a way to speed up the array access.


    Thanks for the help
    -Jim Stapleton


  • Adam i Agnieszka Gasiorowski FNORD

    #2
    Re: Speed of data access in arrays

    Steven James Samuel Stapleton wrote:
    [color=blue]
    > One file, contains under 10MB of data in under 2000 entries, the next
    > contains about 35MB of data in around 8000 entries, and the final contains
    > about 125MB of data in 20000+ entries, and I'm noticing significant lag on
    > the latter, so I was hoping to find a way to speed up the array access.[/color]

    No, sorting won't help.

    --
    Seks, seksiæ, seksolatki... news:pl.soc.sek s.moderowana
    http://hyperreal.info { iWanToDie } WiNoNa ) (
    http://szatanowskie-ladacznice.0-700.pl foReVeR( * )
    Poznaj jej zwiewne kszta³ty... http://www.opera.com 007

    Comment

    • Steven James Samuel Stapleton

      #3
      Re: Speed of data access in arrays

      Does anyone know of a way to speed up the access?

      -Jim Stapleton
      -PCF

      "Adam i Agnieszka Gasiorowski FNORD" <agquarx@venus. ci.uw.edu.pl> wrote in
      message news:3F5C20AD.E 70A59AD@venus.c i.uw.edu.pl...[color=blue]
      > Steven James Samuel Stapleton wrote:
      >[color=green]
      > > One file, contains under 10MB of data in under 2000 entries, the next
      > > contains about 35MB of data in around 8000 entries, and the final[/color][/color]
      contains[color=blue][color=green]
      > > about 125MB of data in 20000+ entries, and I'm noticing significant lag[/color][/color]
      on[color=blue][color=green]
      > > the latter, so I was hoping to find a way to speed up the array access.[/color]
      >
      > No, sorting won't help.
      >
      > --
      > Seks, seksiæ, seksolatki... news:pl.soc.sek s.moderowana
      > http://hyperreal.info { iWanToDie } WiNoNa ) (
      > http://szatanowskie-ladacznice.0-700.pl foReVeR( * )
      > Poznaj jej zwiewne kszta³ty... http://www.opera.com 007[/color]


      Comment

      • Matty

        #4
        Re: Speed of data access in arrays

        Steven James Samuel Stapleton wrote:
        [color=blue]
        > Does anyone know of a way to speed up the access?
        >
        > -Jim Stapleton
        > -PCF
        >
        > "Adam i Agnieszka Gasiorowski FNORD" <agquarx@venus. ci.uw.edu.pl> wrote in
        > message news:3F5C20AD.E 70A59AD@venus.c i.uw.edu.pl...[color=green]
        >> Steven James Samuel Stapleton wrote:
        >>[color=darkred]
        >> > One file, contains under 10MB of data in under 2000 entries, the next
        >> > contains about 35MB of data in around 8000 entries, and the final[/color][/color]
        > contains[color=green][color=darkred]
        >> > about 125MB of data in 20000+ entries, and I'm noticing significant lag[/color][/color]
        > on[color=green][color=darkred]
        >> > the latter, so I was hoping to find a way to speed up the array access.[/color]
        >>
        >> No, sorting won't help.[/color][/color]

        Store it in a DBMS like MySQL - that's what they're designed to do

        Comment

        • Steven James Samuel Stapleton

          #5
          Re: Speed of data access in arrays

          > Store it in a DBMS like MySQL - that's what they're designed to do

          A DBMS is faster than an array?

          -Jim Stapleton


          Comment

          • hexkid

            #6
            Re: Speed of data access in arrays

            Steven James Samuel Stapleton wrote...[color=blue]
            > Does anyone know of a way to speed up the access?[/color]

            Have you tried increasing the memory limit in php.ini?


            memory_limit = 256M


            .... maybe buy more RAM for you computer

            HTH

            Comment

            • Matty

              #7
              Re: Speed of data access in arrays

              Steven James Samuel Stapleton wrote:
              [color=blue][color=green]
              >> Store it in a DBMS like MySQL - that's what they're designed to do[/color]
              >
              > A DBMS is faster than an array?
              >
              > -Jim Stapleton[/color]

              No, but it's faster than dealing with a file, reading it in and parsing
              it, sorting an array, etc, as tyhe code in a DBMS is optimised for the
              tasks, rather than generalised coding support.

              You'll probably also find thjat a good DBMS will scale better on certain
              operations with large sets of data

              Comment

              Working...