getting file size for really big (10GB) files?

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Martin
    #1

    getting file size for really big (10GB) files?

    Hi all.
    I am developing a filemanager that needs to handle big files. While
    testing on some zipped files of 6-7GB each I noticed that filesize(),
    filemtime() and similar php-functions can't handle fikles larger than
    2GB. This is true on the two servers I have regular access to (one
    php4-RedHat, the other php5-Fedora).

    If there a way to ger around these functions somehow? My fallback plan
    is to use the systems functions and then parse the output-text to get
    at the needed data. Naturally I was looking for a prettier way to do
    it.

    I'd appreciate any tips on the subject. thanks.

    Tags: None
    • tony@tony.com
      #2
      Re: getting file size for really big (10GB) files?

      In article <1151763544.147 885.137130@m79g 2000cwm.googleg roups.com>,
      martin@eimermus ic.com says...[color=blue]
      > Hi all.
      > I am developing a filemanager that needs to handle big files. While
      > testing on some zipped files of 6-7GB each I noticed that filesize(),
      > filemtime() and similar php-functions can't handle fikles larger than
      > 2GB. This is true on the two servers I have regular access to (one
      > php4-RedHat, the other php5-Fedora).
      >
      > If there a way to ger around these functions somehow? My fallback plan
      > is to use the systems functions and then parse the output-text to get
      > at the needed data. Naturally I was looking for a prettier way to do
      > it.
      >
      > I'd appreciate any tips on the subject. thanks.
      >
      >[/color]

      I must be honest - I would choose another language if I wanted to do that
      (10 Gb is quite a small file in my world)

      PHP takes the absurd step of converting any integer it doesnt like into
      a float on the fly at runtime - a more dangerous practice I can't
      imagine. The upshot is maybe you'll come unstuck trying to manipulate
      large integers.

      Obviously you can work around preventing that from being a problem but
      you shouldn't have to.

      I dont know if its relevant to your situation but I thought I'd just
      highlight it because it caused me big problems.



      tony

        Comment

        • ImOk
          #3
          Re: getting file size for really big (10GB) files?

          My suggestion is you shell out and execute 'ls' and send output to a
          txt file and then parse the result if you just need the file size.

          Martin wrote:
          Hi all.
          I am developing a filemanager that needs to handle big files. While
          testing on some zipped files of 6-7GB each I noticed that filesize(),
          filemtime() and similar php-functions can't handle fikles larger than
          2GB. This is true on the two servers I have regular access to (one
          php4-RedHat, the other php5-Fedora).
          >
          If there a way to ger around these functions somehow? My fallback plan
          is to use the systems functions and then parse the output-text to get
          at the needed data. Naturally I was looking for a prettier way to do
          it.
          >
          I'd appreciate any tips on the subject. thanks.

            Comment

            • Martin
              #4
              Re: getting file size for really big (10GB) files?


              Thanks for both your replys.

              I have noticed that PHP does not handle big files well in any
              situation. The manual brushes by this and, for example, suggests I use
              sprinf to make an unsigned number to get the correct size for files up
              to 4GB. So very 90s. :)

              I am surprised that PHP can't simply get the same result as 'ls'. I was
              thinking of using it but the downside is that it imposes some
              limitations on the server installation I can use. Ideally I would like
              to avoid exec() and other commands that is often blocked on hosting
              servers. (even though most won't like 10Gb files anyway)

              The bigger problem is really that different installations return vastly
              different results for filesize(). My local php4 (Mac OS X) at least
              returns a number while the php5 (Fedora) server I use throws a warning
              and returns false on the same file. Php5 on Fedora I really expected to
              be up to it.

              Sorry for the little rant. This is one area where I assumed PHP would
              be OK. Developing an application for a few months and at the final
              testing stage finding out these problems is no fun. You can imagine how
              much headscratching I have done to get the uploading and downloading of
              these huge files to work... and then I stumble on this "detail".

              Thanks again guys.

                Comment

                • ImOk
                  #5
                  Re: getting file size for really big (10GB) files?

                  Well, if it makes you feel better (and worse after reading) here isthe
                  bug report. Its supposed to be fixed, but no one has done anything in a
                  few years. I am surprised. I guess people jut use Java for issues like
                  this.

                  PHP :: Bug #27792 :: [PATCH] Functions fail on large files (filesize,is_file,is_dir,readdir)
                  http://bugs.php.net/bug.php?id=27792




                  Martin wrote:
                  Thanks for both your replys.
                  >
                  I have noticed that PHP does not handle big files well in any
                  situation. The manual brushes by this and, for example, suggests I use
                  sprinf to make an unsigned number to get the correct size for files up
                  to 4GB. So very 90s. :)
                  >
                  I am surprised that PHP can't simply get the same result as 'ls'. I was
                  thinking of using it but the downside is that it imposes some
                  limitations on the server installation I can use. Ideally I would like
                  to avoid exec() and other commands that is often blocked on hosting
                  servers. (even though most won't like 10Gb files anyway)
                  >
                  The bigger problem is really that different installations return vastly
                  different results for filesize(). My local php4 (Mac OS X) at least
                  returns a number while the php5 (Fedora) server I use throws a warning
                  and returns false on the same file. Php5 on Fedora I really expected to
                  be up to it.
                  >
                  Sorry for the little rant. This is one area where I assumed PHP would
                  be OK. Developing an application for a few months and at the final
                  testing stage finding out these problems is no fun. You can imagine how
                  much headscratching I have done to get the uploading and downloading of
                  these huge files to work... and then I stumble on this "detail".
                  >
                  Thanks again guys.

                    Comment

                    • micro@fan.com
                      #6
                      Re: getting file size for really big (10GB) files?

                      In article <1151891760.872 363.53690@a14g2 000cwb.googlegr oups.com>,
                      j_macaroni@yaho o.com says...
                      Well, if it makes you feel better (and worse after reading) here isthe
                      bug report. Its supposed to be fixed, but no one has done anything in a
                      few years. I am surprised. I guess people jut use Java for issues like
                      this.
                      >
                      PHP :: Bug #27792 :: [PATCH] Functions fail on large files (filesize,is_file,is_dir,readdir)
                      http://bugs.php.net/bug.php?id=27792

                      >
                      >
                      I cant imagine why anyone would use java for anything at all.

                        Comment

                        • ImOk
                          #7
                          Re: getting file size for really big (10GB) files?

                          Ok, Java is an overkill. But if you have to overcome a bug then maybe
                          use Ruby or Python or Perl.

                          micro@fan.com wrote:
                          In article <1151891760.872 363.53690@a14g2 000cwb.googlegr oups.com>,
                          j_macaroni@yaho o.com says...
                          Well, if it makes you feel better (and worse after reading) here isthe
                          bug report. Its supposed to be fixed, but no one has done anything in a
                          few years. I am surprised. I guess people jut use Java for issues like
                          this.

                          PHP :: Bug #27792 :: [PATCH] Functions fail on large files (filesize,is_file,is_dir,readdir)
                          http://bugs.php.net/bug.php?id=27792

                          I cant imagine why anyone would use java for anything at all.

                            Comment

                            Previous template Next
                            Working...