HELP: Joining 5 large text files

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Stuart

    HELP: Joining 5 large text files

    Hi,

    Please can anyone help me join 5 large (1.8gb) text files togeather
    to create 1 very large file.

    I have some code in PHP but it bombs out at 2gb (seems there is a limit
    and php needs re compiling, something thats not an option for me)

    I don't want to take up all the servers resources so something that uses
    little memory would be very good indeed!

    here's the php code if it help...

    <?php
    set_time_limit( 14400000);

    $file[0] = "file1.txt" ;
    $file[1] = "file2.txt" ;
    $file[2] = "file3.txt" ;
    $file[3] = "file4.txt" ;
    $file[4] = "file5.txt" ;

    $target = "output.txt ";
    $handle = fopen($target, 'a');

    foreach ($file as $var) {
    $c_handle = fopen($var, 'r');
    do {
    $content = fread($c_handle ,1000000);
    fwrite($handle, $content);
    } while (!empty($conten t));
    fclose($c_handl e);
    }
    ?>


    all the best

    Stu
  • bmay2068@hotmail.com

    #2
    Re: HELP: Joining 5 large text files

    stuayre@hotmail .com (Stuart) wrote in message news:<763485f6. 0408190205.ea3d 69f@posting.goo gle.com>...
    [color=blue]
    > I have some code in PHP but it bombs out at 2gb (seems there is a limit
    > and php needs re compiling, something thats not an option for me)[/color]

    2gigs is a magic number for the maximum file size on some OSes; 2^31.
    (You loose a bit due to signed variable usage on some file systems.)

    What OS, Architecture are you using? What type of file system i.e.
    XFS, EXT2, WINDoze

    Cheers!

    Comment

    • nobull@mail.com

      #3
      Re: HELP: Joining 5 large text files

      stuayre@hotmail .com (Stuart) wrote in message news:<763485f6. 0408190205.ea3d 69f@posting.goo gle.com>...[color=blue]
      > Newsgroup: comp.lang.perl[/color]
      [color=blue]
      > I have some code in PHP[/color]

      Perhaps you should try a newsgroup that:

      1) Exists (this one doesn't)
      2) Relates to PHP (this one wouldn't, if it existed)

      Comment

      • Joe Smith

        #4
        Re: HELP: Joining 5 large text files

        Stuart wrote:
        [color=blue]
        > create 1 very large file.
        >
        > I have some code in PHP but it bombs out at 2gb (seems there is a limit
        > and php needs re compiling, something thats not an option for me)[/color]

        You're screwed. Unless PHP is recompiled with LARGE_FILE is enabled,
        you will *not* be able to write anything past byte number 2147483647.
        -Joe

        Comment

        Working...