Function like file() but will read lines X through Y only?

[Sugapablo]

file() works great, until you start looking at file that are very large,
say 134MB.

Is there any function that will return an array for each line of a file,
but only for lines given as parameters?

A function like file("/path/to/file",100,230) which would return lines 100
through 230 as an array.


--
[ Sugapablo ]
[ http://www.sugapablo.net <--personal | http://www.sugapablo.com <--music ]
[ http://www.2ra.org <--political | http://www.subuse.net <--discuss ]

[Bent Stigsen]

Re: Function like file() but will read lines X through Y only?

Sugapablo wrote:[color=blue]
> file() works great, until you start looking at file that are very large,
> say 134MB.
>
> Is there any function that will return an array for each line of a file,
> but only for lines given as parameters?
>
> A function like file("/path/to/file",100,230) which would return lines 100
> through 230 as an array.[/color]

Nope. Have to write it yourself.

There is a lot of lazy ways to do it though. PHP (fopen/fgets), piping
from head to tail, sed, awk, grep.

The shortest to write, given your example, is:

sed -n -e 231q -e 100,+130p '/path/to/file'


But not the best choice for performance in all cases. If you only need
to read a small amount of data from the beginning of the file, then use
fopen/fgets in PHP to fetch the lines you need. Even though PHP is slow
compared to the shell-utils, running shellcommands from PHP is costly
enough to sometimes opt for PHP.

If you need to fetch lines from anywhere in the file, then I think grep
would be better. Not sure how head with tail would perfom, or if some
other tool exist.

Those are the lazy ways. If performance is still a problem, then you
need to do other things.


/Bent

[Janwillem Borleffs]

Re: Function like file() but will read lines X through Y only?

Sugapablo wrote:[color=blue]
> file() works great, until you start looking at file that are very
> large, say 134MB.
>
> Is there any function that will return an array for each line of a
> file, but only for lines given as parameters?
>
> A function like file("/path/to/file",100,230) which would return
> lines 100 through 230 as an array.[/color]

You could try something like the following, which will not take as many
resources, but isn't really fast either:

<?php

function file_from_offse t($file, $start, $end) {
if (!($fp = @fopen($file, "r"))) return false;
$lines = array();
$count = -1;
while (!feof($fp) && (++$count <= $end)) {
$line = fgets($fp, 4096);
if (($count >= $start) && ($count <= $end)) {
$lines[] = $line;
}
}
fclose($fp);
return $lines;
}

print_r(file_fr om_offset("http ://www.google.com/", 10, 13));

?>



JW

[jerry gitomer]

Re: Function like file() but will read lines X through Y only?

Sugapablo wrote:[color=blue]
> file() works great, until you start looking at file that are very large,
> say 134MB.
>
> Is there any function that will return an array for each line of a file,
> but only for lines given as parameters?
>
> A function like file("/path/to/file",100,230) which would return lines 100
> through 230 as an array.
>
>[/color]

Sugapablo,

Someone else has proposed some code for doing what you want and
mentioned that it won't be very fast.

If performance is a consideration you would be better served
using an RDBMS.

HTH
Jerry

[Chung Leong]

Re: Function like file() but will read lines X through Y only?


Sugapablo wrote:[color=blue]
> file() works great, until you start looking at file that are very[/color]
large,[color=blue]
> say 134MB.
>
> Is there any function that will return an array for each line of a[/color]
file,[color=blue]
> but only for lines given as parameters?
>
> A function like file("/path/to/file",100,230) which would return[/color]
lines 100[color=blue]
> through 230 as an array.
>
>
> --
> [ Sugapablo[/color]
][color=blue]
> [ http://www.sugapablo.net <--personal | http://www.sugapablo.com[/color]
<--music ][color=blue]
> [ http://www.2ra.org <--political | http://www.subuse.net[/color]
<--discuss ]

Unless each line contains the same number of characters, you can't read
specific lines within the middle of the file without first reading
everything that comes before.

[Bent Stigsen]

Re: Function like file() but will read lines X through Y only?

jerry gitomer wrote:
[snip][color=blue]
> Sugapablo,
>
> Someone else has proposed some code for doing what you want and
> mentioned that it won't be very fast.[/color]
[snip]

Yup.

I tried different quick and dirty methods, picking out some lines at
different places from a text file with about a million lines (approx.
25MB). ("h|t" is head on file piped to tail)

+---------+---------+-------+-------+-------+-------+-------+
| from | to | PHP | h|t | sed | awk | grep |
+---------+---------+-------+-------+-------+-------+-------+
| 5000 | 5200 | 0.023 | 0.350 | 0.330 | 0.366 | 0.400 |
| 305000 | 305200 | 1.604 | 0.537 | 0.992 | 0.623 | 0.444 |
| 605000 | 605200 | 3.171 | 0.677 | 1.679 | 0.985 | 0.537 |
| 905000 | 905200 | 4.705 | 0.835 | 2.373 | 1.290 | 0.606 |
+---------+---------+-------+-------+-------+-------+-------+

Speedtest is only run once, so not super accurate, but I think
the trend is obvious.


/Bent