Reading a file in reverse order

[Erik Andersson]

Hi!

I need to read a file (line-by-line) in reverse order, without reading the
whole file into memory first.

Is there an easy way of doing this? As in, is there a PHP function I could
use?

The solution has to be platform independent, so "popen("tac $filename")...
wouldn't work.

Example input and ouput:

<myfile.txt input>
line 1
line 2
line 3
</myfile.txt input>

<output from php script>
line 3
line 2
line 1
</output from php script>


All the best,

Erik

--
Erik Andersson ( AussieTraders.c om.au )

http://www.aussietraders.com.au/

- Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
marketplace

[CountScubula]

Re: Reading a file in reverse order

if the files are small, load them into an array $aryFile =
file("filename. txt");

then just reverse the array or read backwords


--
Mike Bradley
http://www.gzentools.com -- free online php tools
"Erik Andersson" <erik@eneit.com > wrote in message
news:40188b55$0 $830$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com
..au...[color=blue]
> Hi!
>
> I need to read a file (line-by-line) in reverse order, without reading the
> whole file into memory first.
>
> Is there an easy way of doing this? As in, is there a PHP function I could
> use?
>
> The solution has to be platform independent, so "popen("tac $filename")...
> wouldn't work.
>
> Example input and ouput:
>
> <myfile.txt input>
> line 1
> line 2
> line 3
> </myfile.txt input>
>
> <output from php script>
> line 3
> line 2
> line 1
> </output from php script>
>
>
> All the best,
>
> Erik
>
> --
> Erik Andersson ( AussieTraders.c om.au )
> http://www.aussietraders.com.au/
> - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> marketplace
>
>[/color]

[Erik Andersson]

Re: Reading a file in reverse order

They're not, hence "without reading the whole file into memory first".

Thanks anyway!

--
Erik Andersson ( AussieTraders.c om.au )

http://www.aussietraders.com.au/

- Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
marketplace

"CountScubu la" <me@scantek.hot mail.com> wrote in message
news:362Sb.1914 4$vG2.15887@new ssvr25.news.pro digy.com...[color=blue]
> if the files are small, load them into an array $aryFile =
> file("filename. txt");
>
> then just reverse the array or read backwords
>
>
> --
> Mike Bradley
> http://www.gzentools.com -- free online php tools
> "Erik Andersson" <erik@eneit.com > wrote in message
>[/color]
news:40188b55$0 $830$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com[color=blue]
> .au...[color=green]
> > Hi!
> >
> > I need to read a file (line-by-line) in reverse order, without reading[/color][/color]
the[color=blue][color=green]
> > whole file into memory first.
> >
> > Is there an easy way of doing this? As in, is there a PHP function I[/color][/color]
could[color=blue][color=green]
> > use?
> >
> > The solution has to be platform independent, so "popen("tac[/color][/color]
$filename")...[color=blue][color=green]
> > wouldn't work.
> >
> > Example input and ouput:
> >
> > <myfile.txt input>
> > line 1
> > line 2
> > line 3
> > </myfile.txt input>
> >
> > <output from php script>
> > line 3
> > line 2
> > line 1
> > </output from php script>
> >
> >
> > All the best,
> >
> > Erik
> >
> > --
> > Erik Andersson ( AussieTraders.c om.au )
> > http://www.aussietraders.com.au/
> > - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> > marketplace
> >
> >[/color]
>
>[/color]

[Tim Van Wassenhove]

Re: Reading a file in reverse order

On 2004-01-29, Erik Andersson <erik@eneit.com > wrote:[color=blue]
> Hi!
>
> I need to read a file (line-by-line) in reverse order, without reading the
> whole file into memory first.
>
> Is there an easy way of doing this? As in, is there a PHP function I could
> use?[/color]

There are some nice GNU-tools:

With cat you get the contents of a file from the beginning to the end
With tac you get the contents of a file from the end to the beginning ;)


--

http://home.mysth.be/~timvw

[CountScubula]

Re: Reading a file in reverse order

Ah yes, sorry, next time I should read more clearly

--
Mike Bradley
http://www.gzentools.com -- free online php tools
"Erik Andersson" <erik@eneit.com > wrote in message
news:4018b23a$0 $827$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com
..au...[color=blue]
> They're not, hence "without reading the whole file into memory first".
>
> Thanks anyway!
>
> --
> Erik Andersson ( AussieTraders.c om.au )
> http://www.aussietraders.com.au/
> - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> marketplace
>
> "CountScubu la" <me@scantek.hot mail.com> wrote in message
> news:362Sb.1914 4$vG2.15887@new ssvr25.news.pro digy.com...[color=green]
> > if the files are small, load them into an array $aryFile =
> > file("filename. txt");
> >
> > then just reverse the array or read backwords
> >
> >
> > --
> > Mike Bradley
> > http://www.gzentools.com -- free online php tools
> > "Erik Andersson" <erik@eneit.com > wrote in message
> >[/color]
>[/color]
news:40188b55$0 $830$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com[color=blue][color=green]
> > .au...[color=darkred]
> > > Hi!
> > >
> > > I need to read a file (line-by-line) in reverse order, without reading[/color][/color]
> the[color=green][color=darkred]
> > > whole file into memory first.
> > >
> > > Is there an easy way of doing this? As in, is there a PHP function I[/color][/color]
> could[color=green][color=darkred]
> > > use?
> > >
> > > The solution has to be platform independent, so "popen("tac[/color][/color]
> $filename")...[color=green][color=darkred]
> > > wouldn't work.
> > >
> > > Example input and ouput:
> > >
> > > <myfile.txt input>
> > > line 1
> > > line 2
> > > line 3
> > > </myfile.txt input>
> > >
> > > <output from php script>
> > > line 3
> > > line 2
> > > line 1
> > > </output from php script>
> > >
> > >
> > > All the best,
> > >
> > > Erik
> > >
> > > --
> > > Erik Andersson ( AussieTraders.c om.au )
> > > http://www.aussietraders.com.au/
> > > - Placing an add is FREE at AussieTraders.c om.au, Australia's buy &[/color][/color][/color]
sell[color=blue][color=green][color=darkred]
> > > marketplace
> > >
> > >[/color]
> >
> >[/color]
>
>[/color]

[Chris Jenkinson]

Re: Reading a file in reverse order

Tim Van Wassenhove wrote:[color=blue]
>
> With cat you get the contents of a file from the beginning to the end
> With tac you get the contents of a file from the end to the beginning ;)
>[/color]

Those don't exist on Windows though.

Does fopen() read it into memory?

Chris

--
Chris Jenkinson

[Tim Van Wassenhove]

Re: Reading a file in reverse order

On 2004-01-29, Chris Jenkinson <talrias@talria s.net> wrote:[color=blue]
> Tim Van Wassenhove wrote:[color=green]
>>
>> With cat you get the contents of a file from the beginning to the end
>> With tac you get the contents of a file from the end to the beginning ;)
>>[/color]
>
> Those don't exist on Windows though.[/color]

Offcourse they do:

http://unxutils.sourceforge.net/

--

http://home.mysth.be/~timvw

[Chris Jenkinson]

Re: Reading a file in reverse order

Tim Van Wassenhove wrote:
[color=blue]
> On 2004-01-29, Chris Jenkinson <talrias@talria s.net> wrote:
>[color=green]
>>Tim Van Wassenhove wrote:
>>[color=darkred]
>>>With cat you get the contents of a file from the beginning to the end
>>>With tac you get the contents of a file from the end to the beginning ;)
>>>[/color]
>>
>>Those don't exist on Windows though.[/color]
>
>
> Offcourse they do:
>
> http://unxutils.sourceforge.net/
>[/color]

Some people won't be in a position to download them and use them, for
any number of reasons. The idea solution would do this without having to
download extra files.

--
Chris Jenkinson

[Tim Van Wassenhove]

Re: Reading a file in reverse order

On 2004-01-29, Chris Jenkinson <talrias@talria s.net> wrote:[color=blue]
> Some people won't be in a position to download them and use them, for
> any number of reasons. The idea solution would do this without having to
> download extra files.[/color]

With fseek you can do positioning of the file pointer (fp). Just put the
fp at the end of the file. Read the char. Position the pointer at
feof-1, read the char. postion at feof-2, read the char, ...

--

http://home.mysth.be/~timvw

[CountScubula]

Re: Reading a file in reverse order

ok, got it,

open the file, then have a function like revLineRead()

and in this function, set the postion marker to 1k below the end of file,
read in 1k into a string, say $inBuffer,
scan $inBuffer up to the first /n and save this in $nextBuffer, and remove
from $inBuffer.

now $aryLines = explode("\n",$i nBuffer);

ok, to read last line:
$r = ary_pop($aryLin es);

if $r == "" then we read in another 1k and put $nextBuffer at end of
$inBuffer, then do recan for first /n

this should do it.

--
Mike Bradley
http://www.gzentools.com -- free online php tools
"Erik Andersson" <erik@eneit.com > wrote in message
news:40188b55$0 $830$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com
..au...[color=blue]
> Hi!
>
> I need to read a file (line-by-line) in reverse order, without reading the
> whole file into memory first.
>
> Is there an easy way of doing this? As in, is there a PHP function I could
> use?
>
> The solution has to be platform independent, so "popen("tac $filename")...
> wouldn't work.
>
> Example input and ouput:
>
> <myfile.txt input>
> line 1
> line 2
> line 3
> </myfile.txt input>
>
> <output from php script>
> line 3
> line 2
> line 1
> </output from php script>
>
>
> All the best,
>
> Erik
>
> --
> Erik Andersson ( AussieTraders.c om.au )
> http://www.aussietraders.com.au/
> - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> marketplace
>
>[/color]

[Theorem]

Re: Reading a file in reverse order

Sure sounds like a homework assignment to me! Just curious what the
application may be.


"Erik Andersson" <erik@eneit.com > wrote in message
news:40188b55$0 $830$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com.a u...[color=blue]
> Hi!
>
> I need to read a file (line-by-line) in reverse order, without reading the
> whole file into memory first.
>
> Is there an easy way of doing this? As in, is there a PHP function I could
> use?
>
> The solution has to be platform independent, so "popen("tac $filename")...
> wouldn't work.
>
> Example input and ouput:
>
> <myfile.txt input>
> line 1
> line 2
> line 3
> </myfile.txt input>
>
> <output from php script>
> line 3
> line 2
> line 1
> </output from php script>
>
>
> All the best,
>
> Erik
>
> --
> Erik Andersson ( AussieTraders.c om.au )
> http://www.aussietraders.com.au/
> - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> marketplace
>
>[/color]

[Steve Weet]

Re: Reading a file in reverse order

Proposed solution below.

I'm fairly new to PHP and very new to OO so please forgive/Point out any
issues/problems


<?php
/*************** *************** *************** ********
** Title.........: RevFile Class
** Version.......: 1.00
** Author........: Steve Weet <sweet@weet.dem on.co.uk>
** Filename......: class.RevFile.p hp
** Last changed..: 30th Jan 2004
** Purpose.......: Allows the display of a file in
** ..............: In reverse order
*************** *************** *************** *********/

// Example Usage
$file = new RevFile("/etc/passwd");

while ( ! $file->sof() ) {
echo $file->GetLine() ;
}

class RevFile {

var $FileName;
var $FileHandle;
var $FilePos;

function RevFile($filena me) {

$this->FileName = $filename;

$this->FileHandle = @fopen($filenam e, "r") or
die("Could not open file $filename\n");

// Find EOF
if ( ! (fseek($this->FileHandle, 0, SEEK_END ) == 0 ))
die ("Could not find end of file in $filename\n");

// Store file position
$this->FilePos = ftell($this->FileHandle);

// Check that file is not empty or doesn;t contain a single newline
if ($this->FilePos < 2 )
die ("File is empty\n");

// Position file pointer just before final newline
// i.e. Skip EOF
$this->FilePos -= 1;
}
function GetLine() {
$pos = $this->FilePos -1;
$ch=" ";
$line = "";
while ($ch != "\n" && $pos >= 0) {
fseek($this->FileHandle, $pos );
$ch = fgetc($this->FileHandle);

// Decrement out pointer and prepend to the line
// if we have not hit the new line
if ( $ch != "\n" ) {
$pos = $pos -1;
$line = $ch . $line;
}
}
$this->FilePos = $pos ;
return $line . "\n";
}

function sof() {
return ($this->FilePos <= 0 );
}
}
?>


Erik Andersson wrote:[color=blue]
> Hi!
>
> I need to read a file (line-by-line) in reverse order, without reading the
> whole file into memory first.
>
> Is there an easy way of doing this? As in, is there a PHP function I could
> use?
>
> The solution has to be platform independent, so "popen("tac $filename")...
> wouldn't work.
>
> Example input and ouput:
>
> <myfile.txt input>
> line 1
> line 2
> line 3
> </myfile.txt input>
>
> <output from php script>
> line 3
> line 2
> line 1
> </output from php script>
>
>
> All the best,
>
> Erik
>
> --
> Erik Andersson ( AussieTraders.c om.au )
> http://www.aussietraders.com.au/
> - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> marketplace
>
>[/color]

[Erik Andersson]

Re: Reading a file in reverse order

It's not... I'm building a homegrown web log analyser which pulls all new
entries in a log file into the database. The database shouldn't have any
duplicates, hence I need to read from the bottom instead of the top when
parsing the log file. It would just be a waste of time reading a massive log
file from the top as all the new entries are at the bottom.

--
Erik Andersson ( AussieTraders.c om.au )

http://www.aussietraders.com.au/

- Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
marketplace

"Theorem" <t h e o r e m @ a x i o m e t r i c . o r g> wrote in message
news:101ith2am3 26977@news.supe rnews.com...[color=blue]
> Sure sounds like a homework assignment to me! Just curious what the
> application may be.
>
>
> "Erik Andersson" <erik@eneit.com > wrote in message
>[/color]
news:40188b55$0 $830$61c65585@u q-127creek-reader-01.brisbane.pip enetworks.com
..au...[color=blue][color=green]
> > Hi!
> >
> > I need to read a file (line-by-line) in reverse order, without reading[/color][/color]
the[color=blue][color=green]
> > whole file into memory first.
> >
> > Is there an easy way of doing this? As in, is there a PHP function I[/color][/color]
could[color=blue][color=green]
> > use?
> >
> > The solution has to be platform independent, so "popen("tac[/color][/color]
$filename")...[color=blue][color=green]
> > wouldn't work.
> >
> > Example input and ouput:
> >
> > <myfile.txt input>
> > line 1
> > line 2
> > line 3
> > </myfile.txt input>
> >
> > <output from php script>
> > line 3
> > line 2
> > line 1
> > </output from php script>
> >
> >
> > All the best,
> >
> > Erik
> >
> > --
> > Erik Andersson ( AussieTraders.c om.au )
> > http://www.aussietraders.com.au/
> > - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> > marketplace
> >
> >[/color]
>
>[/color]

[Erik Andersson]

Re: Reading a file in reverse order

Works absolutely perfect for what I'm using it for! Thanks, alot!

I'm not sure as for what efficiency `fgetc` has, but for this project speed
is not that important.

All the best,

Erik Andersson ( AussieTraders.c om.au )

http://www.aussietraders.com.au/

- Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
marketplace

"Steve Weet" <sweet@dciltd.c om> wrote in message
news:4019A0CE.8 040100@dciltd.c om...[color=blue]
> Proposed solution below.
>
> I'm fairly new to PHP and very new to OO so please forgive/Point out any
> issues/problems
>
>
> <?php
> /*************** *************** *************** ********
> ** Title.........: RevFile Class
> ** Version.......: 1.00
> ** Author........: Steve Weet <sweet@weet.dem on.co.uk>
> ** Filename......: class.RevFile.p hp
> ** Last changed..: 30th Jan 2004
> ** Purpose.......: Allows the display of a file in
> ** ..............: In reverse order
> *************** *************** *************** *********/
>
> // Example Usage
> $file = new RevFile("/etc/passwd");
>
> while ( ! $file->sof() ) {
> echo $file->GetLine() ;
> }
>
> class RevFile {
>
> var $FileName;
> var $FileHandle;
> var $FilePos;
>
> function RevFile($filena me) {
>
> $this->FileName = $filename;
>
> $this->FileHandle = @fopen($filenam e, "r") or
> die("Could not open file $filename\n");
>
> // Find EOF
> if ( ! (fseek($this->FileHandle, 0, SEEK_END ) == 0 ))
> die ("Could not find end of file in $filename\n");
>
> // Store file position
> $this->FilePos = ftell($this->FileHandle);
>
> // Check that file is not empty or doesn;t contain a single[/color]
newline[color=blue]
> if ($this->FilePos < 2 )
> die ("File is empty\n");
>
> // Position file pointer just before final newline
> // i.e. Skip EOF
> $this->FilePos -= 1;
> }
> function GetLine() {
> $pos = $this->FilePos -1;
> $ch=" ";
> $line = "";
> while ($ch != "\n" && $pos >= 0) {
> fseek($this->FileHandle, $pos );
> $ch = fgetc($this->FileHandle);
>
> // Decrement out pointer and prepend to the line
> // if we have not hit the new line
> if ( $ch != "\n" ) {
> $pos = $pos -1;
> $line = $ch . $line;
> }
> }
> $this->FilePos = $pos ;
> return $line . "\n";
> }
>
> function sof() {
> return ($this->FilePos <= 0 );
> }
> }
> ?>
>
>
> Erik Andersson wrote:[color=green]
> > Hi!
> >
> > I need to read a file (line-by-line) in reverse order, without reading[/color][/color]
the[color=blue][color=green]
> > whole file into memory first.
> >
> > Is there an easy way of doing this? As in, is there a PHP function I[/color][/color]
could[color=blue][color=green]
> > use?
> >
> > The solution has to be platform independent, so "popen("tac[/color][/color]
$filename")...[color=blue][color=green]
> > wouldn't work.
> >
> > Example input and ouput:
> >
> > <myfile.txt input>
> > line 1
> > line 2
> > line 3
> > </myfile.txt input>
> >
> > <output from php script>
> > line 3
> > line 2
> > line 1
> > </output from php script>
> >
> >
> > All the best,
> >
> > Erik
> >
> > --
> > Erik Andersson ( AussieTraders.c om.au )
> > http://www.aussietraders.com.au/
> > - Placing an add is FREE at AussieTraders.c om.au, Australia's buy & sell
> > marketplace
> >
> >[/color]
>[/color]

[-Ema-]

Re: Reading a file in reverse order

Erik Andersson wrote:
[color=blue]
> It's not... I'm building a homegrown web log analyser which pulls all new
> entries in a log file into the database. The database shouldn't have any
> duplicates, hence I need to read from the bottom instead of the top when
> parsing the log file. It would just be a waste of time reading a massive log
> file from the top as all the new entries are at the bottom.
>[/color]
This is a search problem in a sorted collection. You should search the
last processed entry in a collection of ordered records (entries in web
log). I think that you can find some example of find algoritms on the web.
Another way (more performant) can be to store the last processed entry
position, and the next time you should parse the file do a seek at the
old position. You should put some control and handle exception but IMHO
this is the correct way.

Regards.
-Ema-