Problem reading with bz2.BZ2File(). Bug?

[Clodoaldo Pinto Neto]

When comparing two files which should be equal the last line is
different:

The first file is a bzip2 compressed file and is read with
bz2.BZ2File()
The second file is the same file uncompressed and read with open()

The first file named file.txt.bz2 is uncompressed with:

$ bunzip2 -k file.txt.bz2

To compare I use this script:
############### ############### #
import bz2

f1 = bz2.BZ2File(r'f ile.txt.bz2', 'r')
f2 = open(r'file.txt ', 'r')
lines = 0
while True:
line1 = f1.readline()
line2 = f2.readline()
if line1 == '':
break
lines += 1
if line1 != line2:
print 'line number:', lines
print repr(line1)
print repr(line2)
f1.close()
f2.close()
############### ###############

The offending file is 5.5 MB. Sorry, i could not reproduce this problem
with a smaller file.

http://fahstats.com/img/file.txt.bz2

Regards, Clodoaldo Pinto Neto

[Fredrik Lundh]

Re: Problem reading with bz2.BZ2File(). Bug?

Clodoaldo Pinto Neto wrote:

The offending file is 5.5 MB. Sorry, i could not reproduce this problem
with a smaller file.

but surely you can post the repr() of the last two lines?

</F>

[Clodoaldo Pinto Neto]

Re: Problem reading with bz2.BZ2File(). Bug?


Fredrik Lundh wrote:

Clodoaldo Pinto Neto wrote:
>

The offending file is 5.5 MB. Sorry, i could not reproduce this problem
with a smaller file.

>
but surely you can post the repr() of the last two lines?

This is the output:

$ python bzp.py
line number: 588317
'\x07'
''

Clodoaldo

[Leo Kislov]

Re: Problem reading with bz2.BZ2File(). Bug?

Clodoaldo Pinto Neto wrote:

Fredrik Lundh wrote:

Clodoaldo Pinto Neto wrote:

The offending file is 5.5 MB. Sorry, i could not reproduce this problem
with a smaller file.

but surely you can post the repr() of the last two lines?

>
This is the output:
>
$ python bzp.py
line number: 588317
'\x07'
''

Confirmed on windows with 2.4 and 2.5:

C:\p>\Python24\ python.exe bzp.py
line number: 588317
'\x1e'
''

C:\p>\Python25\ python.exe bzp.py
line number: 588317
'\x1e'
''

Looks like one byte of garbage is appended at the end of file. Please
file a bug report. As a workaround "rU" mode seems to work fine for
this file.

-- Leo

[Clodoaldo Pinto Neto]

Re: Problem reading with bz2.BZ2File(). Bug?

Leo Kislov wrote:

Confirmed on windows with 2.4 and 2.5:
>
C:\p>\Python24\ python.exe bzp.py
line number: 588317
'\x1e'
''
>
C:\p>\Python25\ python.exe bzp.py
line number: 588317
'\x1e'
''
>
Looks like one byte of garbage is appended at the end of file. Please
file a bug report.

Bug number 1597011

Clodoaldo