file read, binary or text mode

[Michael Hoffman]

Re: file read, binary or text mode

Alan G Isaac wrote:
[color=blue]
> "Roel Schroeven" <rschroev_nospa m_ml@fastmail.f m> wrote in message
> news:OjW4d.2559 17$OR1.13371520 @phobos.telenet-ops.be...
>[color=green]
>>It's safe in the sense that everything goes out exactly as it came in.
>>For example, gzip uses binary mode even when compressing text files. The
>>files may be text, but gzip doesn't care about that. It doesn't care
>>about words, sentences and line endings, but it does care about
>>representin g exactly the bytes that are in the file.[/color]
>
> I think the following is the same question from another angle.[/color]

I think you should consider the same answer from this angle. ;)
[color=blue]
> I have an .zip archive of compressed files that
> I want to decompress. Using the zipfile module,
> I tried
> z=zipfile.ZipFi le(local.zip)
> for zname in z.namelist():
> localtxtfile='c :/puthere/'+zname
> f=open(localtxt file,'w')
> f.write(z.read( zname))
> f.close
>
> The original files were all plain text,
> created on an unspecified platform.[/color]

Are you sure the platform is unspecified? You can find out the platform
by doing zipfile.getinfo (zname).create_ system and then *yuck* looking up
the ID number you get against the list in
<http://www.pkware.com/company/standards/appnote/>.
[color=blue]
> The files I decompressed this way contained
> *two successive* carriage returns
> (ASCII 13) at the end of each line.
> If I change 'w' to 'wb' I get only one
> carriage return at the end of each line.
>
> Why is this extra carriage return added?[/color]

I imagine the file in the archive was created on a DOS-type system,
where the line ending is \r\n. That's what you read in. When you write
it out in "w" mode the \n is expanded to \r\n without checking to see if
there is already a \r beforehand. So you get \r\r\n.

Essentially you should consider the archive file to be read in "rb"
mode. Writing in "w" mode instead of "wb" mode will give you extra
carriage returns.

If you want to be able to get "universal newline" input from your
zipfile, consider piping input through this generator and using "w" mode:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286165

Then you should get the correct line ending for a text file without
regard to the current platform or the one where the archive was created.
--
Michael Hoffman

[Tim Roberts]

Re: file read, binary or text mode

"Alan G Isaac" <aisaac@america n.edu> wrote:[color=blue]
>
>I think the following is the same question from another angle.
>I have an .zip archive of compressed files that
>I want to decompress. Using the zipfile module,
>I tried
>z=zipfile.ZipF ile(local.zip)
>for zname in z.namelist():
> localtxtfile='c :/puthere/'+zname
> f=open(localtxt file,'w')
> f.write(z.read( zname))
> f.close
>
>The original files were all plain text,
>created on an unspecified platform.[/color]

Not true. They were in plain text, created on a DOS/Windows platform.
[color=blue]
>The files I decompressed this way contained
>*two successive* carriage returns
>(ASCII 13) at the end of each line.
>If I change 'w' to 'wb' I get only one
>carriage return at the end of each line.
>
>Why is this extra carriage return added?[/color]

Because the original file inside the zip file contained \r\n. z.read
returns you those exact bytes. When you write "\r\n" to a text file in
Windows, the \r is written as \r, and the \n is written as \r\n. This, you
end up with \r\r\n.
[color=blue]
>My original guess was the using 'w' instead
>of 'wb' would be the right action, since the
>platform for the original files is unspecified
>and the original files are known to be plain text.[/color]

No. If you do not know what your buffer contains, you should always use
'wb' so that those contents are not altered.

That's the real lesson: when you write using 'w' or 'wt', the buffer is
changed on the way out. You only want that if you know exactly what you
are writing.
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[Alan G Isaac]

Re: file read, binary or text mode


"Michael Hoffman" <m.h.3.9.1.with out.dots.at.cam .ac.uk@example. com> wrote in
message news:cj57cj$1d3 $1@pegasus.csx. cam.ac.uk...[color=blue]
> I imagine the file in the archive was created on a DOS-type system,
> where the line ending is \r\n. That's what you read in. When you write
> it out in "w" mode the \n is expanded to \r\n without checking to see if
> there is already a \r beforehand. So you get \r\r\n.[/color]

Thanks; that addresses my basic misconception about writing in textmode.
I had thought that writing in textmode produced a platform specific
conversion of the text written, but I now understand that this only affects
how \n is written.
[color=blue]
> If you want to be able to get "universal newline" input from your
> zipfile, consider piping input through this generator and using "w" mode:
> http://aspn.activestate.com/ASPN/Coo.../Recipe/286165[/color]

Very helpful.

Thanks,
Alan Isaac