Binary data handling ?

[Bill Loren]

Hello ppl,

I'm having difficulties to accomplish some simple chores with binary data.
I'm having a string (?) which I received via an HTTP transactions which is a
binary file.
Problem is the webserver I'm communicating with injected a \x0D before every
\x0A,
and I need to remove those 0x0D characters from my buffer before saving it
to disk.

any ideas ?

I tried the following without any success:
string.replace( "%c%c" % (13,10), "%c" % (10))
string.replace( "\x0d\x0a", "\0x0a")

thx
~B

[Peter Hansen]

Re: Binary data handling ?

Bill Loren wrote:[color=blue]
>
> Hello ppl,
>
> I'm having difficulties to accomplish some simple chores with binary data.
> I'm having a string (?) which I received via an HTTP transactions which is a
> binary file.
> Problem is the webserver I'm communicating with injected a \x0D before every
> \x0A,
> and I need to remove those 0x0D characters from my buffer before saving it
> to disk.[/color]

This sounds wrong. I don't think a properly configured web server should
be transmitting unencoded binary files with newline conversion.
[color=blue]
> any ideas ?
>
> I tried the following without any success:
> string.replace( "%c%c" % (13,10), "%c" % (10))
> string.replace( "\x0d\x0a", "\0x0a")[/color]

Strings are immutable. Are you expecting the above to change the string
(which doesn't happen) or to return a new string with the changes made?
Assign the result of the replace() call to a new variable and it should
work. (Except in the latter example you should have \x0a, not \0x0a.)

-Peter

[Peter Abel]

Re: Binary data handling ?

"Bill Loren" <lorenb2@bezeqi nt.net> wrote in message news:<mailman.1 062079383.31727 .python-list@python.org >...[color=blue]
> Hello ppl,
>
> I'm having difficulties to accomplish some simple chores with binary data.
> I'm having a string (?) which I received via an HTTP transactions which is a
> binary file.
> Problem is the webserver I'm communicating with injected a \x0D before every
> \x0A,
> and I need to remove those 0x0D characters from my buffer before saving it
> to disk.
>
> any ideas ?
>
> I tried the following without any success:
> string.replace( "%c%c" % (13,10), "%c" % (10))
> string.replace( "\x0d\x0a", "\0x0a")
>
> thx
> ~B[/color]
[color=blue][color=green][color=darkred]
>>> # A short function to generate a string with 0x0D's
>>> def make_bin():[/color][/color][/color]
.... bytes=[]
.... for i in range(20):
.... bytes.append(ch r(i))
.... # every 5th char is a 0x0D
.... if not i%5:
.... bytes.append(ch r(0x0D))
.... return ''.join(bytes)
....[color=blue][color=green][color=darkred]
>>> binary=make_bin ()
>>> binary[/color][/color][/color]
'\x00\r\x01\x02 \x03\x04\x05\r\ x06\x07\x08\t\n \r\x0b\x0c\r\x0 e\x0f\r\x10\x11 \x12\x13'[color=blue][color=green][color=darkred]
>>> # split the string at the 0x0D's and join it again
>>> binary_without_ x0D=''.join(bin ary.split(chr(0 x0D)))
>>> #et voila
>>> binary_without_ x0D[/color][/color][/color]
'\x00\x01\x02\x 03\x04\x05\x06\ x07\x08\t\n\x0b \x0c\x0e\x0f\x1 0\x11\x12\x13'[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

Regards
Peter

[Peter Hansen]

Re: Binary data handling ?

Bill Loren wrote:[color=blue]
>
> about the code problem, I did fix that bug you mentioned, but it still
> doesn't work.
> the code is:
> data = data.replace(.. .the two options I mentioned before...)
> but alas... no replacement...[/color]

Since the following clearly works, you must be confused about what
is in the string called "data" prior to the replacement:

C:\>python22
Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "help", "copyright" , "credits" or "license" for more information.[color=blue][color=green][color=darkred]
>>> s = 'test 1\r\ntest 2\rtest 3\ntest4\r\n\r\ ntest5'
>>> s[/color][/color][/color]
'test 1\r\ntest 2\rtest 3\ntest4\r\n\r\ ntest5'[color=blue][color=green][color=darkred]
>>> t = s.replace('\r\n ', '\n')
>>> u = s.replace('\x0d \x0a', '\x0a')
>>> v = s.replace('%c%c ' % (13, 10), '%c' % (10))
>>> t[/color][/color][/color]
'test 1\ntest 2\rtest 3\ntest4\n\ntes t5'[color=blue][color=green][color=darkred]
>>> u[/color][/color][/color]
'test 1\ntest 2\rtest 3\ntest4\n\ntes t5'[color=blue][color=green][color=darkred]
>>> v[/color][/color][/color]
'test 1\ntest 2\rtest 3\ntest4\n\ntes t5'

(Please post to the newsgroup/mailing list instead of mailing directly,
so that others can benefit from or participate in the discussion.)

-Peter

[Tim Roberts]

Re: Binary data handling ?

"Bill Loren" <lorenb2@bezeqi nt.net> wrote:[color=blue]
>
>I'm having difficulties to accomplish some simple chores with binary data.
>I'm having a string (?) which I received via an HTTP transactions which is a
>binary file.
>Problem is the webserver I'm communicating with injected a \x0D before every
>\x0A,
>and I need to remove those 0x0D characters from my buffer before saving it
>to disk.
>
>any ideas ?[/color]

I'll bet you real money that the problem is not in the web server. I'd
wager that the string is correct when you receive it, but that you are
writing it to file like this:
file('out.txt', 'w').write(down load)

On a Windows system, that'll turn all the LFs into CR-LFs. Use this
instead:
file('out.txt', 'wb').write(dow nload)
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[Bill Loren]

Re: Binary data handling ?

Indeed !

thanks !!!

if I use the 'wb' will it work on a unix system, too ?

~B
----- Original Message -----
From: "Tim Roberts" <timr@probo.com >
Newsgroups: comp.lang.pytho n
To: <python-list@python.org >
Sent: Saturday, August 30, 2003 7:11 AM
Subject: Re: Binary data handling ?

[color=blue]
> "Bill Loren" <lorenb2@bezeqi nt.net> wrote:[color=green]
> >
> >I'm having difficulties to accomplish some simple chores with binary[/color][/color]
data.[color=blue][color=green]
> >I'm having a string (?) which I received via an HTTP transactions which[/color][/color]
is a[color=blue][color=green]
> >binary file.
> >Problem is the webserver I'm communicating with injected a \x0D before[/color][/color]
every[color=blue][color=green]
> >\x0A,
> >and I need to remove those 0x0D characters from my buffer before saving[/color][/color]
it[color=blue][color=green]
> >to disk.
> >
> >any ideas ?[/color]
>
> I'll bet you real money that the problem is not in the web server. I'd
> wager that the string is correct when you receive it, but that you are
> writing it to file like this:
> file('out.txt', 'w').write(down load)
>
> On a Windows system, that'll turn all the LFs into CR-LFs. Use this
> instead:
> file('out.txt', 'wb').write(dow nload)
> --
> - Tim Roberts, timr@probo.com
> Providenza & Boekelheide, Inc.
> --
> http://mail.python.org/mailman/listinfo/python-list[/color]