sending binary data over sockets

[thorley@gmail.com]

Greetings, since there was no reponse to my previous post about an
existing FastCGI server in python, I've taken to writing my own. (which
of course I'll share--*if* there's something to share ;)

My problem now, is that I need to send certain binary data over a
socket. That is, I want to make some bytes, and stuff them in a TCP
packet, send them down the pipe, and then listen for a response.

socket.send, as best I can tell, will only send strings. I've read on
the list other conversations where the recommendation was to use xdrlib
or struct. But it appears that is only useful when you control the
client and the server. PLEASE correct me if I'm wrong, but using struct
just encodes the binary data as a string, right?

# Example sending binary data as a string
s = socket.socket()
s.connect(("127 .0.0.1", 8000))
packet = struct.pack('4B ', 1, 2, 3, 4)
s.send(packet)

In my understaing the above just sends the string '\x01\x02\x03\x 04',
not raw binary data. I've looked at Billy the Kid and pcap, which are
cool but not what I need.

Do I have to build my own packets from scratch and encode all the
TCP/IP headers myself to get this to work?

Solutions and Corrections welcome and appreciated.
Thanks very much

--
matthew

[Diez B. Roggisch]

Re: sending binary data over sockets

thorley@gmail.c om schrieb:

Greetings, since there was no reponse to my previous post about an
existing FastCGI server in python, I've taken to writing my own. (which
of course I'll share--*if* there's something to share ;)
>
My problem now, is that I need to send certain binary data over a
socket. That is, I want to make some bytes, and stuff them in a TCP
packet, send them down the pipe, and then listen for a response.
>
socket.send, as best I can tell, will only send strings. I've read on
the list other conversations where the recommendation was to use xdrlib
or struct. But it appears that is only useful when you control the
client and the server. PLEASE correct me if I'm wrong, but using struct
just encodes the binary data as a string, right?

WHich is perfectly alright, as

[Diez B. Roggisch]

Re: sending binary data over sockets

Premature sending syndrome...

Diez B. Roggisch schrieb:

thorley@gmail.c om schrieb:

>Greetings, since there was no reponse to my previous post about an
>existing FastCGI server in python, I've taken to writing my own. (which
>of course I'll share--*if* there's something to share ;)
>>
>My problem now, is that I need to send certain binary data over a
>socket. That is, I want to make some bytes, and stuff them in a TCP
>packet, send them down the pipe, and then listen for a response.
>>
>socket.send, as best I can tell, will only send strings. I've read on
>the list other conversations where the recommendation was to use xdrlib
>or struct. But it appears that is only useful when you control the
>client and the server. PLEASE correct me if I'm wrong, but using struct
>just encodes the binary data as a string, right?

Python strings are binary data and can contain
- in oppostion to e.g. C-strings - null-bytes.

So it is perfectly alright to send "only" strings over a socket.

Diez

[Sukanta]

Re: sending binary data over sockets

thorley@gmail.c om wrote:

Greetings, since there was no reponse to my previous post about an
existing FastCGI server in python, I've taken to writing my own. (which
of course I'll share--*if* there's something to share ;)
>
My problem now, is that I need to send certain binary data over a
socket. That is, I want to make some bytes, and stuff them in a TCP
packet, send them down the pipe, and then listen for a response.
>
socket.send, as best I can tell, will only send strings. I've read on
the list other conversations where the recommendation was to use xdrlib
or struct. But it appears that is only useful when you control the
client and the server. PLEASE correct me if I'm wrong, but using struct
just encodes the binary data as a string, right?
>
# Example sending binary data as a string
s = socket.socket()
s.connect(("127 .0.0.1", 8000))
packet = struct.pack('4B ', 1, 2, 3, 4)
s.send(packet)
>
In my understaing the above just sends the string '\x01\x02\x03\x 04',
not raw binary data. I've looked at Billy the Kid and pcap, which are
cool but not what I need.
>

It will send the 4 bytes, binary, and not the string as you assumed. If
you want to satisfy yourself, run tcpdump (or ethereal) to observe what
is being sent.

Do I have to build my own packets from scratch and encode all the
TCP/IP headers myself to get this to work?
>
Solutions and Corrections welcome and appreciated.
Thanks very much
>
--
matthew

[Grant Edwards]

Re: sending binary data over sockets

On 2006-07-03, thorley@gmail.c om <thorley@gmail. comwrote:

My problem now, is that I need to send certain binary data over a
socket. That is, I want to make some bytes, and stuff them in a TCP
packet, send them down the pipe, and then listen for a response.
>
socket.send, as best I can tell, will only send strings. I've read on
the list other conversations where the recommendation was to use xdrlib
or struct. But it appears that is only useful when you control the
client and the server. PLEASE correct me if I'm wrong, but using struct
just encodes the binary data as a string, right?

Right. Strings are binary data.

# Example sending binary data as a string
s = socket.socket()
s.connect(("127 .0.0.1", 8000))
packet = struct.pack('4B ', 1, 2, 3, 4)
s.send(packet)
>
In my understaing the above just sends the string '\x01\x02\x03\x 04',
not raw binary data.

The string '\x01\x02\x03\x 04' consists of four octets having
the values 0x01, 0x02, 0x03, 0x04.

Are those not the four octets you wanted to send?

--
Grant Edwards grante Yow! I want EARS! I
at want two ROUND BLACK
visi.com EARS to make me feel warm
'n secure!!

[thorley@gmail.com]

Re: sending binary data over sockets

Python strings are binary data and can contain

- in oppostion to e.g. C-strings - null-bytes.
>
So it is perfectly alright to send "only" strings over a socket.

Agreed. I wasn't trying to imply it was 'wrong' just that the receiving
application wouldn't interpret strings correctly.

[thorley@gmail.com]

Re: sending binary data over sockets

It will send the 4 bytes, binary, and not the string as you assumed. If

you want to satisfy yourself, run tcpdump (or ethereal) to observe what
is being sent.

Thanks very much for the prompt reply. I'll take your word for it. I
actually tried ethereal to verify my hypothesis before posting, but
wasn't able to see the raw data bytes. That's probably just a
deficiancy in my ability to steer ethereal.

thanks again
--
matthew

[thorley@gmail.com]

Re: sending binary data over sockets

Are those not the four octets you wanted to send?

Yes. My understanding of struct was broken. Sukanta corrected it for
me.

thank you
--
matthew

[Scott David Daniels]

Re: sending binary data over sockets

thorley@gmail.c om wrote:

Greetings, since there was no reponse to my previous post about an
existing FastCGI server in python, I've taken to writing my own. (which
of course I'll share--*if* there's something to share ;)
>
My problem now, is that I need to send certain binary data over a
socket. That is, I want to make some bytes, and stuff them in a TCP
packet, send them down the pipe, and then listen for a response.

As everyone else has already told you, you have a misconception about
strings and struct and so on. However, if what you want is mutability
in a pattern, read the array module documentation as well.

--Scott David Daniels
scott.daniels@a cm.org

[Simon Forman]

Re: sending binary data over sockets

Grant Edwards wrote:

On 2006-07-03, thorley@gmail.c om <thorley@gmail. comwrote:
>

My problem now, is that I need to send certain binary data over a
socket. That is, I want to make some bytes, and stuff them in a TCP
packet, send them down the pipe, and then listen for a response.

....

In my understaing the above just sends the string '\x01\x02\x03\x 04',
not raw binary data.

>
The string '\x01\x02\x03\x 04' consists of four octets having
the values 0x01, 0x02, 0x03, 0x04.
>
Are those not the four octets you wanted to send?
>
--
Grant Edwards

i.e.

>>len('\x01\x02 \x03\x04')

4