serializing data structures

[Philippe C. Martin]

Hi,

I am trying to figure out the traps/issues about sending data structures
through a TCP/IP socket under the following circumstances:

1) there could be a different O/S on both side of the socket
2) there could be a difference CPU architecure on both side of the
socket
3) there could be a different major release of python (ex 2.3 and 2.4)
on both side of the socket.


I once wrote something in C to do that, but since python usually has a
solution for me ....


Any clue ?

Regards,


Philippe





--
*************** ************
Philippe C. Martin
SnakeCard LLC

https://www.snakecard.com

*************** ************

[Martin v. Löwis]

Re: serializing data structures

Philippe C. Martin wrote:[color=blue]
> I once wrote something in C to do that, but since python usually has a
> solution for me ....[/color]

If you use arbitrary data structures, you could use pickle or xmlrpclib
to marshal the data. If you know the data is restricted to a few
well-known data types, you could use XDR (Demo/rpc/xdr.py) or CORBA
(e.g. through fnorb.sf.net) to come up with a wire format.

Otherwise, if you want to define your own proprietary data format,
you can probably use the struct module to generate messages according
to this format.

However, in any case, you need to define the wire format. Once you made
a choice, Python can help implement it - but Python cannot chose a wire
format for you (although pickle comes really close, as does Dopy -
http://www.mindhog.net/~mmuller/projects/dopy/)

Regards,
Martin

[Philippe C. Martin]

Re: serializing data structures

Thanks a lot.

Regards,

Philippe




On Tue, 01 Feb 2005 00:20:01 +0100, Martin v. Löwis wrote:
[color=blue]
> Philippe C. Martin wrote:[color=green]
>> I once wrote something in C to do that, but since python usually has a
>> solution for me ....[/color]
>
> If you use arbitrary data structures, you could use pickle or xmlrpclib
> to marshal the data. If you know the data is restricted to a few
> well-known data types, you could use XDR (Demo/rpc/xdr.py) or CORBA
> (e.g. through fnorb.sf.net) to come up with a wire format.
>
> Otherwise, if you want to define your own proprietary data format,
> you can probably use the struct module to generate messages according
> to this format.
>
> However, in any case, you need to define the wire format. Once you made
> a choice, Python can help implement it - but Python cannot chose a wire
> format for you (although pickle comes really close, as does Dopy -
> http://www.mindhog.net/~mmuller/projects/dopy/)
>
> Regards,
> Martin[/color]

[Paul Rubin]

Re: serializing data structures

"Philippe C. Martin" <philippecmarti n@sbcglobal.net > writes:[color=blue]
> I am trying to figure out the traps/issues about sending data structures
> through a TCP/IP socket under the following circumstances:
>
> 1) there could be a different O/S on both side of the socket
> 2) there could be a difference CPU architecure on both side of the
> socket
> 3) there could be a different major release of python (ex 2.3 and 2.4)
> on both side of the socket.
>
> I once wrote something in C to do that, but since python usually has a
> solution for me ....[/color]

What do you mean by data structures? If they're limited to basic
objects like strings and ints, try xdrlib. If you want longs, class
instances, etc., there's really no good way in the Python stdlib that
doesn't create security vulnerabilities if the source of data isn't
trusted. See SF bugs 471893 and 467384 for some discussion.