XML vs. cPickle

[Mike]

I know XML is more (processor) costly than cPickle, but how bad is it?
The idea is I want to store data that can be described as XML into my
database as cPickle objects. Except my web framework has no support for
BLOB datatype yet, and I might have to go with XML.

Ideas are appreciated,

Thanks,
Mike

[Alan Kennedy]

Re: XML vs. cPickle

[Mike][color=blue]
> I know XML is more (processor) costly than cPickle, but how bad is it?[/color]

Are you sure you know that?

I'd guess that XML serialisation with cElementTree is both cpu and
memory competitive with cpickle, if not superior. Although I'm too lazy
to fire up the timeit module right now :-)

Also, how quickly the relevant parsers work depends on the input, i.e.
your data structures. Only you can take measurements with your data
structures ....
[color=blue]
> The idea is I want to store data that can be described as XML[/color]

can != should
[color=blue]
> into my
> database as cPickle objects. Except my web framework has no support for
> BLOB datatype yet, and I might have to go with XML.[/color]

Or you could encode the binary pickle in a text-safe encoding such as
base64, and store the result in a text column. Although that will
obviously increase your processing time, both going in and out of the
database.
[color=blue]
> Ideas are appreciated,[/color]

I'd write a few simple prototypes and take some empirical measurements.

HTH,

--
alan kennedy
------------------------------------------------------
email alan: http://xhaus.com/contact/alan

[Mike]

Re: XML vs. cPickle

> I'd guess that XML serialisation with cElementTree is both cpu and[color=blue]
> memory competitive with cpickle, if not superior. Although I'm too lazy
> to fire up the timeit module right now :-)[/color]

That maybe true, but I bet Marshal is the fastest. ...right?
[color=blue]
> Also, how quickly the relevant parsers work depends on the input, i.e.
> your data structures. Only you can take measurements with your data
> structures ....[/color]

True.
[color=blue][color=green]
> > The idea is I want to store data that can be described as XML[/color]
> can != should[/color]

I certainly 'can', I don't think I should.
[color=blue][color=green]
> > into my
> > database as cPickle objects. Except my web framework has no support for
> > BLOB datatype yet, and I might have to go with XML.[/color][/color]
[color=blue]
> Or you could encode the binary pickle in a text-safe encoding such as
> base64, and store the result in a text column. Although that will
> obviously increase your processing time, both going in and out of the
> database.[/color]

base64... (used to convert arbitrary binary data to plain text), sounds
fantastic. Except, I don't think I need it when marshaling anymore
since marshaling gives you clear text anyways. (right?)
[color=blue][color=green]
> > Ideas are appreciated,[/color][/color]
[color=blue]
> I'd write a few simple prototypes and take some empirical measurements.[/color]

I am doing it now. Thanks,

Mike