SimpleJson is slow .... is there any C Compiled version ?

[sanket]

Hello All,

I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.

Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.

Thank you,
Sanket

[Matt Nordhoff]

Re: SimpleJson is slow .... is there any C Compiled version ?

sanket wrote:

Hello All,
>
I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.
>
Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.
>
Thank you,
Sanket

simplejson is not the only JSON library out there. For example, there's
python-cjson, which is written entirely in C:

<http://pypi.python.org/pypi/python-cjson>

There's also an enhanced version of it:

<http://python.cx.hu/python-cjson/>

I think simplejson has some small, optional C bits that will improve
performance if you compile them.

Also, be aware that I think simplejson is being integrated into the
stdlib in Python 2.6.

Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
--

[sanket]

Re: SimpleJson is slow .... is there any C Compiled version ?

On Jul 25, 5:52 pm, Matt Nordhoff <mnordh...@matt nordhoff.comwro te:

sanket wrote:

Hello All,

>

I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.

>

Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.

>

Thank you,
Sanket

>
simplejson is not the only JSON library out there. For example, there's
python-cjson, which is written entirely in C:
>
<http://pypi.python.org/pypi/python-cjson>
>
There's also an enhanced version of it:
>
<http://python.cx.hu/python-cjson/>
>
I think simplejson has some small, optional C bits that will improve
performance if you compile them.
>
Also, be aware that I think simplejson is being integrated into the
stdlib in Python 2.6.
>
Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
--

Thanks Matt.
I used cjson and it appears to be 5-6 times faster.

Thank you,
Sanket

[Richard Levasseur]

Re: SimpleJson is slow .... is there any C Compiled version ?

On Jul 25, 5:52 pm, Matt Nordhoff <mnordh...@matt nordhoff.comwro te:

Also, simplejson and python-cjson might not be entirely compatible:
there's one character that one escapes and the other doesn't, or something.
--

They also have different interface. simplejson uses load/loads/dump/
dumps, whereas cjson using encode/decode (its pretty simple to
monkeypatch cjson with some lambda's so it has the same interface,
though).

Yeah, its frustrating that there are so many json encoders for
python. Its a bit disappointing simplejson is going into the stdlib;
i hope they rewrite it in C?

http://blog.hill-street.net/?p=7

Has a good comparison of encoders.

[Dan Stromberg]

Re: SimpleJson is slow .... is there any C Compiled version ?

On Sat, 26 Jul 2008 00:49:20 -0700, Richard Levasseur wrote:

On Jul 25, 5:52 pm, Matt Nordhoff <mnordh...@matt nordhoff.comwro te:

>Also, simplejson and python-cjson might not be entirely compatible:
>there's one character that one escapes and the other doesn't, or
>something. --

>
They also have different interface. simplejson uses load/loads/dump/
dumps, whereas cjson using encode/decode (its pretty simple to
monkeypatch cjson with some lambda's so it has the same interface,
though).
>
Yeah, its frustrating that there are so many json encoders for python.
Its a bit disappointing simplejson is going into the stdlib; i hope they
rewrite it in C?

C's great for performance, but if you want something featureful and
reliable, you're probably better off in something else - including but
not limited to python, optionally with psyco.

I wish the pypy people would package up their stuff into a form that you
can just ./configure && make && make install and then #! to - I believe
it'd grow/mature faster than it is if they did (and they may have, but
last time I looked it didn't appear so). But once they do, that might be
a convenient way of getting better performance.

[Joshua Kugler]

Re: SimpleJson is slow .... is there any C Compiled version ?

sanket wrote:

Hello All,
>
I have created an API which fetches some data from the database.
I am using simplejson to encode it and return it back.
>
Now the problem is that, this API is being called for millions of
times in a sequence.
I ran a profiler and saw that most of the time is consumed in encoding
my database results in to json.
So I was just wondering is there any C compiled version of simplejson
is available?
or any help regarding this issue would be appreciated.

simplejson does have a C module. It is compiled automatically when
installed. If you're installing on Windows, then it probably isn't getting
compiled.

j