pickle: maximum recursion depth exceeded

[Simon Burton]

Hi,

I am pickling big graphs of data and running into this problem:

[...]
File "/usr/lib/python2.2/pickle.py", line 225, in save
f(self, object)
File "/usr/lib/python2.2/pickle.py", line 414, in save_list
save(element)
File "/usr/lib/python2.2/pickle.py", line 219, in save
self.save_reduc e(callable, arg_tup, state)
File "/usr/lib/python2.2/pickle.py", line 249, in save_reduce
save(state)
File "/usr/lib/python2.2/pickle.py", line 225, in save
f(self, object)
File "/usr/lib/python2.2/pickle.py", line 447, in save_dict
save(value)
File "/usr/lib/python2.2/pickle.py", line 219, in save
self.save_reduc e(callable, arg_tup, state)
File "/usr/lib/python2.2/pickle.py", line 245, in save_reduce
save(arg_tup)
File "/usr/lib/python2.2/pickle.py", line 225, in save
f(self, object)
File "/usr/lib/python2.2/pickle.py", line 374, in save_tuple
save(element)
File "/usr/lib/python2.2/pickle.py", line 225, in save
f(self, object)
File "/usr/lib/python2.2/pickle.py", line 405, in save_list
write(self.put( memo_len))
RuntimeError: maximum recursion depth exceeded

However, it works when i try the smallest examples and use
sys.setrecursio nlimit(4000)

This seems like a limitation in the pickling code. Yes?

The data is perhaps better off in some kind of DB designed for
massively interconnected objects. Any suggestions? ZODB ?

BTW, the data is path searching info for a game, and takes 1-2Mb of memory.

Thankyou,

Simon Burton.

[Anthony Briggs]

Re: pickle: maximum recursion depth exceeded

At 11:29 AM +1100 3/11/03, Simon Burton wrote:[color=blue]
>Hi,
>
>I am pickling big graphs of data and running into this problem:
>
>[...]
> File "/usr/lib/python2.2/pickle.py", line 225, in save
> f(self, object)
> File "/usr/lib/python2.2/pickle.py", line 414, in save_list
> save(element)[/color]

....
[color=blue]
> File "/usr/lib/python2.2/pickle.py", line 225, in save
> f(self, object)
> File "/usr/lib/python2.2/pickle.py", line 405, in save_list
> write(self.put( memo_len))
>RuntimeError : maximum recursion depth exceeded
>
>However, it works when i try the smallest examples and use
>sys.setrecursi onlimit(4000)
>
>This seems like a limitation in the pickling code. Yes?[/color]

I would suspect that you have a loop in your definitions, eg. A
imports B, and B imports A, particularly since you're trying small
examples, and they're still exceeding the recursion depth.

Hope that helps,

Anthony
--
----------------------------------------------------
HyPEraCtiVE? HeY, WhO aRE YoU cALliNg HypERaCtIve?!
aBRiGgS@wEStNeT .cOm.aU
----------------------------------------------------