Analyzing Python GC output - what is a "cell", and what informationis available about it.

[John Nagle]

I'm printing out each entry in "gc.garbage " after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like

<cell at 0x00F7C170: function object at 0x00FDD6B0>

That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?

(I'm looking for a possible memory leak, obviously.)

John Nagle

[Duncan Booth]

Re: Analyzing Python GC output - what is a &quot;cell&quot ;, and what information is available about it.

John Nagle <nagle@animats. comwrote:

I'm printing out each entry in "gc.garbage " after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like
>
><cell at 0x00F7C170: function object at 0x00FDD6B0>
>
That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?
>

Cell objects are created whenever you have a function that references a
variable in an outer scope. e.g.

>>def outer():

x = 42
def inner():
return x
return inner

>>inner = outer()
>>inner.func_cl osure[0]

<cell at 0x00C5D450: int object at 0x009657AC>

>>inner.func_cl osure[0].cell_contents

42


So in your case, cell.cell_conte nts.func_name might help.

[John Nagle]

Re: Analyzing Python GC output - what is a &quot;cell&quot ;, and what informationis available about it.

Duncan Booth wrote:

John Nagle <nagle@animats. comwrote:
>

>I'm printing out each entry in "gc.garbage " after a garbage collection in
>DEBUG_LEAK mode, and I'm seeing many entries like
>>
><cell at 0x00F7C170: function object at 0x00FDD6B0>
>>
>That's the output of "repr". Are "cell" objects created only from
>external C libraries, or can regular Python code generate them? Is there
>any way to find out what the 'function object' is from within Python?
>>

Cell objects are created whenever you have a function that references a
variable in an outer scope. e.g.
>
So in your case, cell.cell_conte nts.func_name might help.

Tried that:

print repr(item).enco de('ascii','rep lace')
print "Type:",type(it em)
try:
print item.cell_conte nts
except Exception, message:
print "Unprintable:", message

<cell at 0x00F88DF0: function object at 0x0100CFB0>
Type: <type 'cell'>
Unprintable: 'cell' object has no attribute 'cell_contents'

So it doesn't have a "cell_conte nts" attribute.

Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'

Tried:
print item.__dict__
got:
'cell' object has no attribute '__dict__'

It looks like this is a low-level PyCellObject not generated from Python code.
Any ideas? I'm using the M2Crypto and MySQLdb libraries, and suspect
a reference count bug in one of those.

John Nagle

[Francesco Guerrieri]

Re: Analyzing Python GC output - what is a &quot;cell&quot ;,and what information is available about it.

On Jan 11, 2008 6:20 PM, John Nagle <nagle@animats. comwrote:

Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'

I don't know nothing about cell objects...
but why don't you try dir(item) instead?

Francesco

[John Nagle]

Re: Analyzing Python GC output - turns out to be MySQLdb problem

Francesco Guerrieri wrote:

On Jan 11, 2008 6:20 PM, John Nagle <nagle@animats. comwrote:

>Tried:
> print item.dir()
>got:
> 'cell' object has no attribute 'dir'

It's a problem inside MySQLdb's "connections.py ":

def _get_unicode_li teral():
def unicode_literal (u, dummy=None):
return db.literal(u.en code(unicode_li teral.charset))
return unicode_literal

Each time a MySQLdb Connection object is created, it generates a closure,
with another instance of the function "unicode_litera l" plus some other
junk. That generates circular references. Eventually those
things get garbage-collected, but if you're running GC in leak detection
mode, they show up.

The reason for the closure is that "unicode_litera l.charset" of the
function is being stored into from outside the function. So there
actually is a reason to use a closure.

John Nagle