efficient Python object count

[darrenr]

Hello,

Does anyone know of an efficient way to get a count of the total
number of Python objects in CPython? The best solution I've been able
to find is len(gc.get_obje cts()) which unfortunately has to walk a C
linked list *and* creates a list containing all of the objects, when
all I need is an object count.

Thanks!

[Christian Heimes]

Re: efficient Python object count

darrenr wrote:

Hello,
>
Does anyone know of an efficient way to get a count of the total
number of Python objects in CPython? The best solution I've been able
to find is len(gc.get_obje cts()) which unfortunately has to walk a C
linked list *and* creates a list containing all of the objects, when
all I need is an object count.

There is no way to get correct numbers with a standard Python compiler.
You have to use a special debug build but that requires debug builds of
all extensions too.
gc.get_object() lists only object that are maintained by gc. Several
types don't use gc, for example str, unicode, int, float, long and more.

Christian

[darrenr]

Re: efficient Python object count

On Nov 6, 7:28 pm, Christian Heimes <li...@cheimes. dewrote:

darrenr wrote:

Hello,

>

Does anyone know of an efficient way to get a count of the total
number of Python objects in CPython? The best solution I've been able
to find is len(gc.get_obje cts()) which unfortunately has to walk a C
linked list *and* creates a list containing all of the objects, when
all I need is an object count.

>
There is no way to get correct numbers with a standard Python compiler.
You have to use a special debug build but that requires debug builds of
all extensions too.
gc.get_object() lists only object that are maintained by gc. Several
types don't use gc, for example str, unicode, int, float, long and more.
>
Christian

Thanks for the quick reply. Could you provide a link to more
information on the debug build you refer to?

[darrenr]

Re: efficient Python object count

Thanks for the quick reply. Could you provide a link to more

information on the debug build you refer to?

A modified version of this algorithm should do the trick for my
purposes, it finds the non-containers that gc ignores. I don't care
how long it takes to compute, I just don't want the process to block
for long periods of time.

http://utcc.utoronto.ca/~cks/space/blog/python/GetAllObjects

[robert]

Re: efficient Python object count

darrenr wrote:

>Thanks for the quick reply. Could you provide a link to more
>information on the debug build you refer to?

>
A modified version of this algorithm should do the trick for my
purposes, it finds the non-containers that gc ignores. I don't care
how long it takes to compute, I just don't want the process to block
for long periods of time.
>
http://utcc.utoronto.ca/~cks/space/b.../GetAllObjects

Once I had a strange memory leak in a long running process. This
turned out to be a problem of objects filing up in gc.garbage
("uncollectable s").
I think those objects where also not reachable through
gc.get_objects( ) - as used by the script above. So be aware. Maybe
you've got the same motive for your counter.
I was somewhat stunn seeing, that obviously almost any serious
bigger app needs to watch and handle/free the gc.garbarge list "by
hand", to remain stable.


Robert

[Christian Heimes]

Re: efficient Python object count

darrenr wrote:

Thanks for the quick reply. Could you provide a link to more
information on the debug build you refer to?

Here you are:

http://svn.python.org/projects/python/branches/release25-maint/Misc/SpecialBuilds.txt

[darrenr]

Re: efficient Python object count

Here you are:http://svn.python.org/projects/pytho...5-maint/Misc/S...

Excellent, thank you.