Memory visualization

[bearophileHUGS@lycos.com]

Icon is a language that share some similarities with Python:
http://www.cs.arizona.edu/icon/

During the execution of a Icon script there are ways to visualize the
memory:
http://www.cs.arizona.edu/icon/progv...mon/memmon.htm

Related pages:
http://www.cs.arizona.edu/icon/progv...p/workshop.htm
http://www.cs.arizona.edu/icon/progvis/progvis.htm

I'd like to know if for Python there is a similar program to
dynamically see the memory in a similar way.
If such tool doesn't exist yet, I'd like to know if it may be diffifult
to create it.

Bye,
bearophile

[Raymond Hettinger]

Re: Memory visualization

[bearophile][color=blue]
> During the execution of a Icon script there are ways to visualize the
> memory:
> http://www.cs.arizona.edu/icon/progv...mon/memmon.htm[/color]
. . .[color=blue]
> I'd like to know if for Python there is a similar program to
> dynamically see the memory in a similar way.
> If such tool doesn't exist yet, I'd like to know if it may be diffifult
> to create it.[/color]

Yes, it would be difficult.

Python gets chunks of memory from the o/s -- those may not be
contiguous. Python has some internal object reuse tricks such that a
chuck of memory may be tied-up in a free-list though no active object
is alive. Extensions can call their own malloc and free so it would be
difficult to ensure that you catch those. Some objects like lists and
dicts have chunks of partially used memory that allow room for growth.

That being said, there is likely some interesting things that could be
done in C by exploiting tp_traverse and by hacking Python's internal
object allocator.


Raymond

[Tim Peters]

Re: Memory visualization

[bearophileHUGS@ lycos.com][color=blue]
> Icon is a language that share some similarities with Python:
> http://www.cs.arizona.edu/icon/
>
> During the execution of a Icon script there are ways to visualize the
> memory:
> http://www.cs.arizona.edu/icon/progv...mon/memmon.htm
>
> Related pages:
> http://www.cs.arizona.edu/icon/progv...p/workshop.htm
> http://www.cs.arizona.edu/icon/progvis/progvis.htm
>
> I'd like to know if for Python there is a similar program to
> dynamically see the memory in a similar way.
> If such tool doesn't exist yet, I'd like to know if it may be diffifult
> to create it.[/color]

Long-time Icon developer Clinton Jeffery gave a presentation on this
topic at PyCon 2005:

Profiling and Visualizing Python Program Behavior
<http://www.python.org/pycon/2005/papers/49/PyCon2005Profil ingandVisualizi ng.pdf>

It was a delight for me to meet him there, since I was an Icon
programmer before Python existed ;-) He had some impressive
preliminary CPython visualization results, which required (a) a lot of
work, and (b) some modest changes to the core interpreter. Alas, I
haven't heard more about it since then. If I were you, I'd ask Dr.
Jeffery what became of that project.

[Stephen Kellett]

Re: Memory visualization

In message <mailman.3224.1 142402912.27775 .python-list@python.org >, Tim
Peters <tim.peters@gma il.com> writes[color=blue]
>[bearophileHUGS@ lycos.com][color=green]
>> I'd like to know if for Python there is a similar program to
>> dynamically see the memory in a similar way.
>> If such tool doesn't exist yet, I'd like to know if it may be diffifult
>> to create it.[/color][/color]

One already exists: Python Memory Validator.

Software Verification provide software tools for Python on the Windows
platform. Software Verification have two commercial products for
coverage and performance profiling and three beta products for flow
tracing, memory analysis and thread deadlock monitoring.

http://www.softwareverify.com

Software Verification run with Python 2.2 upwards - no need to modify
the python source or binary.

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting