Hank @ITGroup wrote:
It doesn't really make that much sense to watch memory usage as you have
been doing. Your first test case appears to trigger a specific
pathology, where the memory allocator actually returns the memory to the
operating system when the garbage collector manages to free all of it.
Most often this doesn't happen - a chunk of memory might be 99.99% free
but still have one small piece used, and so while there is a large
amount of "free" memory for Python to allocate without requesting more
process memory, this won't be reflected in any external measurement.
You are suffering from a pathological condition yourself: the desire to
optimize performance in an area where you do not have any problems. I
would suggest you just enjoy using Python (its memory management doesn't
suck at all, so your title line was inflammatory and simply highlights
your lack of knowledge) and then start to ask these questions again when
you have a real issue that's stopping you from getting real work done.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Apology for the previous offensive title~~
:)
Thanks, Rintsch, Arnaud and Daniel, for replying so soon.
>
I redid the experiment. What following is the record -
>
``starting python`` # == Windows Task Manager:
Python.exe *4,076 *K memory-usage ==
>
>
>
... fd3 = FreqDist()
... for i in range(999999):f d3.inc(i)
>
>
... fd4 = FreqDist()
... for i in range(999999):f d4.inc(i)
... del fd4
# == 28,788 K ==
>
>
That is my question, after ``del``, sometimes the memory space returns
back as nothing happened, sometimes not... ...
What exactly was happening???
>
Best regards to all PYTHON people ~~
!!! Python Team are great !!!
>
:)
Thanks, Rintsch, Arnaud and Daniel, for replying so soon.
>
I redid the experiment. What following is the record -
>
``starting python`` # == Windows Task Manager:
Python.exe *4,076 *K memory-usage ==
>>st1='abcdefg' *999999 # == 10,952 K ==
>>del st1 # == *4,104*K ==
>>st1='abcdefg' *999999 # == 10,952 K ==
>>del st1 # == 4,104 K ==
>>del st1 # == *4,104*K ==
>>st1='abcdefg' *999999 # == 10,952 K ==
>>del st1 # == 4,104 K ==
>>li = ['abcde']*999999 # == 8,024 K ==
>>del li # == *4,108* K ==
>>del li # == *4,108* K ==
>>from nltk import FreqDist # == 17,596 ==
>>fd = FreqDist() # == 17,596 ==
>>for i in range(999999):f d.inc(i) # == 53,412 ==
>>del fd # == *28,780* ==
>>fd2 = FreqDist() # == 28,780 ==
>>for i in range(999999):f d2.inc(i) # == 53,412 ==
>>del fd2 # == 28,780 K ==
>>fd = FreqDist() # == 17,596 ==
>>for i in range(999999):f d.inc(i) # == 53,412 ==
>>del fd # == *28,780* ==
>>fd2 = FreqDist() # == 28,780 ==
>>for i in range(999999):f d2.inc(i) # == 53,412 ==
>>del fd2 # == 28,780 K ==
>>def foo():
... for i in range(999999):f d3.inc(i)
>
>> foo() # == *28,788* K ==
>>def bar():
... for i in range(999999):f d4.inc(i)
... del fd4
# == 28,788 K ==
>>bar() # == 28,788 K ==
>
That is my question, after ``del``, sometimes the memory space returns
back as nothing happened, sometimes not... ...
What exactly was happening???
>
Best regards to all PYTHON people ~~
!!! Python Team are great !!!
>
been doing. Your first test case appears to trigger a specific
pathology, where the memory allocator actually returns the memory to the
operating system when the garbage collector manages to free all of it.
Most often this doesn't happen - a chunk of memory might be 99.99% free
but still have one small piece used, and so while there is a large
amount of "free" memory for Python to allocate without requesting more
process memory, this won't be reflected in any external measurement.
You are suffering from a pathological condition yourself: the desire to
optimize performance in an area where you do not have any problems. I
would suggest you just enjoy using Python (its memory management doesn't
suck at all, so your title line was inflammatory and simply highlights
your lack of knowledge) and then start to ask these questions again when
you have a real issue that's stopping you from getting real work done.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/