Memory leak when creating lots of object

[Godzilla]

Hello,

I have a program that create and pop an object off a queue, but it is
experiencing some memory leakage. I have been unable to detect where
the memory leakage occur. The strange thing is when i replace the
object creation with a plain integer/string, the leak goes away...
Here's the code I used as my test:


import Queue
import threading
import time

q = Queue.Queue(0)

class PDU:
def __init__(self, name=''):
self.name = name

class qTest(threading .Thread):
def __init__(self, name=''):
threading.Threa d.__init__(self )
self.name = name
self.threadCont inue = True

def run(self):
count = 0
while self.threadCont inue:
pdu = q.get(True)
del pdu
count+=1
#print count
if q.qsize() == 0:
print "Total get in bulk", count
count = 0

def stop(self):
self.threadCont inue = False
q.put(1)


if __name__ == "__main__":
try:
qt = qTest()
qt.start()
loopCount = 1000
while True:
for i in range(loopCount ):
pdu = PDU()
## pdu = 1
q.put(pdu)
time.sleep(5)

except KeyboardInterru pt:
print "Keyboard interrupted. Program exit."
except Exception, why:
print why

if qt.isAlive():
qt.stop()

I can see the memory usage increases slowly in Task Manager under XP,
but do not know why. Anyone help?

Thank you.

[Paul Moore]

Re: Memory leak when creating lots of object

On 14 Aug, 05:57, Godzilla <godzillais...@ gmail.comwrote:

Hello,
>
I have a program that create and pop an object off a queue, but it is
experiencing some memory leakage. I have been unable to detect where
the memory leakage occur. The strange thing is when i replace the
object creation with a plain integer/string, the leak goes away...
Here's the code I used as my test:

[...]

I can see the memory usage increases slowly in Task Manager under XP,
but do not know why. Anyone help?

I tried your code on my (Windows XP SP2, Python 2.5) system. No memory
leak here - I left it running for over 5 minutes and memory usage was
static at just under 4MB.

Do you see memory growth with precisely this code? Over what period?
How much?

Paul.

[Terry Reedy]

Re: Memory leak when creating lots of object


"Godzilla" <godzillaismad@ gmail.comwrote in message
news:1187322688 .545415.122010@ i38g2000prf.goo glegroups.com.. .
| What should I do next? Can I force garbage collection manually? If so,
| how do I do that?

Read doc for gc module. I think
import gc
gc.collect()

[Paul Moore]

Re: Memory leak when creating lots of object

On 17 Aug, 04:51, Godzilla <godzillais...@ gmail.comwrote:

On Aug 16, 1:13 am, Paul Moore<p.f.mo... @gmail.comwrote :

On 14 Aug, 05:57, Godzilla <godzillais...@ gmail.comwrote:
Do you see memory growth with precisely this code? Over what period?
How much?

>
I have it running for more than 1 hour... the main application
software runs for about 50 days non stops and the memory just keep
growing...

I'm sorry. Just to be precise, how long would I need to run the code
you posted to see a memory growth?

What should I do next? Can I force garbage collection manually? If so,
how do I do that?

As Terry suggested, look at the gc module (gc.collect). But on
inspection, I don't see a memory leak that would be cured by forcing a
collection, which is why I'd like to reproduce the problem before
offering suggestions...

Paul.