Socket being garbage collected too early

[Scott Robinson]

I have been having trouble with the garbage collector and sockets.
Unfortunately, google keeps telling me that the problem is the garbage
collector ignoring dead (closed?) sockets instead of removing live
ones. My problem is


x.sock=socket.s ocket(socket.AF _INET,socket.SO CK_STREAM)
do_stuff(x.sock )


def do_stuff(sock):
sock_list.appen d(sock)


once do_stuff finishes, x.sock disappears, and I can only believe it
is being garbage collected. I'd like to hear the standard means for
avoiding this issue (gc appears to have only the interface to declare
something garbage, not to declare something not garbage).

Scott Robinson

[Steve Holden]

Re: Socket being garbage collected too early

Scott Robinson wrote:
[color=blue]
> I have been having trouble with the garbage collector and sockets.
> Unfortunately, google keeps telling me that the problem is the garbage
> collector ignoring dead (closed?) sockets instead of removing live
> ones. My problem is
>
>
> x.sock=socket.s ocket(socket.AF _INET,socket.SO CK_STREAM)
> do_stuff(x.sock )
>
>
> def do_stuff(sock):
> sock_list.appen d(sock)
>
>
> once do_stuff finishes, x.sock disappears, and I can only believe it
> is being garbage collected. I'd like to hear the standard means for
> avoiding this issue (gc appears to have only the interface to declare
> something garbage, not to declare something not garbage).
>
> Scott Robinson
>[/color]
Unfortunately, assuming it's being garbage collected might turn out to
be incorrect. What evidence do you have that the socket "disappears "? Do
you get a segmentation ault, or what?

If the socket simply fails to work that would be a different case
altogether, but it seems to me that we need a bit more evodence that the
anecdotal stuff you've provided so far.

Quite apart from anything else, by the way, the code you posted appears
to use a global sock_list. A reference b y that list would in any case
stop the socket from being garbage collected (quite apart from the fact
that the socket module itself will do so as long as the socket is open).

So, could we see an error message, or some other evidence of what is
going on. For example, after the call to do_stuff(), what do you see if you

print sock_list

for example. I think your initial hypothesis is insufficient.

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC +1 703 861 4237 +1 800 494 3119

[Mike Meyer]

Re: Socket being garbage collected too early

Scott Robinson <dscottr@bellat lantic.net> writes:
[color=blue]
> I have been having trouble with the garbage collector and sockets.
> Unfortunately, google keeps telling me that the problem is the garbage
> collector ignoring dead (closed?) sockets instead of removing live
> ones. My problem is
>
>
> x.sock=socket.s ocket(socket.AF _INET,socket.SO CK_STREAM)
> do_stuff(x.sock )
>
>
> def do_stuff(sock):
> sock_list.appen d(sock)
>
>
> once do_stuff finishes, x.sock disappears, and I can only believe it
> is being garbage collected. I'd like to hear the standard means for
> avoiding this issue (gc appears to have only the interface to declare
> something garbage, not to declare something not garbage).[/color]

The code as shown doesn't work:
[color=blue][color=green]
>> import socket
>> def do_stuff(sock):[/color][/color]
.. sock_list.appen d(sock)
..[color=blue][color=green]
>> sock = socket.socket(s ocket.AF_INET,s ocket.SOCK_STRE AM)
>> do_stuff(sock)[/color][/color]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in do_stuff
NameError: global name 'sock_list' is not defined[color=blue][color=green]
>>[/color][/color]

If you add "sock_list = []" just before the def of do_stuff, the code
will work, and your sockets won't get garbage collected.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[David Bolen]

Re: Socket being garbage collected too early

Scott Robinson <dscottr@bellat lantic.net> writes:
[color=blue]
> I have been having trouble with the garbage collector and sockets.[/color]

Are you actually getting errors or is this just theoretical?
[color=blue]
> Unfortunately, google keeps telling me that the problem is the garbage
> collector ignoring dead (closed?) sockets instead of removing live
> ones. My problem is
>
>
> x.sock=socket.s ocket(socket.AF _INET,socket.SO CK_STREAM)
> do_stuff(x.sock )
>
>
> def do_stuff(sock):
> sock_list.appen d(sock)
>
> once do_stuff finishes, x.sock disappears, and I can only believe it
> is being garbage collected.[/color]

Can you clarify this? What do you mean by "x.sock" disappears? Are
you getting a NameError later when trying to use "x.sock"?

x.sock is just a name binding, so it is not really involved in garbage
collection (GC applies to the objects to which names are bound).

In this case, you need to include much more in the way of code (a
fully running, but smallest possible, snippet of code would be best),
since the above can be interpreted many ways. At the least, it's very
important to include information about the namespace within which
those two code snippets run if anyone is likely to be able to give you
a good answer. Also, being very precise about the error condition you
are experiencing (including actual error messages, tracebacks, etc...)
is crucial.

Is 'x' referencing a local or global object, and does that socket code
occur within a method, a function, or what? Also, in do_stuff, where
is sock_list defined? Is it local, global?

If, as written, sock_list is a local name to do_stuff, then that
binding is going to disappear when do_stuff completes, thus, the list
to which it is bound will be destroyed, including all references to
objects that the list may contain. So at that point, when you return
from do_stuff, the only reference to the socket object will be in
x.sock. But if 'x' is also local to the function/method where the
call to do_stuff is, the name binding will be removed when the
function/method returns, at which point there will be no references to
the socket object, and yes, it will be destroyed.

But if sock_list is global, and continues to exist when do_stuff
completes, then the reference it contains to the socket will keep the
socket object alive even if you remove the x.sock binding.

-- David

[Scott Robinson]

Re: Socket being garbage collected too early

On 16 Dec 2004 20:38:29 -0500, David Bolen <db3l@fitlinxx. com> wrote:
[color=blue]
>Scott Robinson <dscottr@bellat lantic.net> writes:
>[color=green]
>> I have been having trouble with the garbage collector and sockets.[/color]
>
>Are you actually getting errors or is this just theoretical?
>[color=green]
>> Unfortunately, google keeps telling me that the problem is the garbage
>> collector ignoring dead (closed?) sockets instead of removing live
>> ones. My problem is
>>
>>
>> x.sock=socket.s ocket(socket.AF _INET,socket.SO CK_STREAM)
>> do_stuff(x.sock )
>>
>>
>> def do_stuff(sock):
>> sock_list.appen d(sock)
>>
>> once do_stuff finishes, x.sock disappears, and I can only believe it
>> is being garbage collected.[/color]
>
>Can you clarify this? What do you mean by "x.sock" disappears? Are
>you getting a NameError later when trying to use "x.sock"?
>
>x.sock is just a name binding, so it is not really involved in garbage
>collection (GC applies to the objects to which names are bound).
>
>In this case, you need to include much more in the way of code (a
>fully running, but smallest possible, snippet of code would be best),
>since the above can be interpreted many ways. At the least, it's very
>important to include information about the namespace within which
>those two code snippets run if anyone is likely to be able to give you
>a good answer. Also, being very precise about the error condition you
>are experiencing (including actual error messages, tracebacks, etc...)
>is crucial.
>
>Is 'x' referencing a local or global object, and does that socket code
>occur within a method, a function, or what? Also, in do_stuff, where
>is sock_list defined? Is it local, global?
>
>If, as written, sock_list is a local name to do_stuff, then that
>binding is going to disappear when do_stuff completes, thus, the list
>to which it is bound will be destroyed, including all references to
>objects that the list may contain. So at that point, when you return
>from do_stuff, the only reference to the socket object will be in
>x.sock. But if 'x' is also local to the function/method where the
>call to do_stuff is, the name binding will be removed when the
>function/method returns, at which point there will be no references to
>the socket object, and yes, it will be destroyed.
>
>But if sock_list is global, and continues to exist when do_stuff
>completes, then the reference it contains to the socket will keep the
>socket object alive even if you remove the x.sock binding.
>
>-- David[/color]
(so much for Python being executable psuedocode).

It looks like I was completely wrong. The problem I ran into was not
checking into baseHTTPserver and looking for self.close_conn ection=1.

I am pretty sure this happened to me before, and I rewrote the code to
avoid sockets being closed after being referenced to a live object,
but I can't reproduce it.

Anyway, here is my debugged test rig. It works. Ignore it.


Scott
[test program follows]
import socket, thread, time

class sock_holder:
pass

HOST = '127.0.0.1'
PORT = 2004

def echo_server(por t):
print "started at",port
s = socket.socket(s ocket.AF_INET, socket.SOCK_STR EAM)
s.bind((HOST, port))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
data = conn.recv(1024)
if not data: break
conn.send(data)
conn.close()

def wait_and_speak( a):
time.sleep(5)
try:
a.sock.send("th is is message 2")
except:
print "error on message 2"
try:
data = a.sock.recv(102 4)
print data
except:
print "error recieving message 2"
a.sock.close()

def other_thread(a) :
s = socket.socket(s ocket.AF_INET, socket.SOCK_STR EAM)
thread.start_ne w(echo_server,( PORT,))
time.sleep(1)
s.connect((HOST ,PORT ))
s.send('Hello, port 1')
data = s.recv(1024)
print data
a.sock=s

thread.start_ne w(wait_and_spea k,(a,))

a=sock_holder()
thread.start_ne w(other_thread, (a,))
time.sleep(10)