thread1:
while 1:
buf = s.read()
process(buf)
thread2:
while 1:
buf = getdata()
s.write(buf)
-
thread1:
while 1:
buf = s.read()
process(buf)
thread2:
while 1:
buf = getdata()
s.write(buf)
-
Re: is socket thread safe?
e2wugui@gmail.c om:[color=blue]
>[code][/color]
I certainly expect socket to be threadsafe. I use it (via urllib2/httplib)
in a multithreaded program, that runs fine with Python 2.3 and 2.4 on both
Windows XP and Linux.
--
René Pijlman
-
Re: is socket thread safe?
e2wugui@gmail.c om wrote:[color=blue]
> thread1:
> while 1:
> buf = s.read()
> process(buf)
>
> thread2:
> while 1:
> buf = getdata()
> s.write(buf)
>[/color]
It is safe, but watch out for this gotcha: If thread B calls
s.close() while thread A is blocked in s.read(), thread A will
never return from the read. My preferred solution is to set
socket timeout to a few seconds, and loop checking a status flag
so I know when to quit.
Steve
Comment
-
Re: is socket thread safe?
Steve Horsley wrote:[color=blue]
> e2wugui@gmail.c om wrote:
>[color=green]
>> thread1:
>> while 1:
>> buf = s.read()
>> process(buf)
>>
>> thread2:
>> while 1:
>> buf = getdata()
>> s.write(buf)
>>
>>[/color]
>
> It is safe, but watch out for this gotcha: If thread B calls
> s.close() while thread A is blocked in s.read(), thread A will
> never return from the read. My preferred solution is to set
> socket timeout to a few seconds, and loop checking a status flag
> so I know when to quit.
>
>[/color]
I think a better thing would be to use something like a condition object
to tie the two threads together and not use any polling loops.
i.e. consumer goes to sleep while data buffer is empty, producer
produces and signals condition object, consumer wakes up and consumes.
To take this a step further, you have a status flag that is set to
something like QUIT or CONSUME and when the condition is triggered wake
up, then examine the status flag to determine if the consumer should
then quit, consume, or whatever else you'd want your consumer thread to do.
-carl
--
Carl J. Van Arsdall
cvanarsdall@mvi sta.com
Build and Release
MontaVista Software
Comment
-
Re: is socket thread safe?
e2wugui@gmail.c om wrote:[color=blue]
> thread1:
> while 1:
> buf = s.read()
> process(buf)
>
> thread2:
> while 1:
> buf = getdata()
> s.write(buf)[/color]
Sockets don't have read() and write() methods. Connected
sockets have recv() and send()/sendall(). Python's socket
module has a makefile() function, but the resulting file
object wouldn't make sense in the read loop above (and
throws away control that network code typically needs).
Is it safe for one thread to receive from a socket while
another sends over the socket? Yes, that much is safe and
perfectly reasonable.
--
--Bryan
Comment
-
Re: is socket thread safe?
In article <F5PIf.30441$F_ 3.27833@newssvr 29.news.prodigy .net>,
Bryan Olson <fakeaddress@no where.org> wrote:
[color=blue]
> Is it safe for one thread to receive from a socket while
> another sends over the socket? Yes, that much is safe and
> perfectly reasonable.[/color]
I hear it works on most common platforms these days,
anyway. I have seen socket implementations where it
didn't, though - the read would end as if interrupted
by a signal.
Donn Cave, donn@u.washingt on.edu
Comment
-
Re: is socket thread safe?
Carl J. Van Arsdall wrote:[color=blue]
> Steve Horsley wrote:
>[color=green]
>> e2wugui@gmail.c om wrote:
>>
>>[color=darkred]
>>> thread1:
>>> while 1:
>>> buf = s.read()
>>> process(buf)
>>>
>>> thread2:
>>> while 1:
>>> buf = getdata()
>>> s.write(buf)[/color]
>>
>>
>> It is safe, but watch out for this gotcha: If thread B calls s.close()
>> while thread A is blocked in s.read(), thread A will never return from
>> the read. My preferred solution is to set socket timeout to a few
>> seconds, and loop checking a status flag so I know when to quit.[/color][/color]
Certainly one needs timeouts to avoid hanging should the remote
side stop. Sockets don't have a read() method, and hanging on
recv() doesn't seem to have anything to do with close().
I didn't find any definitive doc, so I tested using Python
sockets on Linux (Debian/Ubuntu current) and WinXP. A recv()
started before the close() will block/return just as if
close() were never called. The close() neither triggers recv()
to abort, nor prevents it from receiving data and detecting
shutdown.
[color=blue]
> I think a better thing would be to use something like a condition object
> to tie the two threads together and not use any polling loops.
>
> i.e. consumer goes to sleep while data buffer is empty, producer
> produces and signals condition object, consumer wakes up and consumes.[/color]
I can infer two producer-consumer relationships from the example,
but they don't allow a condition object; the writer's consumer and
the reader's producer are on the remote end of the socket. The
socket will already handle the blocking and wake-up.
[color=blue]
> To take this a step further, you have a status flag that is set to
> something like QUIT or CONSUME and when the condition is triggered wake
> up, then examine the status flag to determine if the consumer should
> then quit, consume, or whatever else you'd want your consumer thread to do.[/color]
What problem are you trying to solve? Normal socket sending, receiving,
and shutdown discipline work fine. When the writer is done writing, it
should call sock.shutdown(s ocket.SHUT_WR). When the reader gets zero
bytes from recv(nonzero), that means the remote end has finished
writing, so the reader may call sock.shutdown(s ocket.SHUT_RD).
--
--Bryan
Comment
-
Re: is socket thread safe?
Bryan Olson wrote:[color=blue]
> Carl J. Van Arsdall wrote:
>[color=green]
>> Steve Horsley wrote:
>>
>>[color=darkred]
>>> e2wugui@gmail.c om wrote:
>>>
>>>
>>>
>>>> thread1:
>>>> while 1:
>>>> buf = s.read()
>>>> process(buf)
>>>>
>>>> thread2:
>>>> while 1:
>>>> buf = getdata()
>>>> s.write(buf)
>>>>
>>> It is safe, but watch out for this gotcha: If thread B calls s.close()
>>> while thread A is blocked in s.read(), thread A will never return from
>>> the read. My preferred solution is to set socket timeout to a few
>>> seconds, and loop checking a status flag so I know when to quit.
>>>[/color][/color]
>
> Certainly one needs timeouts to avoid hanging should the remote
> side stop. Sockets don't have a read() method, and hanging on
> recv() doesn't seem to have anything to do with close().
>
> I didn't find any definitive doc, so I tested using Python
> sockets on Linux (Debian/Ubuntu current) and WinXP. A recv()
> started before the close() will block/return just as if
> close() were never called. The close() neither triggers recv()
> to abort, nor prevents it from receiving data and detecting
> shutdown.
>
>
>[color=green]
>> I think a better thing would be to use something like a condition object
>> to tie the two threads together and not use any polling loops.
>>
>> i.e. consumer goes to sleep while data buffer is empty, producer
>> produces and signals condition object, consumer wakes up and consumes.
>>[/color]
>
> I can infer two producer-consumer relationships from the example,
> but they don't allow a condition object; the writer's consumer and
> the reader's producer are on the remote end of the socket. The
> socket will already handle the blocking and wake-up.
>
>[color=green]
>> To take this a step further, you have a status flag that is set to
>> something like QUIT or CONSUME and when the condition is triggered wake
>> up, then examine the status flag to determine if the consumer should
>> then quit, consume, or whatever else you'd want your consumer thread to do.
>>[/color]
>
> What problem are you trying to solve? Normal socket sending, receiving,
> and shutdown discipline work fine. When the writer is done writing, it
> should call sock.shutdown(s ocket.SHUT_WR). When the reader gets zero
> bytes from recv(nonzero), that means the remote end has finished
> writing, so the reader may call sock.shutdown(s ocket.SHUT_RD).
>[/color]
Doh! I read the word threads and got carried away not even realizing
sockets. Well, looks like today i'll just have to remember to drink my
coffee
:-D
-c
--
Carl J. Van Arsdall
cvanarsdall@mvi sta.com
Build and Release
MontaVista Software
Comment
Comment