how to do non-blocking I/O

[J. Xu]

I am a newbie to python. I am writing a client application, I need to
read query results from the server. Mostly I'll have a file object for
the connection to the server. So I use read() or readline() to get the
response. But since these are blocking calls, I have to use a seperate
thread for the input reading. But then I have another problem, my input
thread running in the following loop

while not quit:
str = ins.readline()
.... process the response

so I can't terminate the thread when it's blocking in the readline()
until the connection is closed. So looks like I need to use non-blocking
IO such as select() or poll(). But since I need this application to be
cross-platform, and these are not well supported on Windows, I don't
know what to do.

[Donn Cave]

Re: how to do non-blocking I/O

In article <mailman.105.10 79557901.742.py thon-list@python.org >,
"J. Xu" <xu@reflexsecur ity.com> wrote:
[color=blue]
> I am a newbie to python. I am writing a client application, I need to
> read query results from the server. Mostly I'll have a file object for
> the connection to the server. So I use read() or readline() to get the
> response. But since these are blocking calls, I have to use a seperate
> thread for the input reading. But then I have another problem, my input
> thread running in the following loop
>
> while not quit:
> str = ins.readline()
> .... process the response
>
> so I can't terminate the thread when it's blocking in the readline()
> until the connection is closed. So looks like I need to use non-blocking
> IO such as select() or poll(). But since I need this application to be
> cross-platform, and these are not well supported on Windows, I don't
> know what to do.[/color]

Far as I know, what you want to do may not be possible with
portable functions, so you may have to just implement separately
for each platform.

If you decide to use select or poll, note that they do not
account for data buffered in a file object. This loses data,
in effect, because even though it's in your process buffer,
you won't read it if select says there's no new data. You
can turn buffering off, but that forces readline() to read
1 byte at a time at an unreasonable cost in system calls.
In my opinion it makes much more sense to use recv on the
socket, and not use a file object here.

Donn Cave, donn@u.washingt on.edu

[Peter Hansen]

Re: how to do non-blocking I/O

J. Xu wrote:
[color=blue]
> I am a newbie to python. I am writing a client application, I need to
> read query results from the server. Mostly I'll have a file object for
> the connection to the server. So I use read() or readline() to get the
> response. But since these are blocking calls, I have to use a seperate
> thread for the input reading. But then I have another problem, my input
> thread running in the following loop
>
> while not quit:
> str = ins.readline()
> .... process the response
>
> so I can't terminate the thread when it's blocking in the readline()
> until the connection is closed. So looks like I need to use non-blocking
> IO such as select() or poll(). But since I need this application to be
> cross-platform, and these are not well supported on Windows, I don't
> know what to do.[/color]

select() is perfectly well supported (to a reasonable degree anyway) on
Windows, at least with actual sockets. Don't know whether it applies to
file objects attached to sockets, and never had cause to check...

If you can use the sockets themselves, with .recv() instead, then you
can safely use select() even on Windows.

-Peter