killing thread after timeout

[Jacek Pop³awski]

Hello.

I am going to write python script which will read python command from
socket, run it and return some values back to socket.

My problem is, that I need some timeout. I need to say for example:

os.system("some application.exe ")

and kill it, if it waits longer than let's say 100 seconds

I want to call command on separate thread, then after given timeout -
kill thread, but I realized (after reading Usenet archive) that there is
no way to kill a thread in Python.

How can I implement my script then?

PS. it should be portable - Linux, Windows, QNX, etc

[Jacek Pop³awski]

Re: killing thread after timeout

After reading more archive I think that solution may be to raise an
Exception after timeout, but how to do it portable?

[sp1d3rx@gmail.com]

Re: killing thread after timeout

You'll need a watchdog thread that handles the other threads. The
watchdog thread just builds a table of when threads were started, and
after a certain # of seconds, expires those threads and kills them.

[Steve Horsley]

Re: killing thread after timeout

Jacek Pop³awski wrote:[color=blue]
> Hello.
>
> I am going to write python script which will read python command from
> socket, run it and return some values back to socket.
>
> My problem is, that I need some timeout. I need to say for example:
>
> os.system("some application.exe ")
>
> and kill it, if it waits longer than let's say 100 seconds
>
> I want to call command on separate thread, then after given timeout -
> kill thread, but I realized (after reading Usenet archive) that there is
> no way to kill a thread in Python.
>
> How can I implement my script then?
>
> PS. it should be portable - Linux, Windows, QNX, etc[/color]


Probably the easiest way is to use select with a timeout (see the
docs for library module select). eg.

a, b c = select.select([mySocket], [], [], timeout)
if len(a) == 0:
print 'We timed out'
else:
print 'the socket has something for us'


Steve

[Peter Hansen]

Re: killing thread after timeout

Jacek Pop³awski wrote:[color=blue]
> I am going to write python script which will read python command from
> socket, run it and return some values back to socket.[/color]

(Sounds like a huge security risk, unless you have tight control over
who can connect to that socket.)
[color=blue]
> My problem is, that I need some timeout. I need to say for example:
>
> os.system("some application.exe ")
>
> and kill it, if it waits longer than let's say 100 seconds
>
> I want to call command on separate thread, then after given timeout -
> kill thread, but I realized (after reading Usenet archive) that there is
> no way to kill a thread in Python.[/color]

The issue isn't killing a thread in Python, it's killing the *new
process* which that thread has started. To do that you have to rely on
OS-specific (i.e. non-portable) techniques. Googling for "python kill
process" would probably get you off to a good start.

-Peter

[Bryan Olson]

Re: killing thread after timeout


Jacek Poplawski had written:[color=blue][color=green]
>> I am going to write python script which will read python
>> command from socket, run it and return some values back to
>> socket.
>>
>> My problem is, that I need some timeout.[/color][/color]

Jacek Poplawski wrote:[color=blue]
> After reading more archive I think that solution may be to raise an
> Exception after timeout, but how to do it portable?[/color]

Python allows any thread to raise a KeyboardInterru pt in the
main thread (see thread.interrup t_main), but I don't think there
is any standard facility to raise an exception in any other
thread. I also believe, and hope, there is no support for lower-
level killing of threads; doing so is almost always a bad idea.
At arbitrary kill-times, threads may have important business
left to do, such as releasing locks, closing files, and other
kinds of clean-up.

Processes look like a better choice than threads here. Any
decent operating system will put a deceased process's affairs
in order.


Anticipating the next issues: we need to spawn and connect to
the various worker processes, and we need to time-out those
processes.

First, a portable worker-process timeout: In the child process,
create a worker daemon thread, and let the main thread wait
until either the worker signals that it is done, or the timeout
duration expires. As the Python Library Reference states in
section 7.5.6:

A thread can be flagged as a "daemon thread". The
significance of this flag is that the entire Python program
exits when only daemon threads are left.

The following code outlines the technique:

import threading

work_is_done = threading.Event ()

def work_to_do(*arg s):
# ... Do the work.
work_is_done.se t()

if __name__ == '__main__':
# ... Set stuff up.
worker_thread = threading.Threa d(
target = work_to_do,
args = whatever_params )
worker_thread.s etDaemon(True)
worker_thread.s tart()
work_is_done.wa it(timeout_dura tion)



Next, how do we connect the clients to the worker processes?

If Unix-only is acceptable, we can set up the accepting socket,
and then fork(). The child processes can accept() incomming
connections on its copy of the socket. Be aware that select() on
the process-shared socket is tricky, in that that the socket can
select as readable, but the accept() can block because some
other processes took the connection.


If we need to run on Windows (and Unix), we can have one main
process handle the socket connections, and pipe the data to and
from worker processes. See the popen2 module in the Python
Standard Library.


--
--Bryan

[Bryan Olson]

Re: killing thread after timeout

Bryan Olson wrote:
[Some stuff he thinks is right, but might not answer the real
question]

Definitely look into Peter Hanson's answer.

Olson's answer was about timing-out one's own Python code.


Bryan Olson has heretofore avoided referring to himself in the
third person, and will hence forth endeavor to return to his
previous ways.

--
--Bryan

[Jacek Pop³awski]

Re: killing thread after timeout

Bryan Olson wrote:[color=blue]
> First, a portable worker-process timeout: In the child process,
> create a worker daemon thread, and let the main thread wait
> until either the worker signals that it is done, or the timeout
> duration expires.[/color]

It works on QNX, thanks a lot, your reply was very helpful!
[color=blue]
> If we need to run on Windows (and Unix), we can have one main
> process handle the socket connections, and pipe the data to and
> from worker processes. See the popen2 module in the Python
> Standard Library.[/color]

popen will not work in thread on QNX/Windows, same problem with spawnl
currently I am using:

os.system(comma nd+">file 2>file2")

it works, I just need to finish implementing everything and check how it
may fail...

One more time - thanks for great idea!

[Paul Rubin]

Re: killing thread after timeout

Bryan Olson <fakeaddress@no where.org> writes:[color=blue]
> First, a portable worker-process timeout: In the child process,
> create a worker daemon thread, and let the main thread wait
> until either the worker signals that it is done, or the timeout
> duration expires. As the Python Library Reference states in
> section 7.5.6:[/color]

Maybe the child process can just use sigalarm instead of a separate
thread, to implement the timeout.
[color=blue]
> If Unix-only is acceptable, we can set up the accepting socket,
> and then fork(). The child processes can accept() incomming
> connections on its copy of the socket. Be aware that select() on
> the process-shared socket is tricky, in that that the socket can
> select as readable, but the accept() can block because some
> other processes took the connection.[/color]

To get even more OS-specific, AF_UNIX sockets (at least on Linux) have
a feature called ancillary messages that allow passing file
descriptors between processes. It's currently not supported by the
Python socket lib, but one of these days... . But I don't think
Windows has anything like it. No idea about QNX.

[Jacek Popławski]

Re: killing thread after timeout

Paul Rubin wrote:[color=blue]
> Maybe the child process can just use sigalarm instead of a separate
> thread, to implement the timeout.[/color]

Already tried that, signals works only in main thread.
[color=blue]
> To get even more OS-specific, AF_UNIX sockets (at least on Linux) have
> a feature called ancillary messages that allow passing file
> descriptors between processes. It's currently not supported by the
> Python socket lib, but one of these days... . But I don't think
> Windows has anything like it. No idea about QNX.[/color]

I have solved problem with additional process, just like Bryan Olson
proposed. Looks like all features I wanted are working... :)

[Bryan Olson]

Re: killing thread after timeout

Paul Rubin wrote:[color=blue]
> To get even more OS-specific, AF_UNIX sockets (at least on Linux) have
> a feature called ancillary messages that allow passing file
> descriptors between processes. It's currently not supported by the
> Python socket lib, but one of these days... . But I don't think
> Windows has anything like it.[/color]

It can be done on on Windows.

http://tangentsoft.net/wskfaq/articles/passing-sockets.html

--
--Bryan