Re: Thread killing - I know I know!

[Roger Heathcote]

sjdevnull@yahoo .com wrote:

On May 16, 11:40 am, Roger Heathcote <use...@technic albloke.com>
wrote:<snip>

>Despite many peoples insistence that allowing for the arbitrary killing
>of threads is a cardinal sin and although I have no particular threading
>problem to crack right now I remain interest in the taboo that is thread
>killing. The real world and it's data are messy and imperfect and I can

<snip>

In general, use processes when you can and threads only when you
must. OS designers spent a lot of energy implementing protected
memory, no sense throwing out a fair chunk of that hard work unless
you actually need to.

Fair point, but for sub processes that need to be in close contact with
the original app, or very small functions that you'd like 100s or 1000s
of it seems like a kludge having to spawn whole new processes build in
socket communications and kill via explicit OS calls. I can't see that
approach scaling particularly well but I guess there's no choice.

Does anyone think it likely that the threading module (and threading in
general) will be improved and augmented with features like timeouts and
arbitrary threadicide in the next year or two? Seems there's little
scope for tapping the power of the current generation of multiple cores
with the current pythons, tho I appreciate breakneck speed has never
been a design objective it looks like multicore is set to be the next
generation PC architecture.

Roger Heathcote - technicalbloke. com

[Chris Mellon]

Re: Thread killing - I know I know!

On Mon, May 19, 2008 at 11:36 AM, Roger Heathcote
<usenet@technic albloke.comwrot e:

sjdevnull@yahoo .com wrote:

>>
>On May 16, 11:40 am, Roger Heathcote <use...@technic albloke.com>
>wrote:<snip>

>>>
>>Despite many peoples insistence that allowing for the arbitrary killing
>>of threads is a cardinal sin and although I have no particular threading
>>problem to crack right now I remain interest in the taboo that is thread
>>killing. The real world and it's data are messy and imperfect and I can

>
<snip>

>>
>In general, use processes when you can and threads only when you
>must. OS designers spent a lot of energy implementing protected
>memory, no sense throwing out a fair chunk of that hard work unless
>you actually need to.

>
Fair point, but for sub processes that need to be in close contact with the
original app, or very small functions that you'd like 100s or 1000s of it
seems like a kludge having to spawn whole new processes build in socket
communications and kill via explicit OS calls. I can't see that approach
scaling particularly well but I guess there's no choice.
>

It's not a kludge - the whole reason why killing a thread is undefined
and a "cardinal sin" is because it's in your own address space and you
can't guarantee anything about how it left things when you killed it.
You simply can't have it both ways. If you want to be able to safely
and sanely kill an uncooperative thread (*especially* third party code
you don't control) you have to isolate it from your address space, and
this means a process. This is a fundamental problem with the entire
idea of shared-state concurrency. For the record, you can't use 1000s
of threads either - if you really need 1000s of concurrent things
running you need to re-think your concurrency model and possibly your
choice of languages and environment.

The messiness of the real world is *why* you should use processes, not
a reason to avoid them.

Does anyone think it likely that the threading module (and threading in
general) will be improved and augmented with features like timeouts and
arbitrary threadicide in the next year or two? Seems there's little scope
for tapping the power of the current generation of multiple cores with the
current pythons, tho I appreciate breakneck speed has never been a design
objective it looks like multicore is set to be the next generation PC
architecture.
>

They absolutely should not be. Adding an API which can't work in the
general case and is dangerous even when it can work does not do anyone
any favors.

Roger Heathcote - technicalbloke. com
--

http://mail.python.org/mailman/listinfo/python-list

>

[blaine]

Re: Thread killing - I know I know!

How is it that you recommend killing a pure python thread? And also,
when i use the 'threading' module am I creating a process, or a (gasp)
thread? (Is this an obvious question - and if so, how does one create
a 'process'?)

Thanks!

[Paul Rubin]

Re: Thread killing - I know I know!

blaine <frikker@gmail. comwrites:

How is it that you recommend killing a pure python thread? And also,
when i use the 'threading' module am I creating a process, or a (gasp)
thread?

A thread.

(Is this an obvious question - and if so, how does one create
a 'process'?)

os.fork()

[Gabriel Genellina]

Re: Thread killing - I know I know!

En Mon, 19 May 2008 20:07:06 -0300, Paul Rubin
<"http://phr.cx"@NOSPAM. invalidescribió :

blaine <frikker@gmail. comwrites:

>How is it that you recommend killing a pure python thread?

Make the thread cooperate. It should periodically check some variable or
condition, and cleanly exit when requested.

>And also,
>when i use the 'threading' module am I creating a process, or a (gasp)
>thread?

>
A thread.
>

>(Is this an obvious question - and if so, how does one create
>a 'process'?)

>
os.fork()

A more portable answer: using the subprocess module.

--
Gabriel Genellina

[blaine]

Re: Thread killing - I know I know!

On May 19, 8:16 pm, "Gabriel Genellina" <gagsl-...@yahoo.com.a r>
wrote:

En Mon, 19 May 2008 20:07:06 -0300, Paul Rubin
<"http://phr.cx"@NOSPAM. invalidescribió :
>

blaine <frik...@gmail. comwrites:

How is it that you recommend killing a pure python thread?

>
Make the thread cooperate. It should periodically check some variable or
condition, and cleanly exit when requested.
>

And also,
when i use the 'threading' module am I creating a process, or a (gasp)
thread?

>

A thread.

>

(Is this an obvious question - and if so, how does one create
a 'process'?)

>

os.fork()

>
A more portable answer: using the subprocess module.
>
--
Gabriel Genellina

Thanks for the info. The problem that I'm having is that I get a
thread stuck blocking - waiting for a return from a FIFO (made with
os.mkfifo()).

Blaine

[Rhamphoryncus]

Re: Thread killing - I know I know!

On May 19, 11:31 am, "Chris Mellon" <arka...@gmail. comwrote:

On Mon, May 19, 2008 at 11:36 AM, Roger Heathcote

Fair point, but for sub processes that need to be in close contact with the
original app, or very small functions that you'd like 100s or 1000s of it
seems like a kludge having to spawn whole new processes build in socket
communications and kill via explicit OS calls. I can't see that approach
scaling particularly well but I guess there's no choice.

>
It's not a kludge - the whole reason why killing a thread is undefined
and a "cardinal sin" is because it's in your own address space and you
can't guarantee anything about how it left things when you killed it.
You simply can't have it both ways. If you want to be able to safely
and sanely kill an uncooperative thread (*especially* third party code
you don't control) you have to isolate it from your address space, and
this means a process. This is a fundamental problem with the entire
idea of shared-state concurrency. For the record, you can't use 1000s
of threads either - if you really need 1000s of concurrent things
running you need to re-think your concurrency model and possibly your
choice of languages and environment.

Strictly speaking, you CAN use 1000s of threads, but it requires some
special effort.

It's not a good idea though, at least for the near future. If you
want to kill a thread you have to do it cooperatively, and for the
effort required to get that right you might as well make the whole
thing event-driven.

The messiness of the real world is *why* you should use processes, not
a reason to avoid them.
>

Does anyone think it likely that the threading module (and threading in
general) will be improved and augmented with features like timeouts and
arbitrary threadicide in the next year or two? Seems there's little scope
for tapping the power of the current generation of multiple cores with the
current pythons, tho I appreciate breakneck speed has never been a design
objective it looks like multicore is set to be the next generation PC
architecture.

>
They absolutely should not be. Adding an API which can't work in the
general case and is dangerous even when it can work does not do anyone
any favors.

I agree, if it's not safe there's no point adding it.

However, if you change the parameters a little bit.. it's quite
possible to redefine many blocking functions as cooperative
cancellation points. This is the approach I take with python-
safethread, and it should satisfy most purposes.