Immutable object thread-safety

[k3xji]

Hi all,

This will probably be a long question/short answer, sorry, but I have
wandered net about the subject and really feel cannot find just enough
information.I want to ask my question by giving an example to
explicitly express my understanding which may be also wrong:

So, let's have a string in a module like:
st = 'IAmAstring'
My understanding: Python allocates a string object and binds the st to
that string's reference.

and let's have two threads simply doing following operations on the st
string:

Thread A:
st += 'ThreadAWasHere '

Thread B
st = 'ThreadBWasHere '

Let's think about the following situation:
Thread A reads the st value, and it is currently binded to an object
which is holding the string 'IAmAString'.
So let Thread B starts execution at that point. Note that Thread A
does not add the new string to object, it only reads st object. So,
then Thread B just writes to st, and all of a sudden st is binded to
something which is ThreadBWasHere. And then let's continue Thread A,
it already readed st's reference which is a reference to 'IamAString'.

So, finally if the execution flow is like above, we will have
'IAmAStringThre adAWasHere'. I am assuming this should not be the case.
We should have 'ThreadBWasHere ThreadAWasHere' in the final string,
because we have just consumed the code in ThreadB.

Also, the other question is the operation st = 'ThreadBWasHere ' is
atomic? I mean, if Python does not guarantee if an immutable object
assignment is atomic, then how can we say that the object is thread-
safe? So, if it is an atomic operation, which operators are atomic,
means only assignment'='? How about other operators like +=, or -
=..etc?

I am confused about which data structure to rely on thread-safety, or
operator in Python?

Regards,

[Alcari The Mad]

Re: Immutable object thread-safety

k3xji wrote:

Hi all,
>
This will probably be a long question/short answer, sorry, but I have
wandered net about the subject and really feel cannot find just enough
information.I want to ask my question by giving an example to
explicitly express my understanding which may be also wrong:
>
So, let's have a string in a module like:
st = 'IAmAstring'

When you make an assignment like that, python generates bytecode that
looks like this(straight from dis.dis):
1 0 LOAD_CONST 0 ('IAmAstring')
3 STORE_NAME 0 (a)

That's 1 instruction to load the constant and another one to bind it.

My understanding: Python allocates a string object and binds the st to
that string's reference.
>
and let's have two threads simply doing following operations on the st
string:
>
Thread A:
st += 'ThreadAWasHere '
>
Thread B
st = 'ThreadBWasHere '
>
Let's think about the following situation:
Thread A reads the st value, and it is currently binded to an object
which is holding the string 'IAmAString'.
So let Thread B starts execution at that point. Note that Thread A
does not add the new string to object, it only reads st object. So,
then Thread B just writes to st, and all of a sudden st is binded to
something which is ThreadBWasHere. And then let's continue Thread A,
it already readed st's reference which is a reference to 'IamAString'.
>
So, finally if the execution flow is like above, we will have
'IAmAStringThre adAWasHere'. I am assuming this should not be the case.
We should have 'ThreadBWasHere ThreadAWasHere' in the final string,
because we have just consumed the code in ThreadB.
>

Assuming that thread A and B contain nothing but that code, you will
either get 'ThreadBWasHere ThreadAWasHere' or just 'ThreadBWasHere ',
because by default the interpreter switches threads every 100 bytecode
instructions.

Also, the other question is the operation st = 'ThreadBWasHere ' is
atomic? I mean, if Python does not guarantee if an immutable object
assignment is atomic, then how can we say that the object is thread-
safe? So, if it is an atomic operation, which operators are atomic,
means only assignment'='? How about other operators like +=, or -
=..etc?

The augmented assignment operators first load the current value of the
variable like this:
a += 'asdf'
becomes
1 0 LOAD_NAME 0 (a)
3 LOAD_CONST 0 ('asdf')
6 INPLACE_ADD
7 STORE_NAME 0 (a)

>
I am confused about which data structure to rely on thread-safety, or
operator in Python?

All of the builtin functions(which are implemented in C, like len()) are
atomic(but assigning their output to a value may not be).

I hope this helps,
AlcariTheMad

[Gabriel Genellina]

Re: Immutable object thread-safety

En Sun, 26 Oct 2008 23:25:09 -0200, Alcari The Mad
<AlcariTheMad@g mail.comescribi ó:

>I am confused about which data structure to rely on thread-safety, or
>operator in Python?

All of the builtin functions(which are implemented in C, like len()) are
atomic(but assigning their output to a value may not be).

You can't count on the builtins being atomic. len(x) executes
type(x).__len__ if such method exists, which may execute arbitrary Python
code, even trigger the garbage collector and run absolutely unrelated
things.
See this effbot page for discussion [1] - but in general, since the
language reference doesn't specify whether an operation is atomic or not,
you should not count on it. Use a lock when required.

[1]

http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm

--
Gabriel Genellina