synchronized method

[Max Ischenko]

Hi,

I wrote simple implementation of the "synchroniz ed" methods (a-la Java),
could you please check if it is OK:

def synchronized(me thod):

"""
Guards method execution, similar to Java's synchronized keyword.

The class which uses this method, is required to have a
L{threading.Loc k} primitive as an attribute named 'lock'.
"""

def wrapper(self, *args):
self.lock.acqui re()
try:
return method(self, *args)
finally:
self.lock.relea se()
return wrapper

....I'm no big thread expert

tia.

.... The problem is not that there are problems. The problem is expecting
otherwise and thinking that having problems is a problem.
-- Theodore Rubin

[anton muhin]

Re: synchronized method

Max Ischenko wrote:
[color=blue]
>
> Hi,
>
> I wrote simple implementation of the "synchroniz ed" methods (a-la Java),
> could you please check if it is OK:
>
> def synchronized(me thod):
>
> """
> Guards method execution, similar to Java's synchronized keyword.
>
> The class which uses this method, is required to have a
> L{threading.Loc k} primitive as an attribute named 'lock'.
> """
>
> def wrapper(self, *args):
> self.lock.acqui re()
> try:
> return method(self, *args)
> finally:
> self.lock.relea se()
> return wrapper
>
> ...I'm no big thread expert
>
> tia.
>
> ... The problem is not that there are problems. The problem is expecting
> otherwise and thinking that having problems is a problem.
> -- Theodore Rubin[/color]

I'd add **kwargs too.

anton.

[Graham Dumpleton]

Re: synchronized method

Max Ischenko <max@ucmg.com.u a.remove.it> wrote in message news:<c0a1mt$v7 9$1@hyppo.gu.ne t>...[color=blue]
> Hi,
>
> I wrote simple implementation of the "synchroniz ed" methods (a-la Java),
> could you please check if it is OK:
>
> def synchronized(me thod):
>
> """
> Guards method execution, similar to Java's synchronized keyword.
>
> The class which uses this method, is required to have a
> L{threading.Loc k} primitive as an attribute named 'lock'.
> """
>
> def wrapper(self, *args):
> self.lock.acqui re()
> try:
> return method(self, *args)
> finally:
> self.lock.relea se()
> return wrapper[/color]

Under normal execution the lock will not be released because you return
from the try clause. There is also the case that method isn't defined.

Someone else was asking about this sort of thing back the end of January.
The post is included below. As much as doing this sort of thing may sound
like a good way of handling synchronisation , it still has its problems.

gagenellina@sof tlab.com.ar (Gabriel Genellina) wrote in message news:<946d2ebc. 0401301753.5bbe 87c5@posting.go ogle.com>...[color=blue]
> Hello!
>
> I have a number of objects that are not thread-safe - no locking
> mechanism was originally provided.
> But I want to use them in a thread-safe way. I don't want to analyze
> each class details, looking for where and when I must protect the
> code. So I wrote this sort-of-brute-force locking mechanism using
> threading.RLock () around __getattr__ / __setattr__ / __delattr__.
> The idea is that *any* attribute and method access is protected by the
> lock. The calling thread may acquire the lock multiple times, but
> other threads must wait until it is released.
> This may be too much restrictive, and a bottleneck in performance,
> since *all* accesses are effectively serialized - but it's the best I
> could imagine without deeply analyzing the actual code.
> I *think* this works well, but since I'm not an expert in these
> things, any comments are welcome. Specially what could go wrong, if
> someone could anticipate it.
> Note: As written, this only works for old-style classes. I think that
> just by adding a similar __getattribute_ _ would suffice for new-style
> classes too - is that true?[/color]

A few problems that I can see. First is that this only protects access to
the reference to the data in question. Imagine that the data value
is a dictionary. If I understand it correctly, two different threads
could still get a reference to the same dictionary and then independently
start modifying the dictionary at the same time. This is because you are
then dealing with the dictionary directly and your locks aren't involved
at all.

Second is that it isn't always sufficient to lock at the level of individual
attributes, instead it sometimes necessary to lock on a group of attributes
or even the whole object. This may be because a thread has to be able
to update two attributes at one time and know that another thread will
not change one of them while it is modifying the other.

One hack I have used is always ensuring that access to an object is
through its functional interface and then locking at the level of
function calls. That is provide additional member functions called
lockObject and unlockObject which operate the thread mutex. When
using the class then have:

object.lockObje ct()
object.someFunc 1()
object.someFunc 2()
object.etc()
object.unlockOb ject()

This can sort of be automated a bit by having:

class _SynchronisedMe thod:

def __init__(self,o bject,name):
self._object = object
self._name = name

def __getattr__(sel f,name):
return _SynchronisedMe thod(self._obje ct,"%s.%s"%(sel f._name,name))

def __call__(self,* args):
try:
self._object.lo ckObject()
method = getattr(self._o bject,self._nam e)
result = apply(method,ar gs)
self._object.un lockObject()
except:
self._object.un lockObject()
raise
return result

class _SynchronisedOb ject:

def __init__(self,o bject):
self._object = object

def __getattr__(sel f,name):
return _SynchronisedMe thod(self._obje ct,name)

You can then say:

sync_object = _SynchronisedOb ject(object)

sync_object.som eFunc1()
sync_object.som eFunc2()
sync_object.etc ()

Although this avoids the explicit calls to lock the object, it still isn't the
same thing. This is because in the first example, the lock was held across
multiple function calls whereas in the second case it is only held for a
single call. Thus something like the following could fail:

if sync_object.dat aExists():
sync_object.get Data()

This is because something could have taken the data between the time
the check for data was made and when it was obtained.

Thus automating it like this isn't foolproof either. The closest you will
probably get to a quick solution is to have the lock/unlock functions
for the object and change any code to use them around any sections of
code which you don't want another thread operating on the class at
the same time. You may even have to hold locks on multiple objects
at the same time to ensure that things don't get stuffed up.

Thus it isn't necessarily a simple task to add threading support after
the fact. You probably will have no choice but to analyse the use of
everything and work out to correctly implement locking or even change
the functional interface a bit so functions can be atomic in themselves.
Ie., rather than having to check if data exists before first getting it,
you just try and get it and either rely on an exception or have the
thread wait until data is there.

[Max Ischenko]

Re: synchronized method

Graham Dumpleton wrote:
[color=blue][color=green]
>>I wrote simple implementation of the "synchroniz ed" methods (a-la Java),
>>could you please check if it is OK:
>>
>>def synchronized(me thod):
>>
>> """
>> Guards method execution, similar to Java's synchronized keyword.
>>
>> The class which uses this method, is required to have a
>> L{threading.Loc k} primitive as an attribute named 'lock'.
>> """
>>
>> def wrapper(self, *args):
>> self.lock.acqui re()
>> try:
>> return method(self, *args)
>> finally:
>> self.lock.relea se()
>> return wrapper[/color]
>
>
> Under normal execution the lock will not be released because you return
> from the try clause.[/color]

Hmm. I think you must be wrong:
[color=blue][color=green][color=darkred]
>>> def foo():[/color][/color][/color]
try:
return 1
finally:
print 'ok'

[color=blue][color=green][color=darkred]
>>> foo()[/color][/color][/color]
ok
1
[color=blue]
> There is also the case that method isn't defined.[/color]

Indeed, thanks for pointing this out.
[color=blue]
> Someone else was asking about this sort of thing back the end of January.
> The post is included below. As much as doing this sort of thing may sound
> like a good way of handling synchronisation , it still has its problems.[/color]

[ cut ]

I'll look into it.

[Peter Hansen]

Re: synchronized method

Graham Dumpleton wrote:[color=blue]
>
> Max Ischenko <max@ucmg.com.u a.remove.it> wrote in message news:<c0a1mt$v7 9$1@hyppo.gu.ne t>...[color=green]
> > Hi,
> >
> > I wrote simple implementation of the "synchroniz ed" methods (a-la Java),
> > could you please check if it is OK:
> >
> > def synchronized(me thod):
> >
> > """
> > Guards method execution, similar to Java's synchronized keyword.
> >
> > The class which uses this method, is required to have a
> > L{threading.Loc k} primitive as an attribute named 'lock'.
> > """
> >
> > def wrapper(self, *args):
> > self.lock.acqui re()
> > try:
> > return method(self, *args)
> > finally:
> > self.lock.relea se()
> > return wrapper[/color]
>
> Under normal execution the lock will not be released because you return
> from the try clause. There is also the case that method isn't defined.[/color]

If that were true, the finally clause would be pretty useless in many
cases. (Try it out yourself: it will work properly.)

Also, isn't "method" defined as the argument to the synchronized()
function at the top?

To me, this looks like a pretty reasonable approach to handling
the problem as defined.

-Peter

[Graham Dumpleton]

Re: synchronized method

Peter Hansen <peter@engcorp. com> wrote in message news:<40290916. ECCDE661@engcor p.com>...[color=blue][color=green]
> > Under normal execution the lock will not be released because you return
> > from the try clause. There is also the case that method isn't defined.[/color]
>
> If that were true, the finally clause would be pretty useless in many
> cases. (Try it out yourself: it will work properly.)
>
> Also, isn't "method" defined as the argument to the synchronized()
> function at the top?[/color]

Hmmm, I stand corrected on both counts. It was one of the days I should
have kept my mouth shut. Too stinking hot here at the moment, my brain
isn't working. :-)