Decorator metaclass

**Maric Michaud** · Jun 27 '08, 04:26 PM

Re: Decorator metaclass

Le Friday 23 May 2008 04:28:22 thomas.karolski @googlemail.com, vous avez
écrit :

Hi,
I would like to create a Decorator metaclass, which automatically
turns a class which inherits from the "Decorator" type into a
decorator.
A decorator in this case, is simply a class which has all of its
decorator implementation inside a decorator() method. Every other
attribute access is being proxied to decorator().get Parent().
>
...
>
-------------------------------------------------------
Unfortunately this does not work. The newly defined __init__ method
inside __new__, does a call to impl(*args, **dargs). However, since
the HBar.__init__ calls the Decorator.__ini t__ method, but the
HBar.__init__ method no longer resides inside HBar, but rather inside
HBarImpl (which is no longer a subtype of Decorator), the compiler
complains that Decorator.__ini t__ is not being called with a Decorator
instance as its first argument (which is true).
I tried changing the definition of impl inside __new__ to have
Decorator as one of its bases, but then for some reason impl(*args,
**dargs) asks for 4 arguments (just like __new__) and I have no clue
as to why that happens.
>
Any help on this?
>

The problem with kind of design is that you must break the rules of class
inheritance, and it seems like a strange idea to implement decorators by
inheritance.

Of course you could do all sort of magic with python, but what is your goal?
In your example, should the implementation types inherit from each other ?
In that case, do you want to preserve the same semantic for __init__ as in
standard python class (this could be a hard job) ?

This quick fix seems to work with your example, but add extra magic to
automatically call the super __init__ of the parent implementation, this
could be a bad idea, use with caution ! (I still think it's a bad design,
using composition and proxy classes is much more simple and clear)

class DecoratorType(t ype):
def __new__(cls, name, bases, dct):

# create a new class which will store all of the
implementation
parent_impl_typ e = bases[0] is object and object \
or bases[0]._impl_type
impl = type('%sImpl'%n ame,(parent_imp l_type,),dict(d ct))
dectype = type.__new__(cl s, name, bases, {'_impl_type' :
impl })

# update the old class to implement this implementation
def __init__(self, *args, **dargs):
print args, dargs
new_impl = impl(*args, **dargs)
super(dectype._ impl_type, new_impl).__ini t__(*args,
**dargs)
object.__setatt r__(self, '_impl', new_impl)
def decorator(self) :
return object.__getatt ribute__(self,' _impl')
def __getattribute_ _(self, attr):
if attr=="decorato r":
return
object.__getatt ribute__(self,' decorator')
return getattr(object. __getattribute_ _(
self, 'decorator')(), attr)
dectype.__init_ _ = __init__
dectype.decorat or = decorator
dectype.__getat tribute__ = __getattribute_ _

return dectype

class Decorator(objec t):

__metaclass__ = DecoratorType

class HBar(Decorator) :
def __init__(self, number):
print 'hb:', number
self._number = number
def inc(self):
self._number += 1
def p(self):
print self._number

class HBar2(HBar) :
def __init__(self, number):
print 'hb2:', number
self._hb2 = number
def inc2(self):
self._hb2 += 1
def p2(self):
print self._hb2

hbar = HBar(10)
for each in dir(hbar.decora tor()):
print each

hbar.decorator( ).p()
hbar.decorator( ).inc()
hbar.decorator( ).p()

hb2 = HBar2(5)
hb2.p()
hb2.p2()
hb2.inc()
hb2.p()
hb2.p2()
hb2.inc2()
hb2.p()
hb2.p2()

--
_____________

Maric Michaud
_____________

**Carl Banks** · Jun 27 '08, 04:26 PM

Re: Decorator metaclass

On May 22, 10:28 pm, thomas.karol... @googlemail.com wrote:

Hi,
I would like to create a Decorator metaclass, which automatically
turns a class which inherits from the "Decorator" type into a
decorator.
A decorator in this case, is simply a class which has all of its
decorator implementation inside a decorator() method. Every other
attribute access is being proxied to decorator().get Parent().
>
Here's my attempt:

You got deep stuff going on there, chief, and some of it's wrong.
I'll try to point it out.

-------------------------------------------------------
from new import classobj
>
class DecoratorType(t ype):
def __new__(cls, name, bases, dct):
dct2 = {}
for key, val in dct.iteritems() :
dct2[key] = val

First of all, you can just do dct2 = dct.copy().
Second, since you never use dct again, even copying it is unnecessary.

# create a new class which will store all of the implementation
impl = classobj('%sImp l'%name,(),dct2 )

classobj creates an old-style class, and I'm pretty sure you don't
want to do that. To create a new-style class, use type:

impl = type('%sImpl'%n ame,(),dct)

# update the old class to implement this implementation
def __init__(self, *args, **dargs):
object.__setatt r__(self, '__impl', impl(*args, **dargs))

As your code stands now, object.__setatt r__ isn't necessary here; just
using

self.__impl = impl(*args,**da rgs)

should work fine. I'm guessing you intend to override __setattr__
later?

If you do use object.__setatt r__, I suggest that you might want to
call the superclass's __setattr__ instead of object's. I imagine in
this case the superclass will rarely want to override __setattr__
itself, but in general it's a good idea. In this particular
circumstance, we don't yet have the class object (it doesn't come till
after calling type.__new__) but we do have the parent class. So you
might consider changing the definition of __init__ to this:

basecls = bases[0] if bases else object
def __init__(self, *args, **dargs):
basecls.__setat tr__(self, '__impl', impl(*args, **dargs))

def decorator(self) :
return object.__getatt ribute__(self,' __impl')

Again, consider changing it to

def decorator(self) :
return basecls.__getat tribute(self,'_ _impl')

def __getattribute_ _(self, attr):
if attr=="decorato r":
return object.__getatt ribute__(self,' decorator')
return getattr(object. __getattribute_ _(self, 'decorator')
().getParent(), attr)
dct = {}
dct['__init__'] = __init__
dct['decorator'] = decorator
dct['__getattribute __'] = __getattribute_ _
>
return type.__new__(cl s, name, bases, dct)
>
class Decorator(objec t):
__metaclass__ = DecoratorType

Parenthetical: I don't normally recommend this style, since it
obscures the fact that you're using a custom metaclass to the user.
That is something the user probably would benefit from knowing, if for
no other reason than so they can make a mental note about where to
look first if something goes wrong. I prefer to make the user use the
__metaclass__ attribute.

However, I could see it being desirable for some cases where you're
trying to be as transparent as possible, and indeed it looks as if
that's your goal here.

class HBar(Decorator) :
def __init__(self, number):
Decorator.__ini t__(self)

Ok, at this point you have to ask yourself what you want to do,
because the solution you choose will involve trade-offs.

You will note that Decorator does not define __init__. In fact,
object.__init__ will be called, which does nothing. If you think that
all classes with DecoratorType as their metaclass will be a direct
subclass of Decorator, you can get away with not calling
Decorator.__ini t__ at all.

However, this can cause problems if a user wants to define their own
base class with an __init__ that does something (either by using the
__metaclass__ attribute, or by subclassing a Decorator subclass). In
that case, you will have to make arrangements to pass the decorator
object to the superclass instead of the decorated. This can be pretty
hairy, and it beyond the scope of this reply.

To do it completely transparently, your decorated class will probably
have to maintain a reference to its decorator, and will also have to
derive from a base class that delegates any method calls to the
superclass of the decorator. (Phew.)

That won't be as easy as it sounds.

self._number = number
def inc(self):
self._number += 1
def p(self):
print self._number
>
hbar = HBar(10)
for each in dir(hbar.decora tor()):
print each
>
hbar.decorator( ).p()
hbar.decorator( ).inc()
hbar.decorator( ).p()
-------------------------------------------------------
Unfortunately this does not work. The newly defined __init__ method
inside __new__, does a call to impl(*args, **dargs). However, since
the HBar.__init__ calls the Decorator.__ini t__ method, but the
HBar.__init__ method no longer resides inside HBar, but rather inside
HBarImpl (which is no longer a subtype of Decorator), the compiler
complains that Decorator.__ini t__ is not being called with a Decorator
instance as its first argument (which is true).
I tried changing the definition of impl inside __new__ to have
Decorator as one of its bases, but then for some reason impl(*args,
**dargs) asks for 4 arguments (just like __new__) and I have no clue
as to why that happens.

I believe it's happening because you mixed old-style and new-style
classes. But it's not the right solution anyway.

Any help on this?

Probably the best piece of advice is "Don't try to use Decorator
pattern". :)

Seriously, you might want to see what other people have done in
similar cases. This stuff is tricky to get right, so maybe you should
shamelessly ride the coattails of someone who already ran into all the
issues. One example I can think of is the ZODB Persistent class (it's
a proxy class, so some of the same issues are involved). Perhaps
searching Python cookbook for some proxy or decorator class recipes
will give you ideas.

Carl Banks

**Thomas Karolski** · Jun 27 '08, 04:26 PM

Re: Decorator metaclass

Thanks for pointing out all those mistakes. I think I'm already starting
to grasp all of the python magic going on in there.

Parenthetical: I don't normally recommend this style, since it
obscures the fact that you're using a custom metaclass to the user.
That is something the user probably would benefit from knowing, if for
no other reason than so they can make a mental note about where to
look first if something goes wrong. I prefer to make the user use the
__metaclass__ attribute.

Really just personal preference I think. I'm not really a friend of
declaring variables if there is a more "intuitive" way.

>class HBar(Decorator) :
> def __init__(self, number):
> Decorator.__ini t__(self)

>
Ok, at this point you have to ask yourself what you want to do,
because the solution you choose will involve trade-offs.

Yes, it was probably a bad example. I decided not to call the Decorator
's __init__ method in my new version (which I have posted as a reply to
the reply of Maric Michaud).

You will note that Decorator does not define __init__. In fact,
object.__init__ will be called, which does nothing. If you think that
all classes with DecoratorType as their metaclass will be a direct
subclass of Decorator, you can get away with not calling
Decorator.__ini t__ at all.

Now, inside my new version, I have a class which inherits from both
Decorator and Window, out of which the __init__ for Decorator is not
called. Does this prove to be a problem?

Probably the best piece of advice is "Don't try to use Decorator
pattern". :)

Well, I decided on the decorator pattern, because I want to be able to
change the behavior of classes during run-time. I did not really find
any other pattern which would let me do that.

Regards,
Thomas K.

**Carl Banks** · Jun 27 '08, 04:26 PM

Re: Decorator metaclass

On May 23, 11:42 am, Thomas Karolski <Thomas.Karol.. .@googlemail.co m>
wrote:

You will note that Decorator does not define __init__. In fact,
object.__init__ will be called, which does nothing. If you think that
all classes with DecoratorType as their metaclass will be a direct
subclass of Decorator, you can get away with not calling
Decorator.__ini t__ at all.

>
Now, inside my new version, I have a class which inherits from both
Decorator and Window, out of which the __init__ for Decorator is not
called. Does this prove to be a problem?

It sounds like it could create infinite recursion. If your decorated
class inherits from Decorator, it should also get the DecorateType
metaclass, which should recursively try to create its own decorated
class, ad infinitum. Unless I misunderstand what you meant.

Probably the best piece of advice is "Don't try to use Decorator
pattern". :)

>
Well, I decided on the decorator pattern, because I want to be able to
change the behavior of classes during run-time. I did not really find
any other pattern which would let me do that.

Ah.

Well it is possible to do that in Python. though there's probably not
an official design pattern for it (but then design patterns grew up
around less flexible languages, partially as a way to cope with their
lack of flexibility). Here are a couple things to think about:

If you'd like to change the behavior of individual instances of a
class, you can assign functions to individual instances which act just
like methods. (Warning: this does not work with operators.)

Here is an example:

class A(object):
def say_hello(self) :
print "hello, world"

a = A()

def custom_hello():
print "hello, my name is Inigo Montoya"

a.say_hello = custom_hello

If you'd like to change the behavior of all instances of the class,
then you can assign a new method directly to the class after it was
created:

def new_default_hel lo(self):
print "hello, children"

A.say_hello = new_default_hel lo

Notice that you need to use self when assigning it to the class
object, and not to use self when assigning it to an instance of the
class.

Carl Banks

Decorator metaclass

Decorator metaclass

Comment

Comment

Comment

Comment