Access to static members from inside a method decorator?

[glen.coates.bigworld@gmail.com]

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.

At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMetho ds(), a la:

class Exposed:
@staticmethod
def expose( f ):
...

def getExposedMetho ds( self ):
...

class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...

I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMetho ds() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there). The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.

So how can this be done? It doesn't seem like it's possible to pass a
reference to the enclosing class into the decorator, which in turn
means that static tracking of the list of exposed methods is impossible
(at least, if I want to use decorators).

Any ideas that will enable my initial design, or suggestions for an
elegant, workable alternative would be much appreciated.

Cheers,
Glen

[Bruno Desthuilliers]

Re: Access to static members from inside a method decorator?

glen.coates.big world@gmail.com wrote:

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.
>
At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMetho ds(), a la:
>
class Exposed:
@staticmethod
def expose( f ):
...
>
def getExposedMetho ds( self ):
...
>
class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...
>
I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMetho ds() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there).

Not yet. When your decorator is called, the class object is not yet
created, and what you are decorating is a plain function.

The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.

Exactly.

So how can this be done?

The simplest thing is to use a two-stages scheme : mark the functions as
exposed, then collect them:

def expose(func):
func._exposed = True
return func

def exposed(obj):
return callable(obj) and getattr(obj, '_exposed', False)

class Exposing(object ):
@classmethod
def get_exposed_met hods(cls):
try:
exposeds = cls._exposed_me thods
except AttributeError:
exposeds = []
for name in dir(cls):
obj = getattr(cls, name)
if exposed(obj):
exposeds.append (obj)
cls._exposed_me thods = exposeds
return exposeds

class Parrot(Exposing ):
@expose
def parrot(self, what):
return "%s says %s" % (self, str(what))



HTH
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[glen.coates.bigworld@gmail.com]

Re: Access to static members from inside a method decorator?

Bruno Desthuilliers wrote:

glen.coates.big world@gmail.com wrote:

I'm developing a library at the moment that involves many classes, some
of which have "exposed" capabilities. I'm trying to design a nice
interface for both exposing those capabilities, and inspecting
instances to find out what capabilities they have.

At the moment, I'm leaning towards a superclass (Exposed) that defines
a static method which is a decorator (expose) such that any derived
class can mark a method with @Exposed.expose and it will then be later
returned by getExposedMetho ds(), a la:

class Exposed:
@staticmethod
def expose( f ):
...

def getExposedMetho ds( self ):
...

class Person( Exposed ):
@Exposed.expose
def talk( self, ... ):
...

I'm trying to implement the decorator by having it populate a static
member list of whatever class it's in with a reference to the method.
getExposedMetho ds() would then return the contents of each of those
lists from itself back to Exposed in the class hierarchy. The first
problem was that having a reference to the method (i.e. talk()) does
not allow you to get a reference to the enclosing class (I had hoped
im_class would lead me there).

>
Not yet. When your decorator is called, the class object is not yet
created, and what you are decorating is a plain function.
>

The real hiccup was that explicitly
passing the class as an argument to the decorator generates a undefined
global name error, presumably because at that point of execution the
class object hasn't been fully created/initialised.

>
Exactly.
>

So how can this be done?

>
The simplest thing is to use a two-stages scheme : mark the functions as
exposed, then collect them:
>
def expose(func):
func._exposed = True
return func
>
def exposed(obj):
return callable(obj) and getattr(obj, '_exposed', False)
>
class Exposing(object ):
@classmethod
def get_exposed_met hods(cls):
try:
exposeds = cls._exposed_me thods
except AttributeError:
exposeds = []
for name in dir(cls):
obj = getattr(cls, name)
if exposed(obj):
exposeds.append (obj)
cls._exposed_me thods = exposeds
return exposeds
>
class Parrot(Exposing ):
@expose
def parrot(self, what):
return "%s says %s" % (self, str(what))
>
>
>
HTH
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

Thanks Bruno. I came up with a similar solution today at work, which
involves an 'init' method which is called at the bottom of each module
that defines subclasses of Exposed and sets up static mappings for the
exposed methods. I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

Thanks for the help though,
Glen

[Maric Michaud]

Re: Access to static members from inside a method decorator?

Le jeudi 05 octobre 2006 17:18, glen.coates.big world@gmail.com a écrit :

I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

You can always replace the need of the init method on classes using a
metaclass.

This demonstrates it with Bruno's expose decorator, but it can be done with
your actual init func too.

In [6]: class m(type) :
...: def __init__(self, *a,**kw) :
...: for name, meth in self.__dict__.i tems() :
...: if getattr(meth, '_exposed', False) :
...: print 'exposed :', name
...:
...:

In [7]: class a(object):
...: __metaclass__ = m
...: def f(self) :pass
...: @expose
...: def g(self) :pass
...:
...:
exposed : g

In [8]: class b(a) :
...: @expose
...: def h(self) :pass
...:
...:
exposed : h




--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

[Bruno Desthuilliers]

Re: Access to static members from inside a method decorator?

glen.coates.big world@gmail.com wrote:

Bruno Desthuilliers wrote:

(snip)

>class Exposing(object ):
> @classmethod
> def get_exposed_met hods(cls):
> try:
> exposeds = cls._exposed_me thods
> except AttributeError:
> exposeds = []
> for name in dir(cls):
> obj = getattr(cls, name)
> if exposed(obj):
> exposeds.append (obj)
> cls._exposed_me thods = exposeds
> return exposeds
>>
>class Parrot(Exposing ):
> @expose
> def parrot(self, what):
> return "%s says %s" % (self, str(what))

>
Thanks Bruno. I came up with a similar solution today at work, which
involves an 'init' method which is called at the bottom of each module
that defines subclasses of Exposed and sets up static mappings for the
exposed methods. I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with,

This is only a problem in a framework-style use.

however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

I think you have missed a point in my solution : the 'collect' pass
happens only once for each class - the first time the
get_exposed_met hods is called on the class (or any instance of). It's
then memoized for latter calls.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[Bruno Desthuilliers]

Re: Access to static members from inside a method decorator?

Maric Michaud wrote:

Le jeudi 05 octobre 2006 17:18, glen.coates.big world@gmail.com a écrit :

>I guess my solution is slightly less elegant because
>it requires this ugly explicit init call outside the classes that it
>actually deals with, however it is more efficient because the dir()
>pass happens once on module load, instead of every time I want the list
>of exposed methods.

>
You can always replace the need of the init method on classes using a
metaclass.
>
This demonstrates it with Bruno's expose decorator, but it can be done with
your actual init func too.
>
In [6]: class m(type) :
...: def __init__(self, *a,**kw) :
...: for name, meth in self.__dict__.i tems() :

NB : This will only get methods exposeds in this class - not the one
exposeds in the parent classes...

...: if getattr(meth, '_exposed', False) :
...: print 'exposed :', name
...:
...:
>

(snip)


--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[Steve Holden]

Re: Access to static members from inside a method decorator?

glen.coates.big world@gmail.com wrote:

Bruno Desthuilliers wrote:
>

>>glen.coates.b igworld@gmail.c om wrote:
>>

>>>I'm developing a library at the moment that involves many classes, some
>>>of which have "exposed" capabilities. I'm trying to design a nice
>>>interface for both exposing those capabilities, and inspecting
>>>instances to find out what capabilities they have.
>>>
>>>At the moment, I'm leaning towards a superclass (Exposed) that defines
>>>a static method which is a decorator (expose) such that any derived
>>>class can mark a method with @Exposed.expose and it will then be later
>>>returned by getExposedMetho ds(), a la:
>>>
>>>class Exposed:
>> @staticmethod
>> def expose( f ):
>> ...
>>>
>> def getExposedMetho ds( self ):
>> ...
>>>
>>>class Person( Exposed ):
>> @Exposed.expose
>> def talk( self, ... ):
>> ...
>>>
>>>I'm trying to implement the decorator by having it populate a static
>>>member list of whatever class it's in with a reference to the method.
>>>getExposedMe thods() would then return the contents of each of those
>>>lists from itself back to Exposed in the class hierarchy. The first
>>>problem was that having a reference to the method (i.e. talk()) does
>>>not allow you to get a reference to the enclosing class (I had hoped
>>>im_class would lead me there).

>>
>>Not yet. When your decorator is called, the class object is not yet
>>created, and what you are decorating is a plain function.
>>
>>

>>>The real hiccup was that explicitly
>>>passing the class as an argument to the decorator generates a undefined
>>>global name error, presumably because at that point of execution the
>>>class object hasn't been fully created/initialised.

>>
>>Exactly.
>>
>>

>>>So how can this be done?

>>
>>The simplest thing is to use a two-stages scheme : mark the functions as
>>exposed, then collect them:
>>
>>def expose(func):
> func._exposed = True
> return func
>>
>>def exposed(obj):
> return callable(obj) and getattr(obj, '_exposed', False)
>>
>>class Exposing(object ):
> @classmethod
> def get_exposed_met hods(cls):
> try:
> exposeds = cls._exposed_me thods
> except AttributeError:
> exposeds = []
> for name in dir(cls):
> obj = getattr(cls, name)
> if exposed(obj):
> exposeds.append (obj)
> cls._exposed_me thods = exposeds
> return exposeds
>>
>>class Parrot(Exposing ):
> @expose
> def parrot(self, what):
> return "%s says %s" % (self, str(what))
>>
>>
>>
>>HTH
>>--
>>bruno desthuilliers
>>python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
>>p in 'onurb@xiludom. gro'.split('@')])"

>
>
Thanks Bruno. I came up with a similar solution today at work, which
involves an 'init' method which is called at the bottom of each module
that defines subclasses of Exposed and sets up static mappings for the
exposed methods. I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.
>

Surely the right place to handle "collection " is in a metaclass, where
the metaclass's __call__() method can scan the __dict__ and take
appropriate action on the marked methods? That way it's done just once,
at class definition time, as it should be.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

[glen.coates.bigworld@gmail.com]

Re: Access to static members from inside a method decorator?

Thanks for all the help guys ... in almost every way using a metaclass
seems to be the right solution for what I'm trying to do here. I say
almost because there is one thing that is still confusing me: what is
the most elegant way to provide base-class implementations of methods
that are expected to be overriden by some of the derived classes (or in
the case of metaclasses, the classes that either declare __metaclass__
= Exposed or are derived from such classes).

Here's what I've just knocked out:

import sys

class Exposed( type ):

def __init__( cls, *args, **kw ):

# Track marked exposed methods
cls.s_exposedMe thods = []
for superclass in cls.__mro__:
for name, meth in superclass.__di ct__.items():
if hasattr( meth, "exposed" ):
cls.s_exposedMe thods.append( name )

# Add getExposedMetho ds method
cls.getExposedM ethods = lambda self: self.s_exposedM ethods
cls.bar = lambda self: sys.stdout.writ e( "bar\n" )

@staticmethod
def expose( f ):
f.exposed = True
return f


class Process( object ):

__metaclass__ = Exposed

@Exposed.expose
def foo( self ):
pass


def bar( self ):
print "BAR"
pass


class BotProcess( Process ):

@Exposed.expose
def addBots( self ):
pass

p = Process()
p.bar()

#############

The problem here is that the implementation of 'bar' inside
Exposed.__init_ _ overrides the implementation of bar() in Process,
which makes sense I guess seeing as Exposed.__init_ _() is called after
the class has been initialised. It's not a problem for
getExposedMetho ds() seeing as it's not overriden by any derived
classes.

So what is the accepted way of doing this? Do I need two Exposed
classes, one is the metaclass that handles all the static mapping
stuff, and another provides base implementations of methods and is what
is actually derived from? E.g.:

class ExposedMeta( type ):
...

class Exposed( object ):
...

class Process( Exposed ):
__metaclass__ = ExposedMeta
...

class BotProcess( Process ):
...

Hmmm ... that seems a bit ugly too. If I change the assignments in
Exposed.__init_ _() I guess I can avoid the two-class thing:

cls.getExposedM ethods = getattr( cls, "getExposedMeth ods", lambda
self: self.s_exposedM ethods )
cls.bar = getattr( cls, "bar", lambda self: sys.stdout.writ e( "bar\n"
) )

That's better, but still ugly. Is there a better way?

Thanks for all the help thus far guys,
Glen

Maric Michaud wrote:

Le jeudi 05 octobre 2006 17:18, glen.coates.big world@gmail.com a écrit :

I guess my solution is slightly less elegant because
it requires this ugly explicit init call outside the classes that it
actually deals with, however it is more efficient because the dir()
pass happens once on module load, instead of every time I want the list
of exposed methods.

>
You can always replace the need of the init method on classes using a
metaclass.
>
This demonstrates it with Bruno's expose decorator, but it can be done with
your actual init func too.
>
In [6]: class m(type) :
...: def __init__(self, *a,**kw) :
...: for name, meth in self.__dict__.i tems() :
...: if getattr(meth, '_exposed', False) :
...: print 'exposed :', name
...:
...:
>
In [7]: class a(object):
...: __metaclass__ = m
...: def f(self) :pass
...: @expose
...: def g(self) :pass
...:
...:
exposed : g
>
In [8]: class b(a) :
...: @expose
...: def h(self) :pass
...:
...:
exposed : h
>
>
>
>
--
_____________
>
Maric Michaud
_____________
>
Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

[Peter Otten]

Re: Access to static members from inside a method decorator?

glen.coates.big world@gmail.com wrote:

Thanks for all the help guys ... in almost every way using a metaclass
seems to be the right solution for what I'm trying to do here. I say
almost because there is one thing that is still confusing me: what is
the most elegant way to provide base-class implementations of methods
that are expected to be overriden by some of the derived classes (or in
the case of metaclasses, the classes that either declare __metaclass__
= Exposed or are derived from such classes).
>
Here's what I've just knocked out:

[snip example]

The problem here is that the implementation of 'bar' inside
Exposed.__init_ _ overrides the implementation of bar() in Process,
which makes sense I guess seeing as Exposed.__init_ _() is called after
the class has been initialised. It's not a problem for
getExposedMetho ds() seeing as it's not overriden by any derived
classes.
>
So what is the accepted way of doing this? Do I need two Exposed
classes, one is the metaclass that handles all the static mapping
stuff, and another provides base implementations of methods and is what
is actually derived from? E.g.:

You define one base type with a custom metaclass and inherit from that. Your
example then becomes:

import sys

class ExposedType( type ):
def __init__( cls, *args, **kw ):
# Track marked exposed methods
cls.s_exposedMe thods = []
for superclass in cls.__mro__:
for name, meth in superclass.__di ct__.items():
if hasattr( meth, "exposed" ):
cls.s_exposedMe thods.append( name )

class Exposed:
__metaclass__ = ExposedType
def getExposedMetho ds(self):
return self.s_exposedM ethods
def bar(self):
print "bar\n"
@staticmethod
def expose( f ):
f.exposed = True
return f

class Process( Exposed):
@Exposed.expose
def foo( self ):
pass
def bar( self ):
print "BAR"


class BotProcess( Process ):
@Exposed.expose
def addBots( self ):
pass

p = Process()
p.bar()

This prints "BAR" as expected.

Peter

[urielka]

Re: Access to static members from inside a method decorator?

no need for all that,i wrote a basic Ajax framework for cherrypy that
features a Ajax.Net feature,exposin g functions to JavaScript via
attributes(or in python via decorators),her e is a decorator that run
one time(i.e. before running the actual code) and get the name of the
class

def AddFunction(func):
stack=inspect.stack()
ClsName=stack[1][3]#get the class name from the stack
#now you both have the ClassName and the func object,so you can
populate your list
return func#return the decorated function

it use the module inspect,to inspect the stack.

[glen.coates.bigworld@gmail.com]

Re: Access to static members from inside a method decorator?

Peter Otten wrote:

glen.coates.big world@gmail.com wrote:
>
You define one base type with a custom metaclass and inherit from that. Your
example then becomes:
>
import sys
>
class ExposedType( type ):
def __init__( cls, *args, **kw ):
# Track marked exposed methods
cls.s_exposedMe thods = []
for superclass in cls.__mro__:
for name, meth in superclass.__di ct__.items():
if hasattr( meth, "exposed" ):
cls.s_exposedMe thods.append( name )
>
class Exposed:
__metaclass__ = ExposedType
def getExposedMetho ds(self):
return self.s_exposedM ethods
def bar(self):
print "bar\n"
@staticmethod
def expose( f ):
f.exposed = True
return f
>
class Process( Exposed):
@Exposed.expose
def foo( self ):
pass
def bar( self ):
print "BAR"
>
>
class BotProcess( Process ):
@Exposed.expose
def addBots( self ):
pass
>
p = Process()
p.bar()
>
This prints "BAR" as expected.
>
Peter

Thanks Peter. Yeah I had thought of that earlier, but wasn't sure if
this is a standard design pattern for what I'm trying to achieve. It
seems ugly to me to use 2 classes when you are essentially describing a
single type. Is the Splat/SplatType pairing really a common design
pattern when trying to use metaclasses in this way?

Also ... as for the 'inspect' based solution, yeah I can see that would
work, but it seems very hacky to me and my gut reaction is to avoid
that kind of thing ...

Cheers,
Glen

[Peter Otten]

Re: Access to static members from inside a method decorator?

glen.coates.big world@gmail.com wrote:

Thanks Peter. Yeah I had thought of that earlier, but wasn't sure if
this is a standard design pattern for what I'm trying to achieve. It
seems ugly to me to use 2 classes when you are essentially describing a
single type.

To me both Exposed and ExposedType look more idiomatic than your combined
approach.

Is the Splat/SplatType pairing really a common design
pattern when trying to use metaclasses in this way?

I think so, but look for yourself:

http://www.google.com/codesearch?q=lang%3Apython+__metaclass__&btnG=Search+Code

Refinements to that query welcome...

Peter