Per instance descriptors ?

[bruno at modulix]

Hi

I'm currently playing with some (possibly weird...) code, and I'd have a
use for per-instance descriptors, ie (dummy code):

class DummyDescriptor (object):
def __get__(self, obj, objtype=None):
if obj is None:
return self
return getattr(obj, 'bar', 'no bar')

class MyClass1(object ):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
self.baaz = DummyDescriptor ()

mc1 = MyClass1(bar='b ack')
mc1.baaz
-> <__main__.Dummy Descriptor object at 0x2aaaabc6c390>

Which is of course what one would expect... Now I tried the following
hack^Mworkaroun d:

class MyClass2(MyClas s1):
def __getattribute_ _(self, key):
v = MyClass1.__geta ttribute__(self , key)
if hasattr(v, '__get__'):
return v.__get__(self, self.__class__)
return v

And it *seems* to work just fine:

mc2 = MyClass2(bar='f oo')
mc2.baaz
-> 'foo'

Now the question: is there any obvious (or non-obvious) drawback with
this approach ?

A bit of context:
1/ the real class is a decorator for controller functions in a
mod_python application, and given previous experiences with mod_python,
I'd prefer not mess with the class itself at runtime... still I'd like
to abstract some gory details, and descriptors would be an obvious
choice here.

2/ this is for production code, so robustness is more important than
syntactic sugar - but having this syntactic sugar would be nice...

Any hint ?

TIA
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[Ziga Seilnacht]

Re: Per instance descriptors ?


bruno at modulix wrote:[color=blue]
> Hi
>
> I'm currently playing with some (possibly weird...) code, and I'd have a
> use for per-instance descriptors, ie (dummy code):[/color]

<snip>
[color=blue]
> Now the question: is there any obvious (or non-obvious) drawback with
> this approach ?[/color]

Staticmethods won't work anymore:
[color=blue][color=green][color=darkred]
>>> class Test(object):[/color][/color][/color]
.... @staticmethod
.... def foo():
.... pass
.... def __getattribute_ _(self, name):
.... v = object.__getatt ribute__(self, name)
.... if hasattr(v, '__get__'):
.... return v.__get__(self, self.__class__)
.... return v
....[color=blue][color=green][color=darkred]
>>> test = Test()
>>> test.foo()[/color][/color][/color]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: foo() takes no arguments (1 given)

[color=blue]
> TIA
> --
> bruno desthuilliers
> python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
> p in 'onurb@xiludom. gro'.split('@')])"[/color]

Ziga

[bruno at modulix]

Re: Per instance descriptors ?

Ziga Seilnacht wrote:[color=blue]
> bruno at modulix wrote:
>[color=green]
>>Hi
>>
>>I'm currently playing with some (possibly weird...) code, and I'd have a
>>use for per-instance descriptors, ie (dummy code):[/color]
>
>
> <snip>
>[color=green]
>>Now the question: is there any obvious (or non-obvious) drawback with
>>this approach ?[/color]
>
>
> Staticmethods won't work anymore:
>
>[color=green][color=darkred]
>>>>class Test(object):[/color][/color]
>
> ... @staticmethod
> ... def foo():
> ... pass
> ... def __getattribute_ _(self, name):
> ... v = object.__getatt ribute__(self, name)
> ... if hasattr(v, '__get__'):
> ... return v.__get__(self, self.__class__)
> ... return v
> ...
>[color=green][color=darkred]
>>>>test = Test()
>>>>test.foo( )[/color][/color]
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> TypeError: foo() takes no arguments (1 given)
>[/color]

Hmmm.... Well, I almost never use staticmethods, but I need to check
this out.

(a few minutes later)

Ok, no apparent impact on classmethods. For staticmethod, a quick fix is:

import types
[color=blue]
> ... def __getattribute_ _(self, name):
> ... v = object.__getatt ribute__(self, name)
> ... if not isinstance(v, types.FunctionT ype) \[/color]
and hasattr(v, '__get__'):[color=blue]
> ... return v.__get__(self, self.__class__)
> ... return v[/color]


Thanks Ziga.


Anyone else ? Any good reason to *not* do this ? Or is presumably safe ?
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[Michael Spencer]

Re: Per instance descriptors ?

bruno at modulix wrote:[color=blue]
> Ziga Seilnacht wrote:[color=green]
>> bruno at modulix wrote:
>>[color=darkred]
>>> Hi
>>>
>>> I'm currently playing with some (possibly weird...) code, and I'd have a
>>> use for per-instance descriptors, ie (dummy code):[/color]
>>
>> <snip>
>>[color=darkred]
>>> Now the question: is there any obvious (or non-obvious) drawback with
>>> this approach ?[/color]
>>[/color][/color]
....[color=blue]
>[color=green]
>> ... def __getattribute_ _(self, name):
>> ... v = object.__getatt ribute__(self, name)
>> ... if not isinstance(v, types.FunctionT ype) \[/color]
> and hasattr(v, '__get__'):[color=green]
>> ... return v.__get__(self, self.__class__)
>> ... return v[/color]
>
>[/color]
I may be missing the subtlety of what you're up to, but why is overriding
__getattribute_ _ more desirable than simply defining the descriptor in a subclass?
i.e.,
class MyClass3(MyClas s1):
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
baaz = DummyDescriptor ()


Michael

[Bruno Desthuilliers]

Re: Per instance descriptors ?

Michael Spencer a écrit :[color=blue]
> bruno at modulix wrote:
>[color=green]
>> Ziga Seilnacht wrote:
>>[color=darkred]
>>> bruno at modulix wrote:
>>>
>>>> Hi
>>>>
>>>> I'm currently playing with some (possibly weird...) code, and I'd
>>>> have a
>>>> use for per-instance descriptors, ie (dummy code):
>>>
>>>
>>> <snip>
>>>
>>>> Now the question: is there any obvious (or non-obvious) drawback with
>>>> this approach ?
>>>
>>>[/color][/color]
> ...
>[color=green]
>>[color=darkred]
>>> ... def __getattribute_ _(self, name):
>>> ... v = object.__getatt ribute__(self, name)
>>> ... if not isinstance(v, types.FunctionT ype) \[/color]
>>
>> and hasattr(v, '__get__'):
>>[color=darkred]
>>> ... return v.__get__(self, self.__class__)
>>> ... return v[/color]
>>
>>
>>[/color]
> I may be missing the subtlety of what you're up to, but why is
> overriding __getattribute_ _ more desirable than simply defining the
> descriptor in a subclass?[/color]

The code snippet I gave as an example was not supposed to reflect how I
effectively mean to use per-instance descriptors, it was just a kind of
Minimal Working Code (tm). The real life code is about 500 LOC, and
explaining the whole thing would take far too long. Also, as I said,
this is mostly syntactic sugar - there are simpler, less 'hackish' (but
also less elegant) solutions to the actual 'problem'. So, to answer your
question, no, subclassing would not be a solution - I'd almost need a
subclass per controller function, which would reintroduce the
boilerplate I'm trying to get rid of.

BTW, there may be other use case for per-instance descriptors... Python
is so dynamic that you can almost use it like a prototype-based language.


(snip code)

[Steven Bethard]

Re: Per instance descriptors ?

bruno at modulix wrote:[color=blue]
> Hi
>
> I'm currently playing with some (possibly weird...) code, and I'd have a
> use for per-instance descriptors, ie (dummy code):
>
> class DummyDescriptor (object):
> def __get__(self, obj, objtype=None):
> if obj is None:
> return self
> return getattr(obj, 'bar', 'no bar')
>
> class MyClass1(object ):
> def __init__(self, bar=None):
> if bar is not None:
> self.bar = bar
> self.baaz = DummyDescriptor ()
>
> mc1 = MyClass1(bar='b ack')
> mc1.baaz
> -> <__main__.Dummy Descriptor object at 0x2aaaabc6c390>
>
> Which is of course what one would expect... Now I tried the following
> hack^Mworkaroun d:
>
> class MyClass2(MyClas s1):
> def __getattribute_ _(self, key):
> v = MyClass1.__geta ttribute__(self , key)
> if hasattr(v, '__get__'):
> return v.__get__(self, self.__class__)
> return v
>
> And it *seems* to work just fine:
>
> mc2 = MyClass2(bar='f oo')
> mc2.baaz
> -> 'foo'
>
> Now the question: is there any obvious (or non-obvious) drawback with
> this approach ?[/color]

Don't know if this matters, but if you override __getattribute_ _, you'll
slow down all attribute accesses to this object. If this matters, you
could write something like:

class MyClass(object) :
def __init__(self, bar=None):
if bar is not None:
self.bar = bar
def __getattr__(sel f, name):
if name == 'baaz':
return self.bar
elif name == 'bar':
return 'no bar'

Then you only incur the penalty on the ``baaz`` lookup and the ``bar``
lookup when it's missing -- not on all attribute lookups.

Could you explain again why you don't want baaz to be a class-level
attribute?

STeVe

[Michael Spencer]

Re: Per instance descriptors ?

Bruno Desthuilliers wrote:[color=blue]
> Michael Spencer a écrit :[/color]
[color=blue][color=green]
>> I may be missing the subtlety of what you're up to, but why is
>> overriding __getattribute_ _ more desirable than simply defining the
>> descriptor in a subclass?[/color]
>
> The code snippet I gave as an example was not supposed to reflect how I
> effectively mean to use per-instance descriptors, it was just a kind of
> Minimal Working Code (tm). The real life code is about 500 LOC, and
> explaining the whole thing would take far too long. Also, as I said,
> this is mostly syntactic sugar - there are simpler, less 'hackish' (but
> also less elegant) solutions to the actual 'problem'.So, to answer your
> question, no, subclassing would not be a solution - I'd almost need a
> subclass per controller function, which would reintroduce the
> boilerplate I'm trying to get rid of.
>[/color]
[color=blue]
> BTW, there may be other use case for per-instance descriptors...[/color]

Agreed. Per-instance descriptors could be interesting (that's why the subject
line caught my attention).
But your solution involves a custom __getattribute_ _ in the class, which I would
always avoid if possible (and I gather you're uneasy about it too).
Here, I don't see why that's better than having a descriptor in the class and,
if it needs per-instance behavior, then make it dependent on something provided
by the instance.

e.g.,
class DummyDescriptor 1(object):
def __get__(self, obj, objtype=None):
if isinstance(obj, objtype):
return obj.foo.__get__ (obj)
else:
return self

class MyClass4(object ):
baaz = DummyDescriptor 1()
def __init__(self, foo, bar = None):
self.foo = foo
self.bar = bar
[color=blue][color=green][color=darkred]
>>> mc4 = MyClass4(lambda self: self.bar, "I'm bar")
>>> mc4.baaz
>>> mc4.baaz()[/color][/color][/color]
"I'm bar"[color=blue][color=green][color=darkred]
>>> mc5 = MyClass4(lambda self: "I ignore bar", "I'm another bar")
>>> mc5.baaz()[/color][/color][/color]
"I ignore bar"[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]


Python[color=blue]
> is so dynamic that you can almost use it like a prototype-based language.
>
>[/color]
Almost, yes.

Michael

[bruno at modulix]

Re: Per instance descriptors ?

Michael Spencer wrote:[color=blue]
> Bruno Desthuilliers wrote:
>[/color]
(snip)[color=blue]
>[color=green]
>> BTW, there may be other use case for per-instance descriptors...[/color]
>
>
> Agreed. Per-instance descriptors could be interesting (that's why the
> subject line caught my attention).
> But your solution involves a custom __getattribute_ _ in the class, which
> I would always avoid if possible (and I gather you're uneasy about it too).[/color]

I'm not uneasy about overriding __getattribute_ _, just about overriding
it that way -
[color=blue]
> Here, I don't see why that's better than having a descriptor in the
> class and, if it needs per-instance behavior, then make it dependent on
> something provided by the instance.[/color]

Each instance will need it's own set of descriptors. The class is used
as a decorator for many controller functions. The descriptors are used
to encapsulate some gory implementation details about how to get such or
such object from the framework (they of course depend on instance
specific data, but that's not the point).
[color=blue]
> e.g.,
> class DummyDescriptor 1(object):
> def __get__(self, obj, objtype=None):
> if isinstance(obj, objtype):
> return obj.foo.__get__ (obj)
> else:
> return self
>
> class MyClass4(object ):
> baaz = DummyDescriptor 1()
> def __init__(self, foo, bar = None):
> self.foo = foo
> self.bar = bar[/color]

This would imply a decorator subclass and a descriptor subclass for each
and every controller function - which is what I'm trying to avoid.

(snip)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[bruno at modulix]

Re: Per instance descriptors ?

Steven Bethard wrote:[color=blue]
> bruno at modulix wrote:
>[color=green]
>> Hi
>>
>> I'm currently playing with some (possibly weird...) code, and I'd have a
>> use for per-instance descriptors,[/color][/color]
(snip)
[color=blue][color=green]
>>
>> class MyClass2(MyClas s1):
>> def __getattribute_ _(self, key):
>> v = MyClass1.__geta ttribute__(self , key)
>> if hasattr(v, '__get__'):
>> return v.__get__(self, self.__class__)
>> return v
>>
>> And it *seems* to work just fine:
>>
>> mc2 = MyClass2(bar='f oo')
>> mc2.baaz
>> -> 'foo'
>>
>> Now the question: is there any obvious (or non-obvious) drawback with
>> this approach ?[/color]
>
>
> Don't know if this matters, but if you override __getattribute_ _, you'll
> slow down all attribute accesses to this object.[/color]

Yes, I know, but this shouldn't be a major annoyance here.
[color=blue]
> If this matters, you
> could write something like:
>
> class MyClass(object) :
> def __init__(self, bar=None):
> if bar is not None:
> self.bar = bar
> def __getattr__(sel f, name):
> if name == 'baaz':
> return self.bar
> elif name == 'bar':
> return 'no bar'[/color]

Don't focus on the dummy example I gave - the real descriptors are doing
something a bit less stupid !-)
[color=blue]
> Could you explain again why you don't want baaz to be a class-level
> attribute?[/color]

Because the class is a decorator for many controller functions, and each
controller function will need it's own set of descriptors, so I don't
want to mess with the class.

Think of the decorator as a prototype, each controller function
customizing it according to it's need - this customisation including
the decorator instance attaching descriptors and methods to itself
according to parameters passed at __init__ time. The decorator instance
also passes itself as first arg to the controller function - which then
practically become an instance method too.

Don't tell me, I know this is a somewhat weird architecture, and it
could mostly be done with more conventional subclassing. BTW, this was
how a first implementation worked, and it required almost twice more
code than the new one I'm experimenting, without being half as flexible.

As I said, it's mostly syntactic sugar, but what, I'm lazy enough to
spend time on writing code that will allow me to write less code in the
end !-)


--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[Steven Bethard]

Re: Per instance descriptors ?

bruno at modulix wrote:[color=blue]
> Steven Bethard wrote:[color=green]
>> Could you explain again why you don't want baaz to be a class-level
>> attribute?[/color]
>
> Because the class is a decorator for many controller functions, and each
> controller function will need it's own set of descriptors, so I don't
> want to mess with the class.[/color]

So you're trying to add property-like attributes to functions? That is,
you want something like:

@my_decorator
def f(...):
...

f.foo # calls f._get_foo()


in another post, bruno at modulix wrote:[color=blue]
> This would imply a decorator subclass and a descriptor subclass for
> each and every controller function - which is what I'm trying to
> avoid.[/color]

So you only want one decorator? Doesn't that mean that all functions
will have the same attributes? But if that were true you would only
need one descriptor for all controller functions, so I must not be
understanding that right.

Can you give a little more realistic code sample? I'm still not sure
what it is you really want to do. Don't worry about showing the
implementation you're thinking of. Just show me how you want to use
these things and what it ought to look like.


STeVe

[bruno at modulix]

Re: Per instance descriptors ?

Steven Bethard wrote:
(some smart questions)

Steven , I owe you a *big* thank.

I knew they must have been something wrong, but couldn't point what. Now
I see, and it's of course totally obvious. Using a class as a
decorator, I have of course only one instance of it per function - and
for some attributes, I need an instance per function call.

Duh :(

Well, at least I will have learn some new things...

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"

[Steven Bethard]

Re: Per instance descriptors ?

bruno at modulix wrote:[color=blue]
> Using a class as a
> decorator, I have of course only one instance of it per function - and
> for some attributes, I need an instance per function call.[/color]

Per function call? And you want the attributes on the function, not the
result of calling the function? If so, that'd be pretty difficult...

I guess you could do something like:
[color=blue][color=green][color=darkred]
>>> class FuncWrapper(obj ect):[/color][/color][/color]
.... def __init__(self, func):
.... self.func = func
.... self.call_no = 0
.... self.get_foo = lambda: 'one thing'
.... def __call__(self, *args, **kwargs):
.... self.call_no += 1
.... if self.call_no == 1:
.... del self.get_foo
.... self.get_bar = lambda: 'another thing'
.... else:
.... del self.get_bar
.... self.get_baz = lambda: 'a third thing'
.... return self.func(*args , **kwargs)
....[color=blue][color=green][color=darkred]
>>> @FuncWrapper[/color][/color][/color]
.... def f(*args, **kwargs):
.... print args, kwargs
....[color=blue][color=green][color=darkred]
>>> f.get_foo()[/color][/color][/color]
'one thing'[color=blue][color=green][color=darkred]
>>> f('test 1')[/color][/color][/color]
('test 1',) {}[color=blue][color=green][color=darkred]
>>> f.get_foo[/color][/color][/color]
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
AttributeError: 'FuncWrapper' object has no attribute 'get_foo'[color=blue][color=green][color=darkred]
>>> f.get_bar()[/color][/color][/color]
'another thing'[color=blue][color=green][color=darkred]
>>> f(test=2)[/color][/color][/color]
() {'test': 2}[color=blue][color=green][color=darkred]
>>> f.get_bar[/color][/color][/color]
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
AttributeError: 'FuncWrapper' object has no attribute 'get_bar'[color=blue][color=green][color=darkred]
>>> f.get_baz()[/color][/color][/color]
'a third thing'

But that looks pretty nasty to me. It sounds like your architecture
could use some redesigning -- having different attributes on a function
for each function call seems like a bad idea. Can't you have the
returned objects carry the different attributes?

STeVe

[bruno at modulix]

Re: Per instance descriptors ?

Steven Bethard wrote:

(snip code)[color=blue]
>
> But that looks pretty nasty to me.[/color]

<aol />
[color=blue]
> It sounds like your architecture
> could use some redesigning[/color]

Done - in much more sane way. Got rid of some more boilerplate and of
the whole problem of per-instance descriptors BTW !-)

I should probably sleep more and program less...

Thanks again, Steven. You've been of great help - even if indirectly.
Remind me to buy you a beer if we ever meet !-)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"