Overloading operators on the rigth.

[denis wendum]

In a nutshell:
What is the equivalent of __radd__ (wich overloads the right hand side
of +) when overloading the comparison operators <,>,== and so on. My
first guess __rlt__ for
overloading the rigth hand side of < did not work.
Expanded version:
In a class of numbers I have been able to overload the four arithmetic
operators +,-,* and / by defining the methods __add__ and __radd__
(etc..) in my class. This enables me to evaluate expressions
"instance_of_my _class+"instanc e_of_my_class" ,"instance + non_instance"
(with __add__) as well as "non_instance+i nstance"(with __radd__).
But my guess to define the method __rlt__ in my class in order to be
able to evaluate expressions "non_instan ce < instance" did not work.
So what are the names (if they exist at all) of the comparison operators
<,>,== and so on, when one wants to exend their rigth hand side to a
user defined (instance of a) class?

In case of direct e-mail response please remove the french translation
of nospam in my signature, that is to say the string :
pasdepourriels.

[Alex Martelli]

Re: Overloading operators on the rigth.

denis wendum <wendum.denis@p asdepourriels.e df.fr> wrote:
[color=blue]
> But my guess to define the method __rlt__ in my class in order to be
> able to evaluate expressions "non_instan ce < instance" did not work.[/color]

Sensible guess, but not the way things work.
[color=blue]
> So what are the names (if they exist at all) of the comparison operators
> <,>,== and so on, when one wants to exend their rigth hand side to a
> user defined (instance of a) class?[/color]

Here's how to find out:
[color=blue][color=green][color=darkred]
>>> class foo:[/color][/color][/color]
.... def __getattr__(sel f, name):
.... print 'looking for %r' % name
.... raise AttributeError, name
....[color=blue][color=green][color=darkred]
>>> f=foo()
>>> 12 < f[/color][/color][/color]
looking for '__gt__'
looking for '__coerce__'
looking for '__cmp__'
True[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

See what's happening here? After determining that int (the type of 12)
cannot deal with comparing 12 with an instance of foo, Python's
implementation of < turns to the right hand side... and asks for a
__gt__ method first! Makes sense, because 12 < f is presumably going to
be the same thing as f > 12 ... once it doesn't find __gt__ the
implementation continues by trying to coerce f to an int (by checking if
class foo supports __coerce__) and lastly by trying for the old-style
comparison method __cmp__. But you presumably don't care about that;
rather, the gist of it is: write your __gt__ -- that will also be used
as your "__rlt__" -- and so on!


Alex

[Bengt Richter]

Re: Overloading operators on the rigth.

On Tue, 12 Oct 2004 16:06:52 +0200, aleaxit@yahoo.c om (Alex Martelli) wrote:
[color=blue]
>denis wendum <wendum.denis@p asdepourriels.e df.fr> wrote:
>[color=green]
>> But my guess to define the method __rlt__ in my class in order to be
>> able to evaluate expressions "non_instan ce < instance" did not work.[/color]
>
>Sensible guess, but not the way things work.
>[color=green]
>> So what are the names (if they exist at all) of the comparison operators
>> <,>,== and so on, when one wants to exend their rigth hand side to a
>> user defined (instance of a) class?[/color]
>
>Here's how to find out:
>[color=green][color=darkred]
>>>> class foo:[/color][/color]
>... def __getattr__(sel f, name):
>... print 'looking for %r' % name
>... raise AttributeError, name
>...[color=green][color=darkred]
>>>> f=foo()
>>>> 12 < f[/color][/color]
>looking for '__gt__'
>looking for '__coerce__'
>looking for '__cmp__'
>True[color=green][color=darkred]
>>>>[/color][/color]
>
>See what's happening here? After determining that int (the type of 12)
>cannot deal with comparing 12 with an instance of foo, Python's
>implementati on of < turns to the right hand side... and asks for a
>__gt__ method first! Makes sense, because 12 < f is presumably going to
>be the same thing as f > 12 ... once it doesn't find __gt__ the
>implementati on continues by trying to coerce f to an int (by checking if
>class foo supports __coerce__) and lastly by trying for the old-style
>comparison method __cmp__. But you presumably don't care about that;
>rather, the gist of it is: write your __gt__ -- that will also be used
>as your "__rlt__" -- and so on!
>[/color]

On my system, (NT4, python 2.3.2) what really happens for newstyle classes
isn't apparent from that experiment:
[color=blue][color=green][color=darkred]
>>> class foo(object):[/color][/color][/color]
... def __getattr__(sel f, name):
... print 'looking for %r' % name
... raise AttributeError, name
...[color=blue][color=green][color=darkred]
>>> f=foo()
>>> 12 < f[/color][/color][/color]
True

The old style does work much the same (but note the default end result ;-)
Is that a 2.3.2 bug?
[color=blue][color=green][color=darkred]
>>> class foo:[/color][/color][/color]
... def __getattr__(sel f, name):
... print 'looking for %r' % name
... raise AttributeError, name
...[color=blue][color=green][color=darkred]
>>> f=foo()
>>> 12 < f[/color][/color][/color]
looking for '__gt__'
looking for '__coerce__'
looking for '__cmp__'
False

Regards,
Bengt Richter

[Jeff Epler]

Re: Overloading operators on the rigth.

I think it's a difference in new-style classes that __getattr__ doesn't
have a chance to run when searching for special methods.

I'm sure this is documented, but I didn't see it in

http://python.org/2.2.2/descrintro.html

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFBbIUEJd0 1MZaTXX0RAgC4AJ 9VDc64p8uTr+2Rq IjDqs/wjdcN4wCgiGBs
iyu6fjYUTutHsSO BvTFE1ds=
=Cnor
-----END PGP SIGNATURE-----

[Alex Martelli]

Re: Overloading operators on the rigth.

Bengt Richter <bokr@oz.net> wrote:
...[color=blue][color=green][color=darkred]
> >>>> class foo:[/color]
> >... def __getattr__(sel f, name):[/color][/color]
...[color=blue]
> On my system, (NT4, python 2.3.2) what really happens for newstyle classes
> isn't apparent from that experiment:[/color]

Heh, of course not -- the main difference between old-style and
new-style classes is that, for the latter, implicit lookup of special
methods when needed for operations starts with the *class*, not with the
*instance*. This is a fix of a long-standing design bug, because the
old rule was just impossible to apply consistently -- if calling X meant
looking for X.__call__ rather than X.__class__.__c all__, how could you
ever instantiate X by calling it, when X is a class whose _instances_
are meant to be callable, for example?

Since this has been discussed many times on this group, as well as in
presentations, articles, websites, books, ..., I didn't think it was
necessary to dwell on the issue again, considering it really did not
affect the answer requested by the original poster; apparently I was
wrong in so thinking. _Sigh_ -- next time somebody criticizes my posts
as being too long, I'll retort that of course they are, if I must keep
repeating such issues!-)

I just used an old-style class because, that way, I could show the
lookups happening without having to code a custom metaclass. The
special names being looked for are just the same, whether for classic or
newstyle classes, anyway -- all that changes is _where_ they're looked
up (starting on the instance, vs starting on the class).

[color=blue][color=green][color=darkred]
> >>> class foo(object):[/color][/color]
> ... def __getattr__(sel f, name):
> ... print 'looking for %r' % name
> ... raise AttributeError, name
> ...[color=green][color=darkred]
> >>> f=foo()
> >>> 12 < f[/color][/color]
> True
>
> The old style does work much the same (but note the default end result ;-)
> Is that a 2.3.2 bug?[/color]

No bug: comparisons between objects of disparate types give essentially
arbitrary results (repeatable within one run, but, at least in theory,
potentially different among runs of the same program, I believe).


Alex

[Bengt Richter]

Re: Overloading operators on the rigth.

On Wed, 13 Oct 2004 10:18:28 +0200, aleaxit@yahoo.c om (Alex Martelli) wrote:
[color=blue]
>Bengt Richter <bokr@oz.net> wrote:
> ...[color=green][color=darkred]
>> >>>> class foo:
>> >... def __getattr__(sel f, name):[/color][/color]
> ...[color=green]
>> On my system, (NT4, python 2.3.2) what really happens for newstyle classes
>> isn't apparent from that experiment:[/color]
>
>Heh, of course not -- the main difference between old-style and
>new-style classes is that, for the latter, implicit lookup of special
>methods when needed for operations starts with the *class*, not with the[/color]
Well, it always starts with the class if there's no attribute on the instance,
but the key is whether the "implicit lookup" for methods needed for operations
uses an overridable __getattribute_ _ somewhere, or just chases down the bases
chain looking for the actual __gt__ or whatever, not allowing a hook to synthesize
a result.[color=blue]
>*instance*. This is a fix of a long-standing design bug, because the
>old rule was just impossible to apply consistently -- if calling X meant
>looking for X.__call__ rather than X.__class__.__c all__, how could you
>ever instantiate X by calling it, when X is a class whose _instances_
>are meant to be callable, for example?
>
>Since this has been discussed many times on this group, as well as in
>presentation s, articles, websites, books, ..., I didn't think it was
>necessary to dwell on the issue again, considering it really did not
>affect the answer requested by the original poster; apparently I was
>wrong in so thinking. _Sigh_ -- next time somebody criticizes my posts
>as being too long, I'll retort that of course they are, if I must keep
>repeating such issues!-)
>[/color]
I have suspected you of having voice recognition input ;-)
I don't type that fast, unfortunately.
[color=blue]
>I just used an old-style class because, that way, I could show the
>lookups happening without having to code a custom metaclass. The[/color]
Could you show such a metaclass? I have tried a few things and not succeeded.
It almost makes me think there is no place to hook the internal __getattribute_ _
that looks for the methods for code generated from 12<f and the like.
I'd be interested in seeing it.
[color=blue]
>special names being looked for are just the same, whether for classic or[/color]
^^^^^^^^^^^^^^ but maybe not the same order, see below[color=blue]
>newstyle classes, anyway -- all that changes is _where_ they're looked
>up (starting on the instance, vs starting on the class).
>
>[color=green][color=darkred]
>> >>> class foo(object):[/color]
>> ... def __getattr__(sel f, name):
>> ... print 'looking for %r' % name
>> ... raise AttributeError, name
>> ...[color=darkred]
>> >>> f=foo()
>> >>> 12 < f[/color]
>> True
>>
>> The old style does work much the same (but note the default end result ;-)
>> Is that a 2.3.2 bug?[/color]
>
>No bug: comparisons between objects of disparate types give essentially
>arbitrary results (repeatable within one run, but, at least in theory,
>potentially different among runs of the same program, I believe).
>[/color]
I tried a class with all the methods explicit, and then removed them:
[color=blue][color=green][color=darkred]
>>> class foo(object):[/color][/color][/color]
... def __gt__(self, other): print 'gt'
... def __coerce__(self , other): print 'coerce'
... def __cmp__(self, other): print 'cmp'
...[color=blue][color=green][color=darkred]
>>> f=foo()
>>> 12<f[/color][/color][/color]
gt

Ok, so it looked for __gt__ first. Remove that.[color=blue][color=green][color=darkred]
>>> del foo.__gt__[/color][/color][/color]
[color=blue][color=green][color=darkred]
>>> 12<f[/color][/color][/color]
cmp
It seems to have looked for __cmp__ second, unlike in your post:

+---<from your post>------------------
Here's how to find out:
[color=blue][color=green][color=darkred]
>>> class foo:[/color][/color][/color]
.... def __getattr__(sel f, name):
.... print 'looking for %r' % name
.... raise AttributeError, name
....[color=blue][color=green][color=darkred]
>>> f=foo()
>>> 12 < f[/color][/color][/color]
looking for '__gt__'
looking for '__coerce__'
looking for '__cmp__'
True[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]
+-------------------------------------

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: an integer is required
(didn't like none back from cmp, presumably)

Ok, remove __cmp__[color=blue][color=green][color=darkred]
>>> del foo.__cmp__[/color][/color][/color]

One left:[color=blue][color=green][color=darkred]
>>> 12<f[/color][/color][/color]
coerce
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: __coerce__ didn't return a 2-tuple

NBD, but perhaps the order change is a version difference?
Not that the OP was asking any of this ;-)

Regards,
Bengt Richter

[Alex Martelli]

Re: Overloading operators on the rigth.

Bengt Richter <bokr@oz.net> wrote:
[color=blue]
> On Wed, 13 Oct 2004 10:18:28 +0200, aleaxit@yahoo.c om (Alex Martelli) wrote:
>[color=green]
> >Bengt Richter <bokr@oz.net> wrote:
> > ...[color=darkred]
> >> >>>> class foo:
> >> >... def __getattr__(sel f, name):[/color]
> > ...[color=darkred]
> >> On my system, (NT4, python 2.3.2) what really happens for newstyle classes
> >> isn't apparent from that experiment:[/color]
> >
> >Heh, of course not -- the main difference between old-style and
> >new-style classes is that, for the latter, implicit lookup of special
> >methods when needed for operations starts with the *class*, not with the[/color]
> Well, it always starts with the class if there's no attribute on the instance,[/color]

I find this a strange meaning for the word "start". To ensure the
instance doesn't have the attribute, that's where (conceptually) one
could say the lookup "starts", I think.
[color=blue]
> but the key is whether the "implicit lookup" for methods needed for
> operations uses an overridable __getattribute_ _ somewhere, or just chases
> down the bases chain looking for the actual __gt__ or whatever, not
> allowing a hook to synthesize a result.[/color]

Actually, it reaches right into the appropriate slot of the type
structure. The slots are generated or changed when the type (i.e., the
class object) is generated or changed. So, it may be improper on my
part to call this procedure a 'lookup'.
[color=blue][color=green]
> >I just used an old-style class because, that way, I could show the
> >lookups happening without having to code a custom metaclass. The[/color]
> Could you show such a metaclass? I have tried a few things and not
> succeeded. It almost makes me think there is no place to hook the internal
> __getattribute_ _ that looks for the methods for code generated from 12<f
> and the like. I'd be interested in seeing it.[/color]

I think I'd basically have to repeat what types.ClassType is doing --
essentially, prefill all slots with functions that _do_ perform the
lookups. To be honest, I haven't tried to see if that strategy hits any
snags, I just don't think it would.

[color=blue][color=green]
> >special names being looked for are just the same, whether for classic or[/color]
> ^^^^^^^^^^^^^^
> but maybe not the same order, see below[color=green]
> >newstyle classes, anyway -- all that changes is _where_ they're looked
> >up (starting on the instance, vs starting on the class).[/color][/color]

You're right -- coercion is "fading" so it's now attempted _after_ 3-way
comparison rather than before; so the 'where' isn't quite _all_ that's
changed (I had never coded nor seen a classic class which depended on
coercion for its comparisons, which may explain though not excuse my
imprecision).

[color=blue]
> NBD, but perhaps the order change is a version difference?[/color]

Yep, it's part of coercion slowly going away.
[color=blue]
> Not that the OP was asking any of this ;-)[/color]

No, but they may still be meaningful in some cases, of course.


Alex

[Bengt Richter]

Re: Overloading operators on the rigth.

On Wed, 13 Oct 2004 10:52:57 GMT, bokr@oz.net (Bengt Richter) wrote:
[...][color=blue]
>
>NBD, but perhaps the order change is a version difference?
>Not that the OP was asking any of this ;-)[/color]
Posted before recalling that the order difference showed up
on my system between old-style and new-style, both on 2.3.2,
so at least what I saw is not a version difference.

Someplace in classobject vs typeobject or such?

Regards,
Bengt Richter

[denis wendum]

Re: Overloading operators on the rigth.



Alex Martelli a écrit :
[color=blue]
> denis wendum <wendum.denis@p asdepourriels.e df.fr> wrote:
>
>
>
> Here's how to find out:
>[color=green][color=darkred]
> >>> class foo:[/color][/color]
> ... def __getattr__(sel f, name):
> ... print 'looking for %r' % name
> ... raise AttributeError, name
> ...[color=green][color=darkred]
> >>> f=foo()
> >>> 12 < f[/color][/color]
> looking for '__gt__'
> looking for '__coerce__'
> looking for '__cmp__'
> True[color=green][color=darkred]
> >>>[/color][/color]
>
> See what's happening here?
>
> Alex[/color]

Many thanks for your answer: you didn't just gave me the fish I asked but also

showed me how to use a net for catching other fish.

It seems I triggered a fierce debate with my question, but as far as I'm
concerned your example worked perfectly.

[Alex Martelli]

Re: Overloading operators on the rigth.

denis wendum <wendum.denis@p asdepourriels.e df.fr> wrote:
...[color=blue][color=green]
> > Here's how to find out:
> >[color=darkred]
> > >>> class foo:[/color]
> > ... def __getattr__(sel f, name):
> > ... print 'looking for %r' % name
> > ... raise AttributeError, name
> > ...[color=darkred]
> > >>> f=foo()
> > >>> 12 < f[/color]
> > looking for '__gt__'
> > looking for '__coerce__'
> > looking for '__cmp__'[/color][/color]
...[color=blue]
> Many thanks for your answer: you didn't just gave me the fish I asked but also
> showed me how to use a net for catching other fish.[/color]

You're welcome! Python enjoys excellent "discoverabilit y" (a term
copiously illustrated, by explanation and by example, in Eric Raymond's
excellent book "The Art of Unix Programming", highly recommended in both
paper and free-on-the-net form) -- it shews enough of its workings,
easily enough, that it's well worth doing some sniffing around, and the
ability to hack together a 4-line class that displays things with some
print statements in an interactive session is a good example.

Of course, I _was_ being a tad too glib here, which helps explain...:
[color=blue]
> It seems I triggered a fierce debate with my question, but as far as I'm
> concerned your example worked perfectly.[/color]

....the "debate", which was in fact quite friendly, even for this
community's pretty good standards. Summarizing, I half-cheated by using
an oldstyle class here, so that __getattr__ would indeed get called; a
newstyle class, while having just about the same behavior, does (...to
oversimply things a bit...) more processing at the time the class
statement is executed, it fills up a structure of internal 'slots', so
there is less "rooting around" at the time the < operator executes and
less of a chance to see the wheels in motion (there are of course plenty
of advantages in exchange, both conceptual and practical). Moreover the
'__coerce__' concept is being de-emphasized and deprecated, so the
"lookup" order shifts slightly (coercion is now tried only as a "last
ditch", after 3-way comparison with __cmp__ rather than before).

This doesn't mean you can't do "sniffing around" in newstyle classes,
but it may require a different tack, such as:
[color=blue][color=green][color=darkred]
>>> class foo(object):[/color][/color][/color]
.... def __gt__(self, other):
.... print 'doing gt'
.... return NotImplemented
.... def __cmp__(self, other):
.... print 'doing cmp'
.... return NotImplemented
....[color=blue][color=green][color=darkred]
>>> f=foo()
>>> 2<f[/color][/color][/color]
doing gt
doing cmp
False[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

The "return NotImplemented" may be seen as a way to implement a specific
comparison (say __gt__) in some cases while punting to more generic ways
implicitly -- you may use it to indicate to Python "I don't know how to
do this specific comparison in this specific method, so please try
something else [along the usual orders in which things are tried]"...


Alex

[Bengt Richter]

Re: Overloading operators on the rigth.

On Thu, 14 Oct 2004 10:00:28 +0200, aleaxit@yahoo.c om (Alex Martelli) wrote:
[color=blue]
>denis wendum <wendum.denis@p asdepourriels.e df.fr> wrote:
> ...[color=green][color=darkred]
>> > Here's how to find out:
>> >
>> > >>> class foo:
>> > ... def __getattr__(sel f, name):
>> > ... print 'looking for %r' % name
>> > ... raise AttributeError, name
>> > ...
>> > >>> f=foo()
>> > >>> 12 < f
>> > looking for '__gt__'
>> > looking for '__coerce__'
>> > looking for '__cmp__'[/color][/color]
> ...[color=green]
>> Many thanks for your answer: you didn't just gave me the fish I asked but also
>> showed me how to use a net for catching other fish.[/color]
>
>You're welcome! Python enjoys excellent "discoverabilit y" (a term
>copiously illustrated, by explanation and by example, in Eric Raymond's
>excellent book "The Art of Unix Programming", highly recommended in both
>paper and free-on-the-net form) -- it shews enough of its workings,
>easily enough, that it's well worth doing some sniffing around, and the
>ability to hack together a 4-line class that displays things with some
>print statements in an interactive session is a good example.
>
>Of course, I _was_ being a tad too glib here, which helps explain...:
>[color=green]
>> It seems I triggered a fierce debate with my question, but as far as I'm
>> concerned your example worked perfectly.[/color]
>
>...the "debate", which was in fact quite friendly, even for this
>community's pretty good standards. Summarizing, I half-cheated by using
>an oldstyle class here, so that __getattr__ would indeed get called; a
>newstyle class, while having just about the same behavior, does (...to
>oversimply things a bit...) more processing at the time the class
>statement is executed, it fills up a structure of internal 'slots', so
>there is less "rooting around" at the time the < operator executes and
>less of a chance to see the wheels in motion (there are of course plenty
>of advantages in exchange, both conceptual and practical). Moreover the
>'__coerce__' concept is being de-emphasized and deprecated, so the
>"lookup" order shifts slightly (coercion is now tried only as a "last
>ditch", after 3-way comparison with __cmp__ rather than before).
>
>This doesn't mean you can't do "sniffing around" in newstyle classes,
>but it may require a different tack, such as:
>[color=green][color=darkred]
>>>> class foo(object):[/color][/color]
>... def __gt__(self, other):
>... print 'doing gt'
>... return NotImplemented
>... def __cmp__(self, other):
>... print 'doing cmp'
>... return NotImplemented
>...[color=green][color=darkred]
>>>> f=foo()
>>>> 2<f[/color][/color]
>doing gt
>doing cmp
>False[color=green][color=darkred]
>>>>[/color][/color]
>
>The "return NotImplemented" may be seen as a way to implement a specific
>comparison (say __gt__) in some cases while punting to more generic ways
>implicitly -- you may use it to indicate to Python "I don't know how to
>do this specific comparison in this specific method, so please try
>something else [along the usual orders in which things are tried]"...
>[/color]
I didn't know about returning NotImplemented. That's good to know.
But otherwise the above is effectively what I wound up doing manually
in my other post, starting with all methods and manually deleting them
one by one as I retried the comparison.

Unfortunately, I am still frustrated, because there seems no way (other
than looking in sources) to discover a method whose name you don't know
ahead of time. You can just determine the order of access to known-name
methods. So I'm still looking for the magic metaclass formula that can
reveal the search for unknown methods. Maybe we need a sys._getattrhoo k
that could monitor operation-induced searching?

BTW, I experimented a little further with NotImplemented and the gt,cmp,coerce
methods in a base class vs derived and it looks like the search prefers
a base class gt or cmp over a derived version of the next alternative.

I wonder if a derived instance shouldn't be assumed to know more about doing
the right thing than its base class, if there is any overriding method available.
I.e., is the search really doing what it should in e.g. preferring Base.__gt__
over Derv.__cmp__ and Derv.__coerce__ ?

----< methtrack.py >---------------
class Base(object): pass
class Derv(Base): pass
for cls in (Base, Derv):
cname = cls.__name__
def __gt__(self, other, cname=cname):
print 'doing %s gt' % cname
return NotImplemented
def __cmp__(self, other, cname=cname):
print 'doing %s cmp' % cname
return NotImplemented
def __coerce__(self , other, cname=cname):
print 'doing %s coerce' % cname
return NotImplemented
for m in [__gt__, __cmp__,__coerc e__]:
setattr(cls, m.__name__, m)

print '-- with all:'
2 < Derv()
for cls in (Derv, Base):
for mname in 'gt cmp coerce'.split() :
print '-- sans %s %s:' % (cls.__name__, mname)
delattr(cls, '__%s__'%mname)
2 < Derv()
----------------------------------------------
Results:

[13:26] C:\pywk\clp\exa mples>methtrack .py
-- with all:
doing Derv gt
doing Derv cmp
-- sans Derv gt:
doing Base gt
doing Derv cmp
-- sans Derv cmp:
doing Base gt
doing Base cmp
-- sans Derv coerce:
doing Base gt
doing Base cmp
-- sans Base gt:
doing Base cmp
-- sans Base cmp:
doing Base coerce
-- sans Base coerce:

Regards,
Bengt Richter

[Andrew Durdin]

Re: Overloading operators on the rigth.

On Thu, 14 Oct 2004 20:27:13 GMT, Bengt Richter <bokr@oz.net> wrote:[color=blue]
>
> BTW, I experimented a little further with NotImplemented and the gt,cmp,coerce
> methods in a base class vs derived and it looks like the search prefers
> a base class gt or cmp over a derived version of the next alternative.[/color]

Isn't this just a side-effect of the attribute lookup order? When
looking for __gt__, python'll check the class, then go through the
base classes, and only return AttributeError if it's not found in the
class or any of its bases; after which it tries __cmp__ etc.

Am I understanding this correctly?

[Bengt Richter]

Re: Overloading operators on the rigth.

On Fri, 15 Oct 2004 10:55:59 +1100, Andrew Durdin <adurdin@gmail. com> wrote:
[color=blue]
>On Thu, 14 Oct 2004 20:27:13 GMT, Bengt Richter <bokr@oz.net> wrote:[color=green]
>>
>> BTW, I experimented a little further with NotImplemented and the gt,cmp,coerce
>> methods in a base class vs derived and it looks like the search prefers
>> a base class gt or cmp over a derived version of the next alternative.[/color]
>
>Isn't this just a side-effect of the attribute lookup order? When
>looking for __gt__, python'll check the class, then go through the
>base classes, and only return AttributeError if it's not found in the
>class or any of its bases; after which it tries __cmp__ etc.
>
>Am I understanding this correctly?[/color]
I think so, yes. I was just trying to point out that it might not be
the really logical thing to do. I.e., if a programmer supplies a __cmp__
method in his derived class, why would he do that if he didn't expect it
to be used for comparisons with his special kind of instances? Should he
have to know that __gt__ in a base class will preempt his intent, as e.g.
in the case of 2 < derived_instanc e?

I think, as you say, it's a side-effect of attribute lookup order, but
IWT that semantics should demand looking for all alternatives before going
to the next base, and then doing the same at each stage. That's probably
a pain to implement though ;-)

Regards,
Bengt Richter

[Alex Martelli]

Re: Overloading operators on the rigth.

Bengt Richter <bokr@oz.net> wrote:
[color=blue]
> Unfortunately, I am still frustrated, because there seems no way (other
> than looking in sources) to discover a method whose name you don't know
> ahead of time. You can just determine the order of access to known-name
> methods. So I'm still looking for the magic metaclass formula that can
> reveal the search for unknown methods. Maybe we need a sys._getattrhoo k
> that could monitor operation-induced searching?[/color]

I assume you're talking about types, aka newstyle classes, because for
oldstyle ones __getattr__ does get into play.

Now, the problem is as follows: no searching, stricto sensu, is induced
by an operation, in most cases. Rather, *when the type is built* some
code is responsible for filling out a C-level structure, the 'type'
struct, which mostly holds 'slots' which are pointers to C functions.
This "phase 0" is the time when some special method names may be checked
for, in the class dictionary and bases, in order to determine what to
place in the slots (a null pointer, a pointer to a C function which will
call the appropriate Python function, whatever).

What the operation induces, call it "phase 1", is one or a few checks
regarding those C pointers and sometimes after one is used for a call a
check for NonImplemented as a return value to determine whether to keep
checking. Typically no Python names are involved during phase 1 (the
main exception being classic classes, which are singled out for special
and more complicated treatment for backwards compatibility purposes).

((There may be a "phase 2" when you modify a class object, in which some
slots in the C-level structure are altered; but I don't see any way,
even conceptually, to exploit this for "fishing for unknown names")).

By writing a custom metaclass you may alter phase 0, but being able to
alter a certain complicated processing need not mean you can easily
introspect it. If the slot-filling process was altered to accept and
fully support any mapping in lieu of the class dict, you could thus get
the information about all the names that the slot-filling is looking for
(not necessarily how the presence of such names affects what slots, but
still you might use the bare info on names to inject artificially
constructed methods or other descriptors which later provide some
tracing information, perhaps).

Thinking about phase 1, the best architecture I can imagine for that
involves a C extension which, given a type object, instruments all of
its relevant slots, wrapping each C function it finds with a hook-caller
that is called for the specific purpose of providing tracing information
or other aspect-oriented before/after hooking.

I would assess the amount of work required for either or both projects
as non-trivial, particularly if one wants to do it in such a way as to
not impact performance (so it can stay in some future released Python,
maybe 2.5). It also appears to inevitably involve some reasonably deep
tinkering with internals that are currently coded in C. Of all the
separate aspects involved, the one with potentially highest return would
appear to me to be the ability to use any mapping, not just a dict --
that one might perhaps provide other benefits, once in place. It can be
done without excessive performance penalty by acting similarly to the
implementation of the STORE_LOCAL opcode (I'm not sure how you could
ever get the 'locals dict' of the current frame to be anything but a
real dict at the time a STORE_LOCAL executes, but, if you manage that,
STORE_LOCAL's implementation *is* coded to [a] specialcase dict first
for speed, [b] fallback to more abstract setitem if the locals of the
current frame are not exactly a dict, i.e. if they're an instance of any
dict subclass or anything else whatsoever).

On a parallel thread (about "reimplemen ting Python"), I'm pointing out
one potential advantage of rewriting an existing project in a higher
level language: tinkering and experimentation such as this becomes way
easier. The specific project I'm involved with, pypy, also enables the
tinkerer to know (just about) only Python, not necessitating other
programming languages (well, pyrex would still help for some of the
kinds of tinkering one might want, and so might machine language and
even, in-between, C or Common Lisp; but mucho tinkering could be done at
a purely Python level, if pypy matures and consolidates as we hope).


Alex