super, decorators and gettattribute

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Richard Szopa

    super, decorators and gettattribute

    Hello all,

    I am playing around w/ Python's object system and decorators and I
    decided to write (as an exercise) a decorator that (if applied to a
    method) would call the superclass' method of the same name before
    doing anything (initially I wanted to do something like CLOS
    [1] :before and :end methods, but that turned out to be too
    difficult).

    However, I cannot get it right (specially, get rid of the eval). I
    suspect that I may be misunderstandin g something happening between
    super objects and __getattribute_ _ methods.

    Here's my code:

    def endmethod(fun):
    """Decorato r to call a superclass' fun first.

    If the classes child and parent are defined as below, it should
    work like:
    >>x = child()
    >>x.foo()
    I am parent's foo
    I am child's foo.
    """
    name = fun.__name__
    def decorated(self, *args, **kwargs):
    try:
    super_object = super(self.__cl ass__, self)

    # now I want to achieve something equivalent to calling
    # parent.foo(*arg s, **kwargs)
    # if I wanted to limit it only to this example

    # this doesn't work: in the example, it calls child's foo,
    # entering in an eternal loop (instead of calling parent's
    # foo, as I would expect).

    # super_object.__ getattribute__( name)(*args, **kwargs)

    # this does work, but I feel it's ugly
    eval('super_obj ect.%s(*args, **kwargs)' % name)
    except AttributeError:
    pass # if parent doesn't implement fun, we don't care
    # about it
    return fun(self, *args, **kwargs) # hopefully none

    decorated.__nam e__ = name
    return decorated


    class parent(object):
    def foo(self):
    print 'I am parent\'s foo'

    class child(parent):
    @endmethod
    def foo(self):
    print "I am foo\'s foo."

    if __name__=='__ma in__':
    x = child()
    x.foo()

    Can anybody tell me how to call a superclass method knowing its name?

    Thanks in advance,
    -- Richard

    [1] http://en.wikipedia.org/wiki/Common_Lisp_Object_System
  • Richard Szopa

    #2
    Re: super, decorators and gettattribute

    On Jan 12, 7:45 pm, Richard Szopa <ryszard.sz...@ gmail.comwrote:
    doing anything (initially I wanted to do something like CLOS
    [1] :before and :end methods, but that turned out to be too
    difficult).
    Erm, I meant :before and :after methods.

    -- Richard

    Comment

    • Mike Meyer

      #3
      Re: super, decorators and gettattribute

      On Sat, 12 Jan 2008 10:45:25 -0800 (PST) Richard Szopa <ryszard.szopa@ gmail.comwrote:
      Hello all,
      >
      I am playing around w/ Python's object system and decorators and I
      decided to write (as an exercise) a decorator that (if applied to a
      method) would call the superclass' method of the same name before
      doing anything (initially I wanted to do something like CLOS
      [1] :before and :end methods, but that turned out to be too
      difficult).
      >
      However, I cannot get it right (specially, get rid of the eval). I
      suspect that I may be misunderstandin g something happening between
      super objects and __getattribute_ _ methods.
      >
      Here's my code:
      >
      def endmethod(fun):
      """Decorato r to call a superclass' fun first.
      >
      If the classes child and parent are defined as below, it should
      work like:
      >
      >>x = child()
      >>x.foo()
      I am parent's foo
      I am child's foo.
      """
      name = fun.__name__
      def decorated(self, *args, **kwargs):
      try:
      super_object = super(self.__cl ass__, self)
      There's an apparently common bug here: you don't want to pass super
      self.__class__, but the class that the method is bound to. The two
      aren't the same, as an instance of a subclass will have the subclass
      as self.__class__, and not the current class. So super will return the
      current class or a subclass of it, meaning (since you invoked this
      method from self) you'll wind up invoking this method recursively.
      All of which means your decorator is probably going to have to take
      the class as an argument.
      # now I want to achieve something equivalent to calling
      # parent.foo(*arg s, **kwargs)
      # if I wanted to limit it only to this example
      >
      # this doesn't work: in the example, it calls child's foo,
      # entering in an eternal loop (instead of calling parent's
      # foo, as I would expect).
      >
      # super_object.__ getattribute__( name)(*args, **kwargs)
      >
      # this does work, but I feel it's ugly
      eval('super_obj ect.%s(*args, **kwargs)' % name)
      except AttributeError:
      pass # if parent doesn't implement fun, we don't care
      # about it
      return fun(self, *args, **kwargs) # hopefully none
      >
      decorated.__nam e__ = name
      return decorated
      >
      >
      class parent(object):
      def foo(self):
      print 'I am parent\'s foo'
      >
      class child(parent):
      @endmethod
      def foo(self):
      print "I am foo\'s foo."
      >
      if __name__=='__ma in__':
      x = child()
      x.foo()
      >
      Can anybody tell me how to call a superclass method knowing its name?
      The same way you call any object's methods if you know it's name":

      getattr(super_o bject, name)(*args, **kwargs)

      The code seems to work the way you want:
      >>x.foo()
      I am parent's foo
      I am foo's foo.

      <mike
      --
      Mike Meyer <mwm@mired.or g> http://www.mired.org/consulting.html
      Independent Network/Unix/Perforce consultant, email for more information.

      Comment

      • Richard Szopa

        #4
        Re: super, decorators and gettattribute

        On Jan 12, 9:47 pm, Mike Meyer <mwm-keyword-python.b4b...@m ired.org>
        wrote:
        The same way you call any object's methods if you know it's name":
        >
        getattr(super_o bject, name)(*args, **kwargs)
        Thanks a lot for your answer!

        However, I am very surprised to learn that

        super_object.__ getattr__(name) (*args, **kwargs)

        getattr(super_o bject, name)(*args, **kwargs)

        are not equivalent. This is quite odd, at least when with len()
        and .__len__, str() and .__str__. Do you maybe know what's the
        rationale behind not following that convention by getattr?

        Best regards,

        -- Richard

        Comment

        • Steven D'Aprano

          #5
          Re: super, decorators and gettattribute

          On Sat, 12 Jan 2008 15:47:05 -0500, Mike Meyer wrote:
          There's an apparently common bug here: you don't want to pass super
          self.__class__, but the class that the method is bound to.
          Given an instance method, is it possible to easily determine what class
          it is defined in?

          I thought the im_class attribute might do it, but it apparently just
          points to self.__class__.
          >>class Foo(object):
          .... def foo(self):
          .... pass
          ....
          >>class Bar(Foo):
          .... def bar(self):
          .... pass
          ....
          >>Bar().bar.im_ class # expecting Bar
          <class '__main__.Bar'>
          >>Bar().foo.im_ class # hoping for Foo
          <class '__main__.Bar'>



          --
          Steven

          Comment

          • Marc 'BlackJack' Rintsch

            #6
            Re: super, decorators and gettattribute

            On Sat, 12 Jan 2008 14:23:52 -0800, Richard Szopa wrote:
            However, I am very surprised to learn that
            >
            super_object.__ getattr__(name) (*args, **kwargs)
            >
            getattr(super_o bject, name)(*args, **kwargs)
            >
            are not equivalent. This is quite odd, at least when with len()
            and .__len__, str() and .__str__. Do you maybe know what's the
            rationale behind not following that convention by getattr?
            I think you are confusing `__getattr__` and `__getattribute __` here!
            `getattr()` maps to `__getattr__()` , it's `__getattribute __` that's
            different.

            Ciao,
            Marc 'BlackJack' Rintsch

            Comment

            • Richard Szopa

              #7
              Re: super, decorators and gettattribute

              On Jan 13, 8:59 am, Marc 'BlackJack' Rintsch <bj_...@gmx.net wrote:
              On Sat, 12 Jan 2008 14:23:52 -0800, Richard Szopa wrote:
              However, I am very surprised to learn that
              >
              super_object.__ getattr__(name) (*args, **kwargs)
              >
              getattr(super_o bject, name)(*args, **kwargs)
              >
              are not equivalent. This is quite odd, at least when with len()
              and .__len__, str() and .__str__. Do you maybe know what's the
              rationale behind not following that convention by getattr?
              >
              I think you are confusing `__getattr__` and `__getattribute __` here!
              `getattr()` maps to `__getattr__()` , it's `__getattribute __` that's
              different.
              Well, in my code calling super_object.__ getattr__(name) (*args,
              **kwargs) and getattr(super_o bject, name)(*args, **kwargs) gives
              *different* effects (namely, the latter works, while the former
              doesn't). That kinda suggests that they don't map to each other :-).
              And that makes me feel confused.

              Cheers,

              -- Richard

              Comment

              • thebjorn

                #8
                Re: super, decorators and gettattribute

                On Jan 13, 1:51 pm, Richard Szopa <ryszard.sz...@ gmail.comwrote:
                On Jan 13, 8:59 am, Marc 'BlackJack' Rintsch <bj_...@gmx.net wrote:
                >
                On Sat, 12 Jan 2008 14:23:52 -0800, Richard Szopa wrote:
                However, I am very surprised to learn that
                >
                super_object.__ getattr__(name) (*args, **kwargs)
                >
                getattr(super_o bject, name)(*args, **kwargs)
                >
                are not equivalent. This is quite odd, at least when with len()
                and .__len__, str() and .__str__. Do you maybe know what's the
                rationale behind not following that convention by getattr?
                >
                I think you are confusing `__getattr__` and `__getattribute __` here!
                `getattr()` maps to `__getattr__()` , it's `__getattribute __` that's
                different.
                >
                Well, in my code calling super_object.__ getattr__(name) (*args,
                **kwargs) and getattr(super_o bject, name)(*args, **kwargs) gives
                *different* effects (namely, the latter works, while the former
                doesn't). That kinda suggests that they don't map to each other :-).
                And that makes me feel confused.
                >
                Cheers,
                >
                -- Richard
                They do, except for when it comes to what super(..) returns. It isn't
                really an object in the sense that they're presented in the tutorial,
                but rather a sort of proxy to the methods in the ancestor classes of
                the concrete object (self), relative to the current method's class. I
                can't imagine that sentence would ease any confusion however, suffice
                it to say that you have to call getattr(super(. .), 'name') instead of
                super(..).__get attr__('name') and you have to call super(..).__len __()
                instead of len(super(..)) -- I can't imagine that lessens any
                confusion either :-/

                super(..) is designed to handle situations like this correctly

                class Root(object):
                n = 1

                class Left(Root):
                def foo(self):
                print 'n =', self.n
                print 'super n = ', super(Left, self).n

                class Right(Root):
                n = 2

                class Leaf(Left,Right ):
                n = 3

                x = Leaf()
                x.foo()

                the correct output is

                n = 3
                super n = 2


                -- bjorn

                Comment

                • Richard Szopa

                  #9
                  Re: super, decorators and gettattribute

                  On Jan 13, 3:31 pm, thebjorn <BjornSteinarFj eldPetter...@gm ail.com>
                  wrote:
                  They do, except for when it comes to what super(..) returns. It isn't
                  really an object in the sense that they're presented in the tutorial,
                  but rather a sort of proxy to the methods in the ancestor classes of
                  the concrete object (self), relative to the current method's class. I
                  can't imagine that sentence would ease any confusion however, suffice
                  it to say that you have to call getattr(super(. .), 'name') instead of
                  super(..).__get attr__('name') and you have to call super(..).__len __()
                  instead of len(super(..)) -- I can't imagine that lessens any
                  confusion either :-/
                  Surprisingly, I think your first sentence *does* make something more
                  clear. Let me check if I understand it right: when we call a method on
                  super(Foo, self) it is as if we were calling call-next-method in
                  Common Lisp or Dylan (i.e. the method of the class on the right of Foo
                  in self.mro()). This however does not imply for super to have its dict
                  the same as the class on the right of Foo---it remains the same as
                  self's dict.

                  However, there's one piece that doesn't completely fit to the puzzle:
                  why does getattr work? The help says:

                  getattr(...)
                  getattr(object, name[, default]) -value

                  Get a named attribute from an object; getattr(x, 'y') is
                  equivalent to x.y.
                  When a default argument is given, it is returned when the
                  attribute doesn't
                  exist; without it, an exception is raised in that case.

                  Does it work on the basis that "getattr(x, 'y') is equivalent to x.y"?
                  What is then a "named attribute for an object" in Python? It seems not
                  to be equivalent to the value of the item whose name is 'y' in the
                  object's class __dict__...

                  Cheers,

                  -- Richard

                  Comment

                  • Michele Simionato

                    #10
                    Re: super, decorators and gettattribute

                    On Jan 14, 1:41 pm, Richard Szopa <ryszard.sz...@ gmail.comwrote:
                    However, there's one piece that doesn't completely fit to the puzzle:
                    why does getattr work? The help says:
                    >
                    getattr(...)
                    getattr(object, name[, default]) -value
                    >
                    Get a named attribute from an object; getattr(x, 'y') is
                    equivalent to x.y.
                    When a default argument is given, it is returned when the
                    attribute doesn't
                    exist; without it, an exception is raised in that case.
                    >
                    Does it work on the basis that "getattr(x, 'y') is equivalent to x.y"?
                    What is then a "named attribute for an object" in Python? It seems not
                    to be equivalent to the value of the item whose name is 'y' in the
                    object's class __dict__...
                    >
                    Cheers,
                    >
                    -- Richard
                    I really need to publish this one day or another, since these
                    questions
                    about super keeps coming out:


                    Comment

                    • George Sakkis

                      #11
                      Re: super, decorators and gettattribute

                      On Jan 12, 6:56 pm, Steven D'Aprano <st...@REMOVE-THIS-
                      cybersource.com .auwrote:
                      On Sat, 12 Jan 2008 15:47:05 -0500, Mike Meyer wrote:
                      There's an apparently common bug here: you don't want to pass super
                      self.__class__, but the class that the method is bound to.
                      >
                      Given an instance method, is it possible to easily determine what class
                      it is defined in?
                      >
                      I thought the im_class attribute might do it, but it apparently just
                      points to self.__class__.
                      >
                      >class Foo(object):
                      >
                      ... def foo(self):
                      ... pass
                      ...>>class Bar(Foo):
                      >
                      ... def bar(self):
                      ... pass
                      ...>>Bar().bar. im_class # expecting Bar
                      >
                      <class '__main__.Bar'> >>Bar().foo.im_ class # hoping for Foo
                      >
                      <class '__main__.Bar'>
                      Something like that seems to work for most cases:

                      from inspect import getmro

                      def getdef(obj,attr ):
                      try: objattrs = obj.__dict__
                      except AttributeError:
                      objattrs = obj.__slots__
                      if attr in objattrs:
                      return obj
                      for cls in getmro(obj.__cl ass__):
                      if attr in cls.__dict__:
                      return cls
                      >>getdef(Bar( ), 'bar')
                      <class '__main__.Bar'>
                      >>getdef(Bar( ), 'foo')
                      <class '__main__.Foo'>


                      It probably misses some edge cases but I can't think of any off the
                      top of my head.

                      George

                      Comment

                      • thebjorn

                        #12
                        Re: super, decorators and gettattribute

                        On Jan 14, 1:41 pm, Richard Szopa <ryszard.sz...@ gmail.comwrote:
                        On Jan 13, 3:31 pm, thebjorn <BjornSteinarFj eldPetter...@gm ail.com>
                        wrote:
                        >
                        They do, except for when it comes to what super(..) returns. It isn't
                        really an object in the sense that they're presented in the tutorial,
                        but rather a sort of proxy to the methods in the ancestor classes of
                        the concrete object (self), relative to the current method's class. I
                        can't imagine that sentence would ease any confusion however, suffice
                        it to say that you have to call getattr(super(. .), 'name') instead of
                        super(..).__get attr__('name') and you have to call super(..).__len __()
                        instead of len(super(..)) -- I can't imagine that lessens any
                        confusion either :-/
                        >
                        Surprisingly, I think your first sentence *does* make something more
                        clear. Let me check if I understand it right: when we call a method on
                        super(Foo, self) it is as if we were calling call-next-method in
                        Common Lisp or Dylan
                        I don't remember if CLOS was changed to use C3 Linearization also, but
                        the concept came from Dylan (http://www.webcom.com/haahr/dylan/
                        linearization-oopsla96.html) and that's what is implemented in Python.

                        [...]
                        However, there's one piece that doesn't completely fit to the puzzle:
                        why does getattr work? The help says:
                        >
                        getattr(...)
                        getattr(object, name[, default]) -value
                        >
                        Get a named attribute from an object; getattr(x, 'y') is
                        equivalent to x.y. When a default argument is given, it
                        is returned when the attribute doesn't exist; without it,
                        an exception is raised in that case.
                        >
                        Does it work on the basis that "getattr(x, 'y') is equivalent to x.y"?
                        What is then a "named attribute for an object" in Python? It seems not
                        to be equivalent to the value of the item whose name is 'y' in the
                        object's class __dict__...
                        Conceptually, x.y is always "get the y attribute of x" and the same as
                        getattr(x, 'y'). Depending on the type of x and y, and your
                        familiarity with Python internals, what actually happens during a
                        lookup might be surprising. In the vast majority of cases however, x.y
                        is equivalent to one of

                        x.__dict__['y']
                        type(x).__dict_ _['y']

                        but if you're a language geek like me, you might be excited that in
                        some cases it is

                        type(x).__dict_ _['y'].__get__(x, type(x))

                        which says you get the value of x.y by calling y and passing x as an
                        argument -- if you know CLOS you'll recognize that it's a primitive
                        multi-method call. (there are some other special cases too, although
                        not as exciting ;-)

                        Much more detail can be found in Raymond's paper on descriptors
                        (http://users.rcn.com/python/download/Descriptor.htm) and Michele's
                        paper on super (http://www.phyast.pitt.edu/~micheles/python/
                        super.html).

                        -- bjorn


                        Comment

                        • Ben Finney

                          #13
                          Re: super, decorators and gettattribute

                          Michele Simionato <michele.simion ato@gmail.comwr ites:
                          I really need to publish this one day or another, since these
                          questions about super keeps coming out:
                          >
                          http://www.phyast.pitt.edu/~micheles/python/super.html
                          Yes, please. Your article has improved my understanding just from
                          skimming the main headings :-)

                          --
                          \ "Buy not what you want, but what you need; what you do not need |
                          `\ is expensive at a penny." -- Cato, 234-149 BC, Relique |
                          _o__) |
                          Ben Finney

                          Comment

                          • Richard Szopa

                            #14
                            Re: super, decorators and gettattribute

                            On Jan 14, 11:05 pm, thebjorn <BjornSteinarFj eldPetter...@gm ail.com>
                            wrote:
                            I don't remember if CLOS was changed to use C3 Linearization also, but
                            the concept came from Dylan (http://www.webcom.com/haahr/dylan/
                            linearization-oopsla96.html) and that's what is implemented in Python.
                            The Common Lisp ANSI standard is from 1994, and the article you cite
                            is from 1996, which strongly suggests C3 linearization wasn't included
                            in CLOS. The most important difference between CLOS and C3
                            linearization AFAIK is that the latter enforces monotonicity, while
                            the former doesn't.

                            Of course, it shouldn't be very difficult to implement the C3 behavior
                            in Common Lisp using the de facto standard MetaObject Protocol.

                            (Nb. Dylan was such a nice language... It's a pity it is practically
                            dead right now.)
                            but if you're a language geek like me, you might be excited that in
                            some cases it is
                            >
                            type(x).__dict_ _['y'].__get__(x, type(x))
                            >
                            which says you get the value of x.y by calling y and passing x as an
                            argument -- if you know CLOS you'll recognize that it's a primitive
                            multi-method call. (there are some other special cases too, although
                            not as exciting ;-)
                            Yeah, I also feel the excitement, so probably I am a language geek
                            too ;-). However, this is still quite far away from full fledged
                            multimethods. (OTOH trying to get something more from these primitive
                            multimethods by abusing __get__ looks kind of tempting ;-))

                            Regards,

                            -- Richard

                            Comment

                            • Basilisk96

                              #15
                              Re: super, decorators and gettattribute

                              On Jan 14, 7:53 am, Michele Simionato <michele.simion ...@gmail.com>
                              wrote:
                              I really need to publish this one day or another, since these
                              questions
                              about super keeps coming out:
                              >
                              http://www.phyast.pitt.edu/~micheles/python/super.html
                              Please do. It is a very enlightening discussion, and I'm sure a bunch
                              of folks will benefit from it. And please update it (if necessary) to
                              the current Python version. At the time of that writing, 2.3 must have
                              been King, but oh my, how time flies :-)

                              Cheers,
                              -Basilisk96

                              Comment

                              Working...