Copying objects style questions

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Bob Halley

    Copying objects style questions

    In dnspython I have a set class, SimpleSet. (I don't use Python 2.3's
    sets.Set class so I can keep supporting Python 2.2, and because the
    objects in my sets are mutable). The SimpleSet class has a single
    attribute, "items", which is a list. (I know a list is not going to
    lead to fast set operations in general, but my typical set has only
    one or two elements in it, so the potential performance issues don't
    really matter for my needs.)

    I then subclass SimpleSet to make other kinds of sets, e.g. RRset
    subclasses Rdataset which subclasses SimpleSet. RRsets and Rdatasets
    each add additional attributes.

    I want to have a copy operation which is an "almost shallow" copy.
    Specifically, all of the attributes of the object may be shallow
    copied except for one, the 'items' list of the SimpleSet, for which I
    want a new list containing references to the same elements, so that
    the user of the copy may add or remove elements subsequently without
    affecting the original.

    I can't use copy.copy()'s default behavior, because it is too shallow.
    I don't want to use copy.deepcopy() because it's too deep. I
    contemplated __copy__, __initargs__, __getstate__, and __setstate__,
    but they didn't seem to fit the bill, or seemed more complicated than
    the solution I ended up with (see below).

    I can, of course, write my own copy() method, but I don't want to
    require each subclass of Set have to make a copy() method which
    implements the entire copying effort. Rather I'd like cooperating
    superclasses; I'd like RRset to copy the name, and then let Rdataset
    copy its attributes, and then let SimpleSet do the copy of the items
    attribute.

    My first solution was like clone() in Java:

    In SimpleSet:

    def copy(self):
    """Make a (shallow) copy of the set.

    There is a 'copy protocol' that subclasses of
    this class should use. To make a copy, first
    call your super's copy() method, and use the
    object returned as the new instance. Then
    make shallow copies of the attributes defined
    in the subclass.

    This protocol allows us to write the set
    algorithms that return new instances
    (e.g. union) once, and keep using them in
    subclasses.
    """

    cls = self.__class__
    # I cannot call self.__class__( ) because the
    # __init__ method of the subclasses cannot be
    # called meaningfully with no arguments
    obj = cls.__new__(cls )
    obj.items = list(self.items )
    return obj

    In Rdataset, which subclasses SimpleSet:

    def copy(self):
    obj = super(Rdataset, self).copy()
    obj.rdclass = self.rdclass
    obj.rdtype = self.rdtype
    obj.covers = self.covers
    obj.ttl = self.ttl
    return obj

    I've also noticed that if I just make SimpleSet subclass list instead
    of having an "items" element, then "the right thing" happens with
    copy.copy(). I'm a little leery of subclassing the base types,
    because although I've done it to good effect sometimes, I've also had
    it cause odd problems because the built-in types behave a little
    differently than new-style classes in some cases. Also, at least in
    this case, it fails the is-a test. A set is *not* a list; the fact
    that I'm using a list is an implementation detail that I might not
    want to expose.

    So, what advice do Python experts have for this kind of situation?
    Should I keep the first solution? Should I subclass list in spite of
    my misgivings? Is there some other, more elegant solution I've
    missed?

    Thanks in advance!

    /Bob

  • Alex Martelli

    #2
    Re: Copying objects style questions

    Bob Halley wrote:
    ...[color=blue]
    > I can't use copy.copy()'s default behavior, because it is too shallow.
    > I don't want to use copy.deepcopy() because it's too deep. I[/color]

    So far, so perfectly clear.
    [color=blue]
    > contemplated __copy__, __initargs__, __getstate__, and __setstate__,
    > but they didn't seem to fit the bill, or seemed more complicated than
    > the solution I ended up with (see below).[/color]

    I don't understand this. What's wrong with, e.g.:

    def __copy__(self):
    class EmptyClass: pass
    obj = EmptyClass()
    obj.__class__ = self.__class__
    obj.__dict__.up date(self.__dic t__)
    obj.items = list(self.items )
    return obj

    ??? It seems simpler and more automatic than your 'copy protocol';
    subclasses don't need to do anything special unless they need to
    "deepen" the copy of some of their attributes. Btw, if you're
    using new-style classes, you'll want to use object instead of
    EmptyClass, or start with obj = self.__class__. new(self.__clas s__)
    as you're doing in your protocol, of course -- but the key idea
    is "bulk copy all that's in self's __dict__, then special-case
    only what little needs to be specialcased". And doing it in a
    method called __copy__ means any user of your class needs not
    learn about a new copy protocol but rather just uses copy.copy.

    It may be that I'm not correctly understanding your issues, of
    course, but I hope these suggestions can help.


    Alex

    Comment

    • Bengt Richter

      #3
      Re: Copying objects style questions

      On Wed, 06 Aug 2003 10:23:16 GMT, Alex Martelli <aleax@aleax.it > wrote:
      [color=blue]
      >Bob Halley wrote:
      > ...[color=green]
      >> I can't use copy.copy()'s default behavior, because it is too shallow.
      >> I don't want to use copy.deepcopy() because it's too deep. I[/color]
      >
      >So far, so perfectly clear.
      >[color=green]
      >> contemplated __copy__, __initargs__, __getstate__, and __setstate__,
      >> but they didn't seem to fit the bill, or seemed more complicated than
      >> the solution I ended up with (see below).[/color]
      >
      >I don't understand this. What's wrong with, e.g.:
      >
      >def __copy__(self):
      > class EmptyClass: pass
      > obj = EmptyClass()
      > obj.__class__ = self.__class__
      > obj.__dict__.up date(self.__dic t__)
      > obj.items = list(self.items )
      > return obj
      >
      >??? It seems simpler and more automatic than your 'copy protocol';
      >subclasses don't need to do anything special unless they need to
      >"deepen" the copy of some of their attributes. Btw, if you're
      >using new-style classes, you'll want to use object instead of
      >EmptyClass, or start with obj = self.__class__. new(self.__clas s__)[/color]

      <nits>
      I don't think you meant object() as a direct substitute for EmptyClass() above, right?
      And you meant to put tails on that "new," presumably.
      </nits>
      [color=blue]
      >as you're doing in your protocol, of course -- but the key idea
      >is "bulk copy all that's in self's __dict__, then special-case
      >only what little needs to be specialcased". And doing it in a
      >method called __copy__ means any user of your class needs not
      >learn about a new copy protocol but rather just uses copy.copy.
      >
      >It may be that I'm not correctly understanding your issues, of
      >course, but I hope these suggestions can help.
      >
      >
      >Alex
      >[/color]

      Regards,
      Bengt Richter

      Comment

      • Alex Martelli

        #4
        Re: Copying objects style questions

        Bengt Richter wrote:
        ...[color=blue][color=green]
        >>def __copy__(self):
        >> class EmptyClass: pass
        >> obj = EmptyClass()
        >> obj.__class__ = self.__class__
        >> obj.__dict__.up date(self.__dic t__)
        >> obj.items = list(self.items )
        >> return obj
        >>
        >>??? It seems simpler and more automatic than your 'copy protocol';
        >>subclasses don't need to do anything special unless they need to
        >>"deepen" the copy of some of their attributes. Btw, if you're
        >>using new-style classes, you'll want to use object instead of
        >>EmptyClass, or start with obj = self.__class__. new(self.__clas s__)[/color]
        >
        > <nits>
        > I don't think you meant object() as a direct substitute for EmptyClass()[/color]

        Right -- you couldn't assign __class__ on an object() result, for example.
        [color=blue]
        > above, right? And you meant to put tails on that "new," presumably.
        > </nits>[/color]

        Right again. So, for completeness, with a new-style class you could
        do, e.g. (probably optimal or close to it):

        def __copy__(self):
        obj = self.__class__. __new__(self.__ class__)
        obj.__dict__.up date(self.__dic t__)
        obj.items = list(self.items )
        return obj

        Sorry for the sloppy expression in those hurriedly-written two lines;-).

        To forestall further misunderstandin gs -- this would not work for
        objects with __slots__ (no __dict__) or inheriting from classes with
        __slots__ (wouldn't copy attributes not in __dict__) -- there is
        always more work needed for such cases. But if one uses __slots__
        one *deserves* to have to work a bit harder as a result;-).


        Alex

        Comment

        • Alex Martelli

          #5
          Re: Copying objects style questions

          Bob Halley wrote:
          [color=blue][color=green]
          >> To forestall further misunderstandin gs -- this would not work for
          >> objects with __slots__ (no __dict__) or inheriting from classes with
          >> __slots__ (wouldn't copy attributes not in __dict__) -- there is
          >> always more work needed for such cases. But if one uses __slots__
          >> one *deserves* to have to work a bit harder as a result;-).[/color]
          >
          > I use new-style classes with __slots__ :), because it makes a really
          > noticeable difference in memory usage if you use dnspython to work
          > with a good-sized zone (i.e. one with tens of thousands of records).[/color]

          If you have huge numbers of instances of a class, it sure may well make
          sense to give the class __slots__ in order to save memory -- that's what it
          was introduced for. But then, giving that class a __copy__, or the like,
          which knows about its slots (or using the deep dark magic of _reduce_ex,
          I guess...), if you need to perform fancier copies than ordinary copy.copy
          or copy.deepcopy would do for you, seems fair enough to me:-).


          Alex

          Comment

          Working...