Reload Tricks

[Kamilche]

I want my program to be able to reload its code dynamically. I have a
large hierarchy of objects in memory. The inheritance hierarchy of
these objects are scattered over several files.

I find that after reloading the appropriate files, and overwriting the
__class__ of object instances, one more thing is necessary: reloading
the __bases__ of each reloaded class. If I don't do this, the modules
reloaded first point to old versions of the classes from later modules,
and when the later module is reloaded, it doesn't update the
inheritance hierarchy of classes already loaded.

This appears to be working... but now I'm wondering, what else did it
not change? Can I expect more toes to be blown off?

--Kamilche

[Michael Spencer]

Re: Reload Tricks

Kamilche wrote:[color=blue]
> I want my program to be able to reload its code dynamically. I have a
> large hierarchy of objects in memory. The inheritance hierarchy of
> these objects are scattered over several files.
>
> I find that after reloading the appropriate files, and overwriting the
> __class__ of object instances, one more thing is necessary: reloading
> the __bases__ of each reloaded class. If I don't do this, the modules
> reloaded first point to old versions of the classes from later modules,
> and when the later module is reloaded, it doesn't update the
> inheritance hierarchy of classes already loaded.
>
> This appears to be working... but now I'm wondering, what else did it
> not change? Can I expect more toes to be blown off?
>
> --Kamilche
>[/color]

There are some cases when re-assigning __class__ isn't possible, for example:[color=blue][color=green][color=darkred]
>>> class A(object):[/color][/color][/color]
... pass
...[color=blue][color=green][color=darkred]
>>> class B(dict):[/color][/color][/color]
... pass
...[color=blue][color=green][color=darkred]
>>> class C:[/color][/color][/color]
... pass
...[color=blue][color=green][color=darkred]
>>> a = A()
>>> a.__class__ = B[/color][/color][/color]
Traceback (most recent call last):
File "<input>", line 1, in ?
TypeError: __class__ assignment: 'A' object layout differs from 'B'[color=blue][color=green][color=darkred]
>>> a.__class__ = C[/color][/color][/color]
Traceback (most recent call last):
File "<input>", line 1, in ?
TypeError: __class__ must be set to new-style class, not 'classobj' object[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

An alternative approach (with some pros and cons) is to modify the class in
place, using something like:
[color=blue][color=green][color=darkred]
>>> def reclass(cls, to_cls):[/color][/color][/color]
... """Updates attributes of cls to match those of to_cls"""
...
... DONOTCOPY = ("__name__","__ bases__","__bas e__",
... "__dict__", "__doc__","__we akref__")
...
... fromdict = cls.__dict__
... todict = to_cls.__dict__
...
... # Delete any attribute present in the new class
... [delattr(cls,att r) for attr in fromdict.keys()
... if not((attr in todict) or (attr in DONOTCOPY)) ]
...
... for to_attr, to_obj in todict.iteritem s():
...
... if to_attr in DONOTCOPY:
... continue
...
... # This overwrites all functions, even if they haven't changed.
... if type(to_obj) is types.MethodTyp e:
... func = to_obj.im_func
... to_obj = types.MethodTyp e(func,None, cls)
...
... setattr(cls, to_attr,to_obj)
...[color=blue][color=green][color=darkred]
>>> class A(object):[/color][/color][/color]
... attr = "A"
...[color=blue][color=green][color=darkred]
>>> class B(object):[/color][/color][/color]
... attr = "B"
...[color=blue][color=green][color=darkred]
>>> a = A()
>>> reclass(A,B)
>>> a.attr[/color][/color][/color]
'B'[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

This copies attributes of old and new-style classes (in fact anything with a
__dict__ so probably a module would work too)

You still run into problems trying to re-assigning __bases__ to incompatible
objects, but this one-attribute-at-a-time approach gives you the potential to
intercept problem cases. In the example above, problems are avoided by not
copying __bases__.

An additional advantage of this aprpoach is that you don't need to keep track of
class instances, in order to change their __class__. Instances automatically
acquire the new behavior

One wart is that class docstrings are not writeable, so cannot be copied. Why?

Michael

[Kamilche]

Re: Reload Tricks

That's a powerful advantage - not having to track class instances.
Thanks for the tip! I just got done doing it 'my way' though, now I'll
have to change it. It took me all day! :-D

[Kamilche]

Re: Reload Tricks

Would it be possible to just not copy any attribute that starts and
ends with '__'? Or are there some important attributes being copied?

[Alex Martelli]

Re: Reload Tricks

Kamilche <klachemin@comc ast.net> wrote:
[color=blue]
> I want my program to be able to reload its code dynamically. I have a
> large hierarchy of objects in memory. The inheritance hierarchy of
> these objects are scattered over several files.[/color]

Michael Hudson has a nice custom metaclass for that in Activestate's
online cookbook -- I made some enhancements to it as I edited it for the
forthcoming 2nd edition of the cookbook (due out in a couple of months),
but the key ideas are in the online version too (sorry, no URL at hand).


Alex

[Michael Spencer]

Re: Reload Tricks

Kamilche wrote:[color=blue]
> I want my program to be able to reload its code dynamically. I have a
> large hierarchy of objects in memory. The inheritance hierarchy of
> these objects are scattered over several files.
>[/color]
Michael Spencer wrote:[color=blue]
> An alternative approach (with some pros and cons) is to modify the class in place, using something like:
>[color=green][color=darkred]
> >>> def reclass(cls, to_cls):[/color][/color]
> ... """Updates attributes of cls to match those of to_cls"""
> ...
> ... DONOTCOPY = ("__name__","__ bases__","__bas e__",
> ... "__dict__", "__doc__","__we akref__")[/color]
etc...

Kamilche wrote:[color=blue]
> Would it be possible to just not copy any attribute that starts and
> ends with '__'? Or are there some important attributes being copied?[/color]


Possible? of course, it's Python ;-)

But there are many 'magic' attributes for behavior that you probably do want to
copy:

e.g., __getitem__, __setitem__ etc...

See: http://docs.python.org/ref/specialnames.html

Michael Hudson's recipe:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/160164

does auto-reloading "automatically" , at the price of changing the type of the
classes you want to manage. It's a very convenient approach for interactive
development (which is the recipe's stated purpose). It works by tracking
instances and automatically updating their class. If your program relies on
class identity, you may run into problems.


Michael