saving and reloading variables for interactive work

[Darren Dale]

If I am working interactively with 10 variables and wanted to pickle
them and reload them tomorrow using the same variable names, would there
be a direct way of doing it?

Thanks,
Darren

[Christopher T King]

Re: saving and reloading variables for interactive work

On Tue, 27 Jul 2004, Darren Dale wrote:
[color=blue]
> If I am working interactively with 10 variables and wanted to pickle
> them and reload them tomorrow using the same variable names, would there
> be a direct way of doing it?[/color]

Reminds me of my days in AppleSoft BASIC :)

Unfortunately, there is no quick & easy way to do this in Python, since
there is no easy way to differentiate unpicklable things (such as modules
and functions) from picklable ones (your variables), nor to make Pickle
gracefully ignore such objects. If you can get all your variable names
into a list, however, you could use this code:
[color=blue][color=green][color=darkred]
>>> foo = 5
>>> bar = 6
>>> myvars = ['foo','bar']
>>> import pickle
>>> pickle.dump(dic t([(n,vars()[n]) for n in myvars]),open('myvars' ,'w'))[/color][/color][/color]

[time passes]
[color=blue][color=green][color=darkred]
>>> import pickle
>>> vars().update(p ickle.load(open ('myvars')))
>>> foo[/color][/color][/color]
5[color=blue][color=green][color=darkred]
>>> bar[/color][/color][/color]
6

Pythonistas will cringe at my gratuitous use of vars(), but Python
currently provides no method of using getattr()/setattr() to get/set
module-level variables. getattr(__main_ _,...), anyone?

[Darren Dale]

Re: saving and reloading variables for interactive work

>[color=blue]
> Unfortunately, there is no quick & easy way to do this in Python, since
> there is no easy way to differentiate unpicklable things (such as modules
> and functions) from picklable ones (your variables), nor to make Pickle
> gracefully ignore such objects. If you can get all your variable names
> into a list, however, you could use this code:
>[/color]

Could you determine the picklable types using something like this? Its
kinda ugly, but maybe something related would work:

myvars = {}
for key in vars().keys():
myvar = vars()[key]
if str(type(myvar) ) == "<type 'module'>": pass
elif: str(type(myvar) ) == "<type 'function'>": pass
elif: str(type(myvar) ) == "<type 'unpicklable'>" : pass
else: myvars.update({ key:myvar})

pickle.dump(myv ars,open('myvar s','wb'),-1)


Maybe a try/except would be appropriate, but the pickle docs ay that an
unspecified number of bytes may have been written to the file before the
PicklingError exception is raised.

Further comments would be greatly appreciated.

[Christopher T King]

Re: saving and reloading variables for interactive work

On Wed, 28 Jul 2004, Darren Dale wrote:
[color=blue]
> Could you determine the picklable types using something like this? Its
> kinda ugly, but maybe something related would work:
>
> myvars = {}
> for key in vars().keys():
> myvar = vars()[key]
> if str(type(myvar) ) == "<type 'module'>": pass
> elif: str(type(myvar) ) == "<type 'function'>": pass
> elif: str(type(myvar) ) == "<type 'unpicklable'>" : pass
> else: myvars.update({ key:myvar})[/color]

You can do this a bit more directly using the types module to check the
types (with a little bit of code restructuring):

from types import *
myvars = {}
for key,myvar in vars().items():
if not isinstance(myva r,(ModuleType,F unctionType)):
myvars[key]=myvar

A list comprehension would get you a nice, unreadable one-liner if it's so
desired:

from types import *
myvars = dict([(key,myvar) for key,myvar in vars().items() \
if not isinstance(myva r,(ModuleType,F unctionType))])

The problem with these types is there's no real guarantee what can be
pickled and what can't, and of those things that can be pickled, what
makes sense (functions can be pickled, but their code isn't stored).
There's currently no ispicklable() function
[color=blue]
> Maybe a try/except would be appropriate, but the pickle docs ay that an
> unspecified number of bytes may have been written to the file before the
> PicklingError exception is raised.[/color]

You could use dumps() instead of dump() for testing picklability before
picklation, though it smells of hackishness:

def ispicklable(ite m):
try:
pickle.dumps(it em)
except PicklingError:
return False
return True

Then you can just use ispicklable() in place of the 'not isinstance()'
mess I wrote.

I've done some poking though, and there a better way than that. It seems
all picklable objects define a __reduce__ method, so you could generalize
this check like this:

def ispicklable(ite m):
try:
item.__reduce__ ()
except KeyError,TypeEr ror:
return False
return True

Dunno if any of this helps; sorry if parts make no sense, I've retyped
parts of this a couple times :P

[Darren Dale]

Re: saving and reloading variables for interactive work

>[color=blue]
> I've done some poking though, and there a better way than that. It seems
> all picklable objects define a __reduce__ method, so you could generalize
> this check like this:
>
> def ispicklable(ite m):
> try:
> item.__reduce__ ()
> except KeyError,TypeEr ror:
> return False
> return True
>[/color]

I think this may only be true for new-style classes, or objects that
pickle "is unfamiliar with". I tried calling this method for strings,
dicts, lists, and it complains that these are all unpicklable.
object().__redu ce__() does work though.

I'll give it some thought.