Importing a class without knowing the module

[Franck PEREZ]

Hello,

I'm developing a small XML marshaller and I'm facing an annoying
issue. Here's some sample code:

########### My test application ############
class Foo(object):
#The class I'd like to serialize
pass

import myMarshaller
foo = Foo()
s = myMarshaller.du mps(foo) #works fine, spits something like <object
class = "Foo"...>
another_foo = loads(s) #fails, see below

########### My marshaller (in its own module) ############
def loads(s):
#First, get class name (here "Foo")
klass = eval(className) #fails because "Foo" is not in the
marshaller's namespace !

How could I tell the marshaller to locate the Foo class and any other
class I try to deserialize ? I've tried to pass my test application's
globals() to the marshaller, it works but it's dirty IMHO... I've
tried also to locate the class (here "Foo") somewhere in sys.modules
in the "loads" method, but it was heavy and unsuccessful.

Thanks a lot for your help !
Franck

[Mike Meyer]

Re: Importing a class without knowing the module

Franck PEREZ <franck.perez@g mail.com> writes:[color=blue]
> ########### My test application ############
> class Foo(object):
> #The class I'd like to serialize
> pass
>
> import myMarshaller
> foo = Foo()
> s = myMarshaller.du mps(foo) #works fine, spits something like <object
> class = "Foo"...>
> another_foo = loads(s) #fails, see below
>
> ########### My marshaller (in its own module) ############
> def loads(s):
> #First, get class name (here "Foo")
> klass = eval(className) #fails because "Foo" is not in the
> marshaller's namespace !
>
> How could I tell the marshaller to locate the Foo class and any other
> class I try to deserialize ?[/color]

How about adding Foo.__file__ to the serialized data?

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Mike Meyer]

Re: Importing a class without knowing the module

Franck PEREZ <franck.perez@g mail.com> writes:[color=blue]
> ########### My test application ############
> class Foo(object):
> #The class I'd like to serialize
> pass
>
> import myMarshaller
> foo = Foo()
> s = myMarshaller.du mps(foo) #works fine, spits something like <object
> class = "Foo"...>
> another_foo = loads(s) #fails, see below
>
> ########### My marshaller (in its own module) ############
> def loads(s):
> #First, get class name (here "Foo")
> klass = eval(className) #fails because "Foo" is not in the
> marshaller's namespace !
>
> How could I tell the marshaller to locate the Foo class and any other
> class I try to deserialize ?[/color]

How about adding Foo.__file__ to the serialized data?

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Franck PEREZ]

Re: Importing a class without knowing the module

I thought about it, but it would make the XML file depend on the
machine... no more portability...

On 11/18/05, Mike Meyer <mwm@mired.or g> wrote:[color=blue]
> Franck PEREZ <franck.perez@g mail.com> writes:[color=green]
> > ########### My test application ############
> > class Foo(object):
> > #The class I'd like to serialize
> > pass
> >
> > import myMarshaller
> > foo = Foo()
> > s = myMarshaller.du mps(foo) #works fine, spits something like <object
> > class = "Foo"...>
> > another_foo = loads(s) #fails, see below
> >
> > ########### My marshaller (in its own module) ############
> > def loads(s):
> > #First, get class name (here "Foo")
> > klass = eval(className) #fails because "Foo" is not in the
> > marshaller's namespace !
> >
> > How could I tell the marshaller to locate the Foo class and any other
> > class I try to deserialize ?[/color]
>
> How about adding Foo.__file__ to the serialized data?
>
> <mike
> --
> Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
> Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
> --
> http://mail.python.org/mailman/listinfo/python-list
>[/color]

[Franck PEREZ]

Re: Importing a class without knowing the module

I thought about it, but it would make the XML file depend on the
machine... no more portability...

On 11/18/05, Mike Meyer <mwm@mired.or g> wrote:[color=blue]
> Franck PEREZ <franck.perez@g mail.com> writes:[color=green]
> > ########### My test application ############
> > class Foo(object):
> > #The class I'd like to serialize
> > pass
> >
> > import myMarshaller
> > foo = Foo()
> > s = myMarshaller.du mps(foo) #works fine, spits something like <object
> > class = "Foo"...>
> > another_foo = loads(s) #fails, see below
> >
> > ########### My marshaller (in its own module) ############
> > def loads(s):
> > #First, get class name (here "Foo")
> > klass = eval(className) #fails because "Foo" is not in the
> > marshaller's namespace !
> >
> > How could I tell the marshaller to locate the Foo class and any other
> > class I try to deserialize ?[/color]
>
> How about adding Foo.__file__ to the serialized data?
>
> <mike
> --
> Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
> Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
> --
> http://mail.python.org/mailman/listinfo/python-list
>[/color]

[Alex Martelli]

Re: Importing a class without knowing the module

Franck PEREZ <franck.perez@g mail.com> wrote:
[color=blue]
> I thought about it, but it would make the XML file depend on the
> machine... no more portability...[/color]
...[color=blue][color=green]
> > How about adding Foo.__file__ to the serialized data?[/color][/color]

Your top-posting makes this discourse weird (why put your comments
BEFORE the text you're commenting on?!), but anyway I think that using
the class's __module__ rather than __file__ should be what you want.


Alex

[Alex Martelli]

Re: Importing a class without knowing the module

Franck PEREZ <franck.perez@g mail.com> wrote:
[color=blue]
> I thought about it, but it would make the XML file depend on the
> machine... no more portability...[/color]
...[color=blue][color=green]
> > How about adding Foo.__file__ to the serialized data?[/color][/color]

Your top-posting makes this discourse weird (why put your comments
BEFORE the text you're commenting on?!), but anyway I think that using
the class's __module__ rather than __file__ should be what you want.


Alex

[Mike Meyer]

Re: Importing a class without knowing the module

[Format recovered from top posting.]

Franck PEREZ <franck.perez@g mail.com> writes:[color=blue]
> On 11/18/05, Mike Meyer <mwm@mired.or g> wrote:[color=green]
>> Franck PEREZ <franck.perez@g mail.com> writes:[color=darkred]
>> > ########### My test application ############
>> > class Foo(object):
>> > #The class I'd like to serialize
>> > pass
>> >
>> > import myMarshaller
>> > foo = Foo()
>> > s = myMarshaller.du mps(foo) #works fine, spits something like <object
>> > class = "Foo"...>
>> > another_foo = loads(s) #fails, see below
>> >
>> > ########### My marshaller (in its own module) ############
>> > def loads(s):
>> > #First, get class name (here "Foo")
>> > klass = eval(className) #fails because "Foo" is not in the
>> > marshaller's namespace !
>> >
>> > How could I tell the marshaller to locate the Foo class and any other
>> > class I try to deserialize ?[/color]
>>
>> How about adding Foo.__file__ to the serialized data?[/color]
> I thought about it, but it would make the XML file depend on the
> machine... no more portability...[/color]

They already depend on the machine. You can't take them to an arbitary
machine and reconstruct them: it has to have the classes the XML file
depends on somewhere on it. You can use the module name if you have it
available. If not, deriving the module name from the file name is
about the best you can do.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Mike Meyer]

Re: Importing a class without knowing the module

[Format recovered from top posting.]

Franck PEREZ <franck.perez@g mail.com> writes:[color=blue]
> On 11/18/05, Mike Meyer <mwm@mired.or g> wrote:[color=green]
>> Franck PEREZ <franck.perez@g mail.com> writes:[color=darkred]
>> > ########### My test application ############
>> > class Foo(object):
>> > #The class I'd like to serialize
>> > pass
>> >
>> > import myMarshaller
>> > foo = Foo()
>> > s = myMarshaller.du mps(foo) #works fine, spits something like <object
>> > class = "Foo"...>
>> > another_foo = loads(s) #fails, see below
>> >
>> > ########### My marshaller (in its own module) ############
>> > def loads(s):
>> > #First, get class name (here "Foo")
>> > klass = eval(className) #fails because "Foo" is not in the
>> > marshaller's namespace !
>> >
>> > How could I tell the marshaller to locate the Foo class and any other
>> > class I try to deserialize ?[/color]
>>
>> How about adding Foo.__file__ to the serialized data?[/color]
> I thought about it, but it would make the XML file depend on the
> machine... no more portability...[/color]

They already depend on the machine. You can't take them to an arbitary
machine and reconstruct them: it has to have the classes the XML file
depends on somewhere on it. You can use the module name if you have it
available. If not, deriving the module name from the file name is
about the best you can do.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Alex Martelli]

Re: Importing a class without knowing the module

Mike Meyer <mwm@mired.or g> wrote:
...[color=blue][color=green][color=darkred]
> >> How about adding Foo.__file__ to the serialized data?[/color]
> > I thought about it, but it would make the XML file depend on the
> > machine... no more portability...[/color]
>
> They already depend on the machine. You can't take them to an arbitary
> machine and reconstruct them: it has to have the classes the XML file
> depends on somewhere on it. You can use the module name if you have it
> available. If not, deriving the module name from the file name is
> about the best you can do.[/color]

I disagree with the last sentence. From a filepath of
'/zip/zap/zop/zup.py', it's VERY hard to say whether the module name is
to be zup, zop.zup, or zap.zop.zup -- it depends on which directories
are on sys.path and which have __init__.py, which is impossible to tell
at unmarshaling time. If you use Foo.__module__, you should get the
*module* name correctly, independently from sys.path or __init__.py's,
or .pth files for that matter -- so, I do NOT agree that using __file__
is "about the best you can do" for this use case.


Alex

[Alex Martelli]

Re: Importing a class without knowing the module

Mike Meyer <mwm@mired.or g> wrote:
...[color=blue][color=green][color=darkred]
> >> How about adding Foo.__file__ to the serialized data?[/color]
> > I thought about it, but it would make the XML file depend on the
> > machine... no more portability...[/color]
>
> They already depend on the machine. You can't take them to an arbitary
> machine and reconstruct them: it has to have the classes the XML file
> depends on somewhere on it. You can use the module name if you have it
> available. If not, deriving the module name from the file name is
> about the best you can do.[/color]

I disagree with the last sentence. From a filepath of
'/zip/zap/zop/zup.py', it's VERY hard to say whether the module name is
to be zup, zop.zup, or zap.zop.zup -- it depends on which directories
are on sys.path and which have __init__.py, which is impossible to tell
at unmarshaling time. If you use Foo.__module__, you should get the
*module* name correctly, independently from sys.path or __init__.py's,
or .pth files for that matter -- so, I do NOT agree that using __file__
is "about the best you can do" for this use case.


Alex

[Mike Meyer]

Re: Importing a class without knowing the module

aleax@mail.comc ast.net (Alex Martelli) writes:[color=blue]
> Mike Meyer <mwm@mired.or g> wrote:[color=green][color=darkred]
>> >> How about adding Foo.__file__ to the serialized data?
>> > I thought about it, but it would make the XML file depend on the
>> > machine... no more portability...[/color]
>> They already depend on the machine. You can't take them to an arbitary
>> machine and reconstruct them: it has to have the classes the XML file
>> depends on somewhere on it. You can use the module name if you have it
>> available. If not, deriving the module name from the file name is
>> about the best you can do.[/color]
> I disagree with the last sentence. From a filepath of
> '/zip/zap/zop/zup.py', it's VERY hard to say whether the module name is
> to be zup, zop.zup, or zap.zop.zup -- it depends on which directories
> are on sys.path and which have __init__.py, which is impossible to tell
> at unmarshaling time. If you use Foo.__module__, you should get the
> *module* name correctly, independently from sys.path or __init__.py's,
> or .pth files for that matter -- so, I do NOT agree that using __file__
> is "about the best you can do" for this use case.[/color]

You should read the next-to-last sentence, which says to use the
module name if you have it. The last sentence starts "If not" -
meaning you don't have the module name. *That's* the case for which
the file name is about the best you can do.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Mike Meyer]

Re: Importing a class without knowing the module

aleax@mail.comc ast.net (Alex Martelli) writes:[color=blue]
> Mike Meyer <mwm@mired.or g> wrote:[color=green][color=darkred]
>> >> How about adding Foo.__file__ to the serialized data?
>> > I thought about it, but it would make the XML file depend on the
>> > machine... no more portability...[/color]
>> They already depend on the machine. You can't take them to an arbitary
>> machine and reconstruct them: it has to have the classes the XML file
>> depends on somewhere on it. You can use the module name if you have it
>> available. If not, deriving the module name from the file name is
>> about the best you can do.[/color]
> I disagree with the last sentence. From a filepath of
> '/zip/zap/zop/zup.py', it's VERY hard to say whether the module name is
> to be zup, zop.zup, or zap.zop.zup -- it depends on which directories
> are on sys.path and which have __init__.py, which is impossible to tell
> at unmarshaling time. If you use Foo.__module__, you should get the
> *module* name correctly, independently from sys.path or __init__.py's,
> or .pth files for that matter -- so, I do NOT agree that using __file__
> is "about the best you can do" for this use case.[/color]

You should read the next-to-last sentence, which says to use the
module name if you have it. The last sentence starts "If not" -
meaning you don't have the module name. *That's* the case for which
the file name is about the best you can do.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Alex Martelli]

Re: Importing a class without knowing the module

Mike Meyer <mwm@mired.or g> wrote:
...[color=blue][color=green][color=darkred]
> >> >> How about adding Foo.__file__ to the serialized data?[/color][/color][/color]
...[color=blue][color=green][color=darkred]
> >> depends on somewhere on it. You can use the module name if you have it
> >> available. If not, deriving the module name from the file name is
> >> about the best you can do.[/color]
> > I disagree with the last sentence. From a filepath of[/color][/color]
...[color=blue]
> You should read the next-to-last sentence, which says to use the
> module name if you have it. The last sentence starts "If not" -
> meaning you don't have the module name. *That's* the case for which
> the file name is about the best you can do.[/color]

I see! Thanks for clarifying. Could you please give me an example of a
Foo class which has a Foo.__file__ attribute but not a Foo.__module__
attribute? Sorry, must be some moment of weakness on my mind's part
(quite possible, since I am recovering from recent surgery), but I
cannot think of a situation where that would be the case (while classes
with __module__ and w/o __file__ are the normal situation). Were there
no case in which, given a class, you can learn about its file (by a
__file__ attribute) but not about its module (by a __module__
attribute), I would of course hope that my inability to parse that
sentence of yours, which would under such hypothetical circumstaces be
an absurd hypothesis, might be more forgivable.


Alex

[Alex Martelli]

Re: Importing a class without knowing the module

Mike Meyer <mwm@mired.or g> wrote:
...[color=blue][color=green][color=darkred]
> >> >> How about adding Foo.__file__ to the serialized data?[/color][/color][/color]
...[color=blue][color=green][color=darkred]
> >> depends on somewhere on it. You can use the module name if you have it
> >> available. If not, deriving the module name from the file name is
> >> about the best you can do.[/color]
> > I disagree with the last sentence. From a filepath of[/color][/color]
...[color=blue]
> You should read the next-to-last sentence, which says to use the
> module name if you have it. The last sentence starts "If not" -
> meaning you don't have the module name. *That's* the case for which
> the file name is about the best you can do.[/color]

I see! Thanks for clarifying. Could you please give me an example of a
Foo class which has a Foo.__file__ attribute but not a Foo.__module__
attribute? Sorry, must be some moment of weakness on my mind's part
(quite possible, since I am recovering from recent surgery), but I
cannot think of a situation where that would be the case (while classes
with __module__ and w/o __file__ are the normal situation). Were there
no case in which, given a class, you can learn about its file (by a
__file__ attribute) but not about its module (by a __module__
attribute), I would of course hope that my inability to parse that
sentence of yours, which would under such hypothetical circumstaces be
an absurd hypothesis, might be more forgivable.


Alex