Finding a module's sub modules at runtime

[Joshua J. Kugler]

[If this is documented somewhere, please just point me there. I googled on
the terms that made sense to me, and didn't find anything.]

So, I have:

ModTest
__init__.py
AModule.py
BModule.py
CModule.py

All works fine. However, when I import ModTest, I would like it to discover
and store the names of the modules beneath it, and construct a list, say
mod_list, that I can access later to find the names of the sub-modules in
this module. Kind of setting __all__ at run time, I guess (yes, I'm aware
of the case caveats).

I figured __init__.py coudl take its own __path__ and walk the directory to
find all .py files other than __init__.py, but that seemed hackish. Is
there an "official" way to do this? Or a better way?

To give "context:" all the modules will have classes that have the same
name, same methods etc. One of the modules will be picked depending on
which implementation is needed.

Thanks!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer

http://www.eeinternet.com

PGP Key: http://pgp.mit.edu/  ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

[kyosohma@gmail.com]

Re: Finding a module's sub modules at runtime

On Mar 28, 2:44 pm, "Joshua J. Kugler" <jos...@eeinter net.comwrote:

[If this is documented somewhere, please just point me there. I googled on
the terms that made sense to me, and didn't find anything.]
>
So, I have:
>
ModTest
__init__.py
AModule.py
BModule.py
CModule.py
>
All works fine. However, when I import ModTest, I would like it to discover
and store the names of the modules beneath it, and construct a list, say
mod_list, that I can access later to find the names of the sub-modules in
this module. Kind of setting __all__ at run time, I guess (yes, I'm aware
of the case caveats).
>
I figured __init__.py coudl take its own __path__ and walk the directory to
find all .py files other than __init__.py, but that seemed hackish. Is
there an "official" way to do this? Or a better way?
>
To give "context:" all the modules will have classes that have the same
name, same methods etc. One of the modules will be picked depending on
which implementation is needed.
>
Thanks!
>
j
>
--
Joshua Kugler
Lead System Admin -- Senior Programmerhttp://www.eeinternet. com
PGP Key:http://pgp.mit.edu/ ID 0xDB26D7CE
>
--
Posted via a free Usenet account fromhttp://www.teranews.co m

I think you need to research how to create documentation. When I
import a module/package, I can then type help(moduleName ) and it'll
give me the module or package's contents.

http://mail.python.org/pipermail/python-list/2003-February/192069.html

You may be able to figure out how to do this just be studying the
"help" module itself.

Mike

[Joshua J. Kugler]

Re: Finding a module's sub modules at runtime

On Wednesday 28 March 2007 12:04, kyosohma@gmail. com wrote:

>All works fine. However, when I import ModTest, I would like it to
>discover and store the names of the modules beneath it, and construct a
>list, say mod_list, that I can access later to find the names of the
>sub-modules in
>this module. Kind of setting __all__ at run time, I guess (yes, I'm
>aware of the case caveats).
>>
>I figured __init__.py coudl take its own __path__ and walk the directory
>to
>find all .py files other than __init__.py, but that seemed hackish. Is
>there an "official" way to do this? Or a better way?
>>
>To give "context:" all the modules will have classes that have the same
>name, same methods etc. One of the modules will be picked depending on
>which implementation is needed.

>
I think you need to research how to create documentation. When I
import a module/package, I can then type help(moduleName ) and it'll
give me the module or package's contents.
>

http://mail.python.org/pipermail/python-list/2003-February/192069.html

>
You may be able to figure out how to do this just be studying the
"help" module itself.
>
Mike

Well, it seems the "help" module is a built in, and has no .py file
anywhere. The epydoc package uses the imports.file_mo dules(dirname)
function, which just walks the directory tree. Thanks for the
pointer...just confirmed that I have to do something I wanted to avoid.
But I guess if I use Python's os module and the __path__ string, it should
still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer

http://www.eeinternet.com

PGP Key: http://pgp.mit.edu/  ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

[Alex Martelli]

Re: Finding a module's sub modules at runtime

Joshua J. Kugler <joshua@eeinter net.comwrote:

still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.

Sorry, where is Python "gathering that information anyway"? Unless I'm
mistaken, Python by default does not walk directory trees of subpackages
-- what makes you think it does?


Alex

[Joshua J. Kugler]

Re: Finding a module's sub modules at runtime

On Thursday 29 March 2007 07:33, Alex Martelli wrote:

Joshua J. Kugler <joshua@eeinter net.comwrote:
>

>still be nicely portable. It just seems that since Python is gathering
>that information anyway, it should make it available without me having to
>walk the directory tree.

>
Sorry, where is Python "gathering that information anyway"? Unless I'm
mistaken, Python by default does not walk directory trees of subpackages
-- what makes you think it does?

Hmm...right: dynamic language, runtime binding. It would go out and find
whether or not Module1.Module2 existed only when import was called. Sorry,
my bad. I am assuming, however, that the help function walks the directory
tree, or else I would not get output like this:

>>import ModTest
>>help(ModTes t)

Help on package ModTest:

NAME
ModTest

FILE
/usr/local/lib/python2.4/site-packages/ModTest/__init__.py

PACKAGE CONTENTS
AModule
BModule
CModule

Thanks for the clarification.

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer

http://www.eeinternet.com

PGP Key: http://pgp.mit.edu/  ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

[Alex Martelli]

Re: Finding a module's sub modules at runtime

Joshua J. Kugler <joshua@eeinter net.comwrote:

On Thursday 29 March 2007 07:33, Alex Martelli wrote:
>

Joshua J. Kugler <joshua@eeinter net.comwrote:

still be nicely portable. It just seems that since Python is gathering
that information anyway, it should make it available without me having to
walk the directory tree.

Sorry, where is Python "gathering that information anyway"? Unless I'm
mistaken, Python by default does not walk directory trees of subpackages
-- what makes you think it does?

>
Hmm...right: dynamic language, runtime binding. It would go out and find
whether or not Module1.Module2 existed only when import was called. Sorry,
my bad. I am assuming, however, that the help function walks the directory
tree, or else I would not get output like this:
>

>import ModTest
>help(ModTest )

Help on package ModTest:
>
NAME
ModTest
>
FILE
/usr/local/lib/python2.4/site-packages/ModTest/__init__.py
>
PACKAGE CONTENTS
AModule
BModule
CModule
>
Thanks for the clarification.

Sure, pydoc (which help calls under the code) does that, with a nice mix
of inspect, os, and pkgutil.iter_mo dules calls. pkgutil.iter_mo dules
may in fact be most of what you need:

>>help(pkgutil. iter_modules)

Help on function iter_modules in module pkgutil:

iter_modules(pa th=None, prefix='')
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.

'path' should be either None or a list of paths to look for
modules in.

'prefix' is a string to output on the front of every module name
on output.

You will still have to call it recursively, though, I believe.


Alex

[Joshua J. Kugler]

Re: Finding a module's sub modules at runtime

On Thursday 29 March 2007 17:58, Alex Martelli wrote:

Sure, pydoc (which help calls under the code) does that, with a nice mix
of inspect, os, and pkgutil.iter_mo dules calls. pkgutil.iter_mo dules
may in fact be most of what you need:
>

>>>help(pkgutil .iter_modules)

Help on function iter_modules in module pkgutil:
>
iter_modules(pa th=None, prefix='')
Yields (module_loader, name, ispkg) for all submodules on path,
or, if path is None, all top-level modules on sys.path.
>
'path' should be either None or a list of paths to look for
modules in.
>
'prefix' is a string to output on the front of every module name
on output.

OK, that looks nice...but what version of Python is that?
http://docs.python.org/lib/module-pkgutil.html only shows one function (and
that's 2.5) and my python 2.4 installation is similarly lacking an
iter_modules() function for the pkgutil module. Is this a 2.6 thing?

Suppose I could just copy the code from here:
http://mail.python.org/pipermail/pyt...il/051452.html and
add it to my module. :)

Thanks for the pointer!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer

http://www.eeinternet.com

PGP Key: http://pgp.mit.edu/  ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com

[Robert Kern]

Re: Finding a module's sub modules at runtime

Joshua J. Kugler wrote:

On Thursday 29 March 2007 17:58, Alex Martelli wrote:

>Sure, pydoc (which help calls under the code) does that, with a nice mix
>of inspect, os, and pkgutil.iter_mo dules calls. pkgutil.iter_mo dules
>may in fact be most of what you need:
>>

>>>>help(pkguti l.iter_modules)

>Help on function iter_modules in module pkgutil:
>>
>iter_modules(p ath=None, prefix='')
> Yields (module_loader, name, ispkg) for all submodules on path,
> or, if path is None, all top-level modules on sys.path.
>>
> 'path' should be either None or a list of paths to look for
> modules in.
>>
> 'prefix' is a string to output on the front of every module name
> on output.

>
OK, that looks nice...but what version of Python is that?
http://docs.python.org/lib/module-pkgutil.html only shows one function (and
that's 2.5) and my python 2.4 installation is similarly lacking an
iter_modules() function for the pkgutil module. Is this a 2.6 thing?

No, 2.5. The documentation is not up to date. Read the source.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

[Joshua J. Kugler]

Re: Finding a module's sub modules at runtime

On Monday 02 April 2007 16:33, Robert Kern wrote:

>>>>>help(pkgut il.iter_modules )
>>Help on function iter_modules in module pkgutil:
>>>
>>iter_modules( path=None, prefix='')
>> Yields (module_loader, name, ispkg) for all submodules on path,
>> or, if path is None, all top-level modules on sys.path.
>>>
>> 'path' should be either None or a list of paths to look for
>> modules in.
>>>
>> 'prefix' is a string to output on the front of every module name
>> on output.

>>
>OK, that looks nice...but what version of Python is that?
>http://docs.python.org/lib/module-pkgutil.html only shows one function
>(and that's 2.5) and my python 2.4 installation is similarly lacking an
>iter_modules () function for the pkgutil module. Is this a 2.6 thing?

>
No, 2.5. The documentation is not up to date. Read the source.
>

Gotcha. Thanks...well, since we're using 2.4, that will have to wait. For
the archives, here is what I've come up with.

Contents of the __init__.py for a module.

import os
_myDir = __path__[0]

def mod_list():
"""
A quick hack that retrieves all the sub modules in a directory that has
an __init__.py file. I could use pkgutil.iter_mo dules, but that is
Python 2.5 only, and this should work with several versions of Python.
"""
modList = []
modHash = {}
isModule = False
for ii in os.walk(_myDir) :
if ii[0] == _myDir:
for f in ii[2]:
# If there is no __init__ file, then the directory
# upon which mod_list() is operating is not a module
if f[0:8] == '__init__':
isModule = True
elif f[-3:] == '.py':
modHash[f[:-3]] = True
elif f[-4:] == '.pyc' or f[-4:] == '.pyo':
modHash[f[:-4]] = True
if isModule:
modList = modHash.keys()
modList.sort()
return(modList)
else:
return(None)

Hope that helps someone!

j

--
Joshua Kugler
Lead System Admin -- Senior Programmer

http://www.eeinternet.com

PGP Key: http://pgp.mit.edu/  ID 0xDB26D7CE

--
Posted via a free Usenet account from http://www.teranews.com