Automatically loading and initialising objects from a pluginsdirectory

**Diez B. Roggisch** · Jul 22 '08, 12:45 PM

Re: Automatically loading and initialising objects from a plugins directory

Dave Challis wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
>
I'm trying to write some code which:
1. Finds all modules in a plugin directory
2. Imports those modules
3. Creates an instance of each object defined in the module (each module
will contain exactly 1 object, which is a subclass of 'Plugin')
>
The closest I've come so far is with something like:
>
In plugin.py:
# taken from http://docs.python.org/lib/built-in-funcs.html
def my_import(name) :
mod = __import__(name )
components = name.split('.')
for comp in components[1:]:
mod = getattr(mod, comp)
return mod
>
def import_plugins( ):
mods = []
for filename in os.listdir('/plugins'):
if filename.endswi th('.py'):
name = os.path.splitex t(filename)[0]
mods.append(my_ import('plugins .' + name))
return mods
>
class Plugin(object):
pass
>
>
In plugins/exampleplugin.p y:
class ExamplePlugin(P lugin):
def __init__(self):
pass
>
>
Calling import_plugins( ) then gives me a list containing references to
modules.
>
How can I loop through that list and create an instance of whatever
object was defined within the module? (In this case I'd want to
construct an instance of ExamplePlugin)

Like this:

for name in dir(plugin):
thing = getattr(plugin, name)
try:
if issubclass(thin g, Plugin):
thing()
except ValueError: # issubclass sucks
pass

Diez

**Dave Challis** · Jul 22 '08, 04:25 PM

Re: Automatically loading and initialising objects from a pluginsdirector y

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Diez B. Roggisch wrote:

Dave Challis wrote:
>

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>>
>I'm trying to write some code which:
>1. Finds all modules in a plugin directory
>2. Imports those modules
>3. Creates an instance of each object defined in the module (each module
>will contain exactly 1 object, which is a subclass of 'Plugin')
>>
>The closest I've come so far is with something like:
>>
>In plugin.py:
># taken from http://docs.python.org/lib/built-in-funcs.html
>def my_import(name) :
> mod = __import__(name )
> components = name.split('.')
> for comp in components[1:]:
> mod = getattr(mod, comp)
> return mod
>>
>def import_plugins( ):
> mods = []
> for filename in os.listdir('/plugins'):
> if filename.endswi th('.py'):
> name = os.path.splitex t(filename)[0]
> mods.append(my_ import('plugins .' + name))
> return mods
>>
>class Plugin(object):
> pass
>>
>>
>In plugins/exampleplugin.p y:
>class ExamplePlugin(P lugin):
> def __init__(self):
> pass
>>
>>
>Calling import_plugins( ) then gives me a list containing references to
>modules.
>>
>How can I loop through that list and create an instance of whatever
>object was defined within the module? (In this case I'd want to
>construct an instance of ExamplePlugin)

>
>
Like this:
>
for name in dir(plugin):
thing = getattr(plugin, name)
try:
if issubclass(thin g, Plugin):
thing()
except ValueError: # issubclass sucks
pass
>
Diez

Thanks for that, it helped as a starting point. I had some trouble with
using issubclass though (issubclass(Plu gin, Plugin) returns true), which
was complicating things.

I modified your code to the following instead (which may well have it's
own pitfalls I'm not aware of!):

for name in dir(plugin):
thing = getattr(plugin, name)
if hasattr(thing, '__bases__') and \
getattr(thing, '__bases__')[0] == Plugin:
thing()

Cheers,
Dave

- --
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~

>< o ) <><

. ( (
) Dave Challis <>< ) ) ><>
_(__dsc@ecs.sot on.ac.uk_______ _______________ _____(____(____ ______________
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIhge8v26 GZvAVVFERAh86AJ 91DVYPPZ/1x3rjoezFL1P5FX uHNwCZAXdr
DgjzmTNV1/H8vcQ/2Hax3js=
=ZAtM
-----END PGP SIGNATURE-----

**Fredrik Lundh** · Jul 22 '08, 04:35 PM

Re: Automatically loading and initialising objects from a pluginsdirector y

Dave Challis wrote:

Thanks for that, it helped as a starting point. I had some trouble with
using issubclass though (issubclass(Plu gin, Plugin) returns true), which
was complicating things.
>
I modified your code to the following instead (which may well have it's
own pitfalls I'm not aware of!):
>
for name in dir(plugin):
thing = getattr(plugin, name)
if hasattr(thing, '__bases__') and \
getattr(thing, '__bases__')[0] == Plugin:
thing()

so now you're no longer supporting mixins and multiple-level
inheritance? why not just tweak Diez' example a little:

for name in dir(plugin):

thing = getattr(plugin, name)

# make sure this is a plugin
try:
if thing is Plugin:
continue
if not issubclass(thin g, Plugin):
continue
except ValueError: # issubclass sucks
continue # not a class

thing() # probably needs error handling around this

(I also moved the thing call out of the thing validation part, to allow
you to distinguish between ValueErrors caused by issubclass and errors
caused by things)

(btw, another approach would be to use a metaclass to make Plugin
classes register themselves on import)

</F>

**Dave Challis** · Jul 25 '08, 11:35 AM

Re: Automatically loading and initialising objects from a pluginsdirector y

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Fredrik Lundh wrote:

so now you're no longer supporting mixins and multiple-level
inheritance? why not just tweak Diez' example a little:
>
for name in dir(plugin):
>
thing = getattr(plugin, name)
>
# make sure this is a plugin
try:
if thing is Plugin:
continue
if not issubclass(thin g, Plugin):
continue
except ValueError: # issubclass sucks
continue # not a class
>
thing() # probably needs error handling around this
>
(I also moved the thing call out of the thing validation part, to allow
you to distinguish between ValueErrors caused by issubclass and errors
caused by things)
>
(btw, another approach would be to use a metaclass to make Plugin
classes register themselves on import)
>
</F>

Thanks, that works much better. I've only been using python a couple of
months, so still getting to grips with how it treats classes/subclasses.

I'll have a look into metaclasses too, haven't stumbled upon those yet
at all.

Cheers,
Dave

- --
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
) <><
<>< (
Dave Challis )
____dsc@ecs.sot on.ac.uk_______(_______ _______________ _______________ ______
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIibiHv26 GZvAVVFERAgcmAJ 4tmC7jp6TTb3Dx2 Lw+rKSmJkcSLQCf YFsA
e0ZLOf8lQXhqHcz/Me8ok0E=
=Qcg7
-----END PGP SIGNATURE-----

**Fredrik Lundh** · Jul 25 '08, 11:55 AM

Re: Automatically loading and initialising objects from a pluginsdirector y

Dave Challis wrote:

I'll have a look into metaclasses too, haven't stumbled upon those yet
at all.

It's a potentially brain-exploding topic, though, so if the above
solution works for you, you might want to leave it at that ;-)

But very briefly, a metaclass is a something that's responsible for
creating a class, much like an ordinary class is responsible for
creating an object. When Python executes the following statement,

class Spam:
attrib = 1
def func(self):
pass
# <-- end of class statement

it will create a new scope for the class content, execute the class
body, and then, when it reaches the end, call the "metaclass" to create
the actual class object. The metaclass is given the requested name
("Spam" in this case), any base classes, and a dictionary containing
everything from the class scope ("attrib" and "func", in this case).
The thing that's returned is assigned to the "Spam" variable.

The default metaclass ("type") just creates an ordinary class object,
but if you replace that with your own metaclas, you can completely
override that behaviour, or just extend it (e.g. by registering the
subclasses in a common registry). Like, say, this:

registry = [] # list of subclasses

class Plugin(object):
class __metaclass__(t ype):
def __init__(cls, name, bases, dict):
type.__init__(n ame, bases, dict)
registry.append ((name, cls))

class SpamPlugin(Plug in):
pass

class BaconPlugin(Plu gin):
pass

for name, cls in registry:
if cls is not Plugin:
print name, cls

Here, the presence of an inner __metaclass__ class (which is a subclass
of "type") causes Python's class machinery to use that class instead of
"type" when creating class objects for Plugin or any subclass thereof.
The extra code in the __init__ method just all plugins to a list.

For more on this, see e.g.

Unifying types and classes in Python 2.2

http://www.python.org/download/releases/2.2/descrintro/#metaclasses

The official home of the Python Programming Language

</F>

**kpd** · Jul 29 '08, 01:55 PM

Re: Automatically loading and initialising objects from a pluginsdirector y

On Jul 25, 7:50 am, Fredrik Lundh <fred...@python ware.comwrote:

It's a potentially brain-exploding topic,

-that you made very understandable. Thanks for posting that
explanation and example.