Creating (rather) generic plugin framework?

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Edvard Majakari

    #1

    Creating (rather) generic plugin framework?

    Hi,

    My idea is to create a system working as follows: each module knows
    path to plugin directory, and that directory contains modules which
    may add hooks to some points in the code.

    Inspired by http://www.python.org/pycon/2005/pap...onHooking.html

    I would create a class like this:

    class Printer:

    def printit(self, msg):
    stuff = self.beforePrin tHook(msg)
    if stuff:
    msg = stuff
    print msg
    self.afterPrint Hook(msg)

    def beforePrintHook (self, msg): pass
    def afterPrintHook( self, msg): pass

    Now, in the spirit of py.test, I'd like API to be practically no API at all :)
    moreover, deploying a plugin must consist simply of adding appropriate file to
    plugins directory, nothing more, and removing it would uninstall it. The
    plugin should be able to somehow result in all future invocations to
    Printer.printit () call hooks specified in the plugin. Now, the plugin module
    for class above /might/ be along the following lines (I'm thinking of stuff
    here, so I don't know yet what would be the most appropriate way):

    ### a very simple plugin which uppercases all data fed to it.

    extensions = {'Printer': 'PrinterHook'}

    class PrinterHook:

    def beforePrintHook (self, msg):
    return msg.upper()
    def afterPrintHook( self, msg):
    print "Called afterPrintHook with msg %s" % msg


    Now, I have a very rude (I think) implementation which has two methods, first
    the one that loads plugin modules:

    def find_plugins():
    mods = [mod for mod in os.listdir(PLUG IN_DIR) if mod.endswith('. py')]

    # for each module in plugin dir, import the module and setup hooks. Hooks
    # must have equal method names in plugin module as in the original class.
    for mod in mods:
    name = os.path.splitex t(mod)[0]
    fobj, fpath, desc = imp.find_module (os.path.join(P LUGIN_DIR, name))
    module = imp.load_module (name, fobj, fpath, desc)
    set_hooks(modul e)

    ....then the other that is responsible for setting up hooks

    def set_hooks(mod):
    # mod.extensions has "base" class names as keys, (hook, priority) as
    # values
    for k, hook in mod.extensions. items():
    # get class object
    hook_cls = mod.__dict__[hook]

    try:
    base = globals()[k]
    except KeyError:
    print "No such class to insert hooks to:", k
    else:
    for item in base.__dict__:
    if item.endswith(' Hook'):
    # Override original (no-op) hook method
    # uhh.. kludgety kludge
    base.__dict__[item] = hook_cls.__dict __[item]

    now, my plugindemo.py just does as follows:

    find_plugins()
    p = Printer()
    p.printit('Hell o, world!')

    which prints

    $ python2.4 plugindemo.py
    HELLO, WORLD!
    Called afterPrintHook with msg HELLO, WORLD!

    But hey, this has many downsides. First off, mechanism doesn't support
    arbitrary namespaces. Here, class identifier in the plugin must be as it is
    seen from the module which calls the plugin (not a problem for me, but could
    be more elegant; probably a mapping between logical class identifiers and
    actual class names, hmm?). Second, if one wants to use many hooks (with
    priority for conflicts), it is not possible now; set_hooks always overrides
    potentially existing hooks. And probably many other problems that are not
    obvious to me, but for the simple purpose I have in mind, it seems to work.

    This is the first plugin system in Python I'm writing, so I can be a way off
    the correct path..
  • bruno at modulix

    #2
    Re: Creating (rather) generic plugin framework?

    Edvard Majakari wrote:[color=blue]
    > Hi,
    >
    > My idea is to create a system working as follows: each module knows
    > path to plugin directory, and that directory contains modules which
    > may add hooks to some points in the code.
    >
    > Inspired by http://www.python.org/pycon/2005/pap...onHooking.html
    >
    > I would create a class like this:
    >
    > class Printer:
    >
    > def printit(self, msg):
    > stuff = self.beforePrin tHook(msg)
    > if stuff:
    > msg = stuff
    > print msg
    > self.afterPrint Hook(msg)
    >
    > def beforePrintHook (self, msg): pass
    > def afterPrintHook( self, msg): pass
    >
    > Now, in the spirit of py.test, I'd like API to be practically no API at all :)
    > moreover, deploying a plugin must consist simply of adding appropriate file to
    > plugins directory, nothing more, and removing it would uninstall it. The
    > plugin should be able to somehow result in all future invocations to
    > Printer.printit () call hooks specified in the plugin. Now, the plugin module
    > for class above /might/ be along the following lines (I'm thinking of stuff
    > here, so I don't know yet what would be the most appropriate way):
    >
    > ### a very simple plugin which uppercases all data fed to it.
    >
    > extensions = {'Printer': 'PrinterHook'}
    >
    > class PrinterHook:
    >
    > def beforePrintHook (self, msg):
    > return msg.upper()
    > def afterPrintHook( self, msg):
    > print "Called afterPrintHook with msg %s" % msg
    >
    >
    > Now, I have a very rude (I think) implementation which has two methods, first
    > the one that loads plugin modules:
    >[/color]

    (snip code)
    [color=blue]
    >
    > But hey, this has many downsides. First off, mechanism doesn't support
    > arbitrary namespaces. Here, class identifier in the plugin must be as it is
    > seen from the module which calls the plugin (not a problem for me, but could
    > be more elegant; probably a mapping between logical class identifiers and
    > actual class names, hmm?). Second, if one wants to use many hooks (with
    > priority for conflicts), it is not possible now; set_hooks always overrides
    > potentially existing hooks. And probably many other problems that are not
    > obvious to me, but for the simple purpose I have in mind, it seems to work.[/color]

    Just a couple of ideas:
    - using decorators for plugin hooks ? ie:

    import hooks

    class Whatever(anythi ng):
    @hooks.hook(for ='Printer.befor ePrintHook',pri ority=42)
    def my_function_wit h_a_long_name(s elf, *args, **kw):
    pass

    The decorator would take care of "registerin g" the hook where relevant,
    ie, storing it in a class attribute of the hooked class ?

    which leads to:

    - in the hooked class, use a dict class attribute for hooks:

    from hooks import run_hooks

    class Printer
    # will be filled (and could even be created)
    # by the @hook decorator
    _hooks = {}

    def print(self, msg):
    # run_hooks will take care of selecting appropriate
    # hooks (by looking up the class attribute _hooks)
    # and running'em in order
    msg = run_hooks(self, 'Printer.before PrintHook', msg)
    print msg
    run_hooks(self, 'Printer.afterP rintHook', msg)



    My 2 cents... I don't even know if this can be implemented (but I don't
    see why it couldn't).
    [color=blue]
    > This is the first plugin system in Python I'm writing, so I can be a way off
    > the correct path..[/color]

    <aol>

    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in 'onurb@xiludom. gro'.split('@')])"

    Comment

    • bruno at modulix

      #3
      Re: Creating (rather) generic plugin framework?

      Edvard Majakari wrote:[color=blue]
      > Hi,
      >
      > My idea is to create a system working as follows: each module knows
      > path to plugin directory, and that directory contains modules which
      > may add hooks to some points in the code.
      >
      > Inspired by http://www.python.org/pycon/2005/pap...onHooking.html
      >
      > I would create a class like this:
      >
      > class Printer:
      >
      > def printit(self, msg):
      > stuff = self.beforePrin tHook(msg)
      > if stuff:
      > msg = stuff
      > print msg
      > self.afterPrint Hook(msg)
      >
      > def beforePrintHook (self, msg): pass
      > def afterPrintHook( self, msg): pass
      >
      > Now, in the spirit of py.test, I'd like API to be practically no API at all :)
      > moreover, deploying a plugin must consist simply of adding appropriate file to
      > plugins directory, nothing more, and removing it would uninstall it. The
      > plugin should be able to somehow result in all future invocations to
      > Printer.printit () call hooks specified in the plugin. Now, the plugin module
      > for class above /might/ be along the following lines (I'm thinking of stuff
      > here, so I don't know yet what would be the most appropriate way):
      >
      > ### a very simple plugin which uppercases all data fed to it.
      >
      > extensions = {'Printer': 'PrinterHook'}
      >
      > class PrinterHook:
      >
      > def beforePrintHook (self, msg):
      > return msg.upper()
      > def afterPrintHook( self, msg):
      > print "Called afterPrintHook with msg %s" % msg
      >
      >
      > Now, I have a very rude (I think) implementation which has two methods, first
      > the one that loads plugin modules:
      >[/color]

      (snip code)
      [color=blue]
      >
      > But hey, this has many downsides. First off, mechanism doesn't support
      > arbitrary namespaces. Here, class identifier in the plugin must be as it is
      > seen from the module which calls the plugin (not a problem for me, but could
      > be more elegant; probably a mapping between logical class identifiers and
      > actual class names, hmm?). Second, if one wants to use many hooks (with
      > priority for conflicts), it is not possible now; set_hooks always overrides
      > potentially existing hooks. And probably many other problems that are not
      > obvious to me, but for the simple purpose I have in mind, it seems to work.[/color]

      Just a couple of ideas:
      - using decorators for plugin hooks ? ie:

      import hooks

      class Whatever(anythi ng):
      @hooks.hook(for ='Printer.befor ePrintHook',pri ority=42)
      def my_function_wit h_a_long_name(s elf, *args, **kw):
      pass

      The decorator would take care of "registerin g" the hook where relevant,
      ie, storing it in a class attribute of the hooked class ?

      which leads to:

      - in the hooked class, use a dict class attribute for hooks:

      from hooks import run_hooks

      class Printer
      # will be filled (and could even be created)
      # by the @hook decorator
      _hooks = {}

      def print(self, msg):
      # run_hooks will take care of selecting appropriate
      # hooks (by looking up the class attribute _hooks)
      # and running'em in order
      msg = run_hooks(self, 'Printer.before PrintHook', msg)
      print msg
      run_hooks(self, 'Printer.afterP rintHook', msg)



      My 2 cents... I don't even know if this can be implemented (but I don't
      see why it couldn't).
      [color=blue]
      > This is the first plugin system in Python I'm writing, so I can be a way off
      > the correct path..[/color]

      <aol>

      --
      bruno desthuilliers
      python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
      p in 'onurb@xiludom. gro'.split('@')])"

      Comment

      Working...