Injecting code into a function

[George Sakkis]

Is there a general way of injecting code into a function, typically
before and/or after the existing code ? I know that for most purposes,
an OO solution, such as the template pattern, is a cleaner way to get
the same effect, but it's not always applicable (e.g. if you have no
control over the design and you are given a function to start with). In
particular, I want to get access to the function's locals() just before
it exits, i.e. something like:

def analyzeLocals(f unc):
func_locals = {}
def probeFunc():
# insert func's code here
sys._getframe(1 ).f_locals["func_local s"].update(locals( ))
probeFunc()
# func_locals now contains func's locals

So, how can I add func's code in probeFunc so that the injected code
(the update line here) is always called before the function exits ?
That is, don't just inject it lexically in the end of the function if
there are more than one exit points. I guess a solution will involve a
good deal bytecode hacking, on which i know very little; if there's a
link to a (relatively) simple HOWTO, it would be very useful.

Thanks,
George

[ajikoe@gmail.com]

Re: Injecting code into a function

use eval.
eval will accept any string and treat the string as a code.

pujo

[Steffen Glückselig]

Re: Injecting code into a function

Perhalps metaclasses are of interest to you. You can decorate existing
methods with additional behavior using metaclasses. A simple example
can be found at http://soiland.no/software/logmeta
I've gathered some more links under

http://www.gungfu.de/facts/wiki/field.php?pagename=Main.MetaProgramming

regards
Steffen

[Ron]

Re: Injecting code into a function

George Sakkis wrote:
[color=blue]
> Is there a general way of injecting code into a function, typically
> before and/or after the existing code ? I know that for most purposes,
> an OO solution, such as the template pattern, is a cleaner way to get
> the same effect, but it's not always applicable (e.g. if you have no
> control over the design and you are given a function to start with). In
> particular, I want to get access to the function's locals() just before
> it exits, i.e. something like:
>
> def analyzeLocals(f unc):
> func_locals = {}
> def probeFunc():
> # insert func's code here
> sys._getframe(1 ).f_locals["func_local s"].update(locals( ))
> probeFunc()
> # func_locals now contains func's locals
>
> So, how can I add func's code in probeFunc so that the injected code
> (the update line here) is always called before the function exits ?
> That is, don't just inject it lexically in the end of the function if
> there are more than one exit points. I guess a solution will involve a
> good deal bytecode hacking, on which i know very little; if there's a
> link to a (relatively) simple HOWTO, it would be very useful.
>
> Thanks,
> George[/color]

I'd like to know this as well. :)

I think you will have to modify the function func in some way to get
locals when it exits.

def func():
x = 20
y = 40
func.locals = locals() # inserted line

func()
print func.locals


On a related note, I'd like to know how to import locals into a function.

Cheers,
Ron

[Paddy]

Re: Injecting code into a function

Try searching for: 'python aspect-oriented' as aspect oriented
programming is about modifying existing class-methods (not exactly
functions which is what you asked for).
You might also do a search for "AOP considered harmful"

http://www.infosun.fmi.uni-passau.de/st/papers/EIWAS04/stoerzer04aop_harmful.pdf

The main point is that when you are reading the source you don't know
what the code is as it may be augmented by an "external" change.

[Steve Holden]

Re: Injecting code into a function

George Sakkis wrote:[color=blue]
> Is there a general way of injecting code into a function, typically
> before and/or after the existing code ? I know that for most purposes,
> an OO solution, such as the template pattern, is a cleaner way to get
> the same effect, but it's not always applicable (e.g. if you have no
> control over the design and you are given a function to start with). In
> particular, I want to get access to the function's locals() just before
> it exits, i.e. something like:
>
> def analyzeLocals(f unc):
> func_locals = {}
> def probeFunc():
> # insert func's code here
> sys._getframe(1 ).f_locals["func_local s"].update(locals( ))
> probeFunc()
> # func_locals now contains func's locals
>
> So, how can I add func's code in probeFunc so that the injected code
> (the update line here) is always called before the function exits ?
> That is, don't just inject it lexically in the end of the function if
> there are more than one exit points. I guess a solution will involve a
> good deal bytecode hacking, on which i know very little; if there's a
> link to a (relatively) simple HOWTO, it would be very useful.
>
> Thanks,
> George
>[/color]
A decorator would seem to be the sensible way to do this, assuming you
are using Python 2.4.

def decorated(func) :
def wrapper(arg1, arg2, arg3):
print "Arg2:", arg2
func(arg1)
print "Arg3:", arg3
return wrapper

@decorated
def f1(x):
print "F1:", x

f1('ARG1', 'ARG2', 'ARG3')

Arg2: ARG2
F1: ARG1
Arg3: ARG3

All the decorator really does is compute one function from another.
There's been enough discussion on the list recently that I won't repeat
the theory.

regards
Steve
--
Steve Holden +1 703 861 4237 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/

[Lonnie Princehouse]

Re: Injecting code into a function

I expect you could combine the following with decorators as an easy way
to grab a function's locals just before it exits... you could also use
exec or eval to execute code in the function's local namespace.
---------------------------------------

# Try it:

def trace_returns(f rame, event, arg):
if event == 'return':
print "[frame locals just before return: %s]" % frame.f_locals
return trace_returns

def foo(a, b):
return a + b

import sys
sys.settrace(tr ace_returns)

foo(1,2)

[George Sakkis]

Re: Injecting code into a function

"Lonnie Princehouse" wrote:
[color=blue]
> I expect you could combine the following with decorators as an easy[/color]
way[color=blue]
> to grab a function's locals just before it exits... you could also[/color]
use[color=blue]
> exec or eval to execute code in the function's local namespace.
> ---------------------------------------
>
> # Try it:
>
> def trace_returns(f rame, event, arg):
> if event == 'return':
> print "[frame locals just before return: %s]" % frame.f_locals
> return trace_returns
>
> def foo(a, b):
> return a + b
>
> import sys
> sys.settrace(tr ace_returns)
>
> foo(1,2)
>[/color]

Thanks, that's the closest to what I wanted. A minor point I didn't
quite get from the documentation is how to set a local trace instead of
a global (sys) trace. Also, there's no sys.gettrace() to return the
current tracer; is there a way around this ?

George

[Kay Schluehr]

Re: Injecting code into a function

George Sakkis wrote:
[color=blue]
> Thanks, that's the closest to what I wanted. A minor point I didn't
> quite get from the documentation is how to set a local trace instead[/color]
of[color=blue]
> a global (sys) trace.[/color]

You never do. "Local" in this context only means that those local trace
functions are called inside the one global trace-function and return
the global trace function again.

In Lonnies example there are no local-trace functions at all.
[color=blue]
> Also, there's no sys.gettrace() to return the
> current tracer; is there a way around this ?[/color]

The default tracer is None i.e. no debugging. The programmer has to
control his tracer which might not be to hard:


class Analyzer:
def trace_returns(s elf, frame, event, arg):
if event == 'return':
self.func_local s = frame.f_locals
return self.trace_retu rns

def analyzeLocals(s elf, func,*args,**kw ):
sys.settrace(se lf.trace_return s)
func(*args,**kw )
sys.settrace(No ne)

Ciao,
Kay

[Lonnie Princehouse]

Re: Injecting code into a function

I don't know of a way to get the current global trace function. This
could certainly cause trouble if you're trying to be compatible with
other packages that want to use their own trace functions (like psyco,
or debuggers). Does anyone know how to get the global trace?

On the other hand, the local trace function is in the f_trace attribute
of a frame.

It looks like global trace functions only get the "call" event, and are
expected to return a local trace function that will receive "line" and
"return" events, so you will need a global trace in order to set local
traces (setting myframe.f_trace explicitly doesn't seem to do it).

[Kay Schluehr]

Re: Injecting code into a function


Lonnie Princehouse wrote:[color=blue]
> I don't know of a way to get the current global trace function. This
> could certainly cause trouble if you're trying to be compatible with
> other packages that want to use their own trace functions (like[/color]
psyco,[color=blue]
> or debuggers). Does anyone know how to get the global trace?
>
> On the other hand, the local trace function is in the f_trace[/color]
attribute[color=blue]
> of a frame.[/color]

Oh, I overlooked this. Then the solution becomes simple:

sys._getframe() .f_trace

Test:
[color=blue][color=green][color=darkred]
>>> an = Analyzer()
>>> sys.settrace(an .trace_returns)
>>> sys._getframe() .f_trace[/color][/color][/color]
<bound method Analyzer.trace_ returns of <__main__.Analy zer instance at
0x010015D0>>

[color=blue]
> It looks like global trace functions only get the "call" event, and[/color]
are[color=blue]
> expected to return a local trace function that will receive "line"[/color]
and[color=blue]
> "return" events, so you will need a global trace in order to set[/color]
local[color=blue]
> traces (setting myframe.f_trace explicitly doesn't seem to do it).[/color]

I think that Your trace_returns function is actually a global trace
that returns itself and does not handle the 'call' event.

If You look at the code in the debug-module bdb.py the 'call' event
gets handled by the local trace function dispactch_call( ):

def trace_dispatch( self, frame, event, arg):
if self.quitting:
return # None
if event == 'line':
return self.dispatch_l ine(frame)
if event == 'call':
return self.dispatch_c all(frame, arg)
if event == 'return':
return self.dispatch_r eturn(frame, arg)
if event == 'exception':
return self.dispatch_e xception(frame, arg)
print 'bdb.Bdb.dispat ch: unknown debugging event:', `event`
return self.trace_disp atch

Ciao,
Kay

[Bengt Richter]

Re: Injecting code into a function

On 25 Apr 2005 03:32:38 -0700, "George Sakkis" <gsakkis@rutger s.edu> wrote:
[color=blue]
>Is there a general way of injecting code into a function, typically
>before and/or after the existing code ? I know that for most purposes,
>an OO solution, such as the template pattern, is a cleaner way to get
>the same effect, but it's not always applicable (e.g. if you have no
>control over the design and you are given a function to start with). In
>particular, I want to get access to the function's locals() just before
>it exits, i.e. something like:
>
>def analyzeLocals(f unc):
> func_locals = {}
> def probeFunc():
> # insert func's code here
> sys._getframe(1 ).f_locals["func_local s"].update(locals( ))
> probeFunc()
> # func_locals now contains func's locals
>
>So, how can I add func's code in probeFunc so that the injected code
>(the update line here) is always called before the function exits ?
>That is, don't just inject it lexically in the end of the function if
>there are more than one exit points. I guess a solution will involve a
>good deal bytecode hacking, on which i know very little; if there's a
>link to a (relatively) simple HOWTO, it would be very useful.
>[/color]
I'm not clear on what your real goal is, but if you just want a snapshot
of what locals() is just before exiting func, that could be done with
a byte-code-hacking decorator with usage looking something like

#func defined before this
func_locals = {}
@getlocals(func , func_locals)
def probefunc(): pass

which would make a probefunc function that would be identical to func
except that before exiting, it would do func_locals.upd ate(locals()).
(you might want func_locals to be a list and do func_locals.app end(locals())
in case func is recursive and you are interested in the all the locals).

Alternatively, if this is a debugging thing, you might want to look into
sys.settrace -- as in this thing I cobbled together (not tested beyond what you see ;-):

----< tracelocals.py >----------------------------------------------------
class TraceLocals(obj ect):
from sys import settrace
def __init__(self, *names, **kw):
self.names = set(names)
self.func_local s = kw.get('func_lo cals', []) # [(name,locals()) , ...] tuples
def _gwatch(self, frame, event, arg):
"""
Global scope watcher. When a new scope is entered, returns the local
scope watching method _lwatch to do the work for that.
"""
if event=='call':
name = frame.f_code.co _name # name of executing scope
if name in self.names: return self._lwatch # name is one whose locals we want

def _lwatch(self, frame, event, arg):
if event == 'return':
self.func_local s.append((frame .f_code.co_name ,frame.f_locals ))
else:
return self._lwatch # keep watching for return event

def on(self):
"""Set the system trace hook to enable tracing. """
self.settrace(s elf._gwatch)
def off(self):
"""Reset the system trace hook to disable tracing. """
self.settrace(N one)

def main(modname, entry, *names):
print 'main(', modname, entry, names,')'
tr = TraceLocals(*na mes)
mod = __import__(modn ame)
try:
tr.on()
getattr(mod, entry)()
finally:
tr.off()
return tr.func_locals

def test():
tr = TraceLocals(*'t 1 t2 t3'.split())
def t1():
x ='t1'
def t2(y=123):
y*=2
def t3():
t1()
t2()
t2('hello ')
try:
tr.on()
t3()
finally:
tr.off()
for name, loc in tr.func_locals: print '%5s: %s' %(name, loc)

if __name__=='__ma in__':
import sys
args = sys.argv[1:]
if not args:
raise SystemExit(
'Usage: python tracelocals.py (-test | module entry name+)\n'
)
if args[0]=='-test': test()
else:
print args
func_locals = main(args[0], args[1], *args[2:])
for name, loc in func_locals: print '%5s: %s' %(name, loc)
--------------------------------------------------------------------------
Test result:

[22:37] C:\pywk\clp\sak kis\tracelocals >py24 tracelocals.py -test
t1: {'x': 't1'}
t2: {'y': 246}
t2: {'y': 'hello hello '}
t3: {'t2': <function t2 at 0x02EE8ED4>, 't1': <function t1 at 0x02EE8E9C>}

Note that t3 is seeing t1 and t2 in its locals -- I think because they're
visible as cell vars in test. If you put t1-t3 in a separate module, you don't see it:

----< tmod.py >---------------
def t1():
print '-- t1'
x ='t1'
def t2(y=123):
print '-- t2'
y*=2
def t3():
print '-- t3'
t1()
t2()
t2('hello ')
-----------------------------

[22:42] C:\pywk\clp\sak kis\tracelocals >py24 tracelocals.py tmod t3 t1 t2 t3
['tmod', 't3', 't1', 't2', 't3']
main( tmod t3 ('t1', 't2', 't3') )
-- t3
-- t1
-- t2
-- t2
t1: {'x': 't1'}
t2: {'y': 246}
t2: {'y': 'hello hello '}
t3: {}

[22:46] C:\pywk\clp\sak kis\tracelocals >py24 tracelocals.py tmod t3 t2
['tmod', 't3', 't2']
main( tmod t3 ('t2',) )
-- t3
-- t1
-- t2
-- t2
t2: {'y': 246}
t2: {'y': 'hello hello '}

Notice that the -- side effects from all being called in the last, but only t2 being captured.

Maybe this will give you something to expand to your needs.

Regards,
Bengt Richter

[George Sakkis]

Re: Injecting code into a function

> Oh, I overlooked this. Then the solution becomes simple:[color=blue]
>
> sys._getframe() .f_trace
>
> Test:
>[color=green][color=darkred]
> >>> an = Analyzer()
> >>> sys.settrace(an .trace_returns)
> >>> sys._getframe() .f_trace[/color][/color]
> <bound method Analyzer.trace_ returns of <__main__.Analy zer instance[/color]
at[color=blue]
> 0x010015D0>>[/color]

Does this work for you non-interactively ? I tried running it from a
script or importing it from a module but it returns None. Very
strange...

George

[George Sakkis]

Re: Injecting code into a function

> I'm not clear on what your real goal is, but if you just want a
snapshot[color=blue]
> of what locals() is just before exiting func, that could be done with
> a byte-code-hacking decorator with usage looking something like
>
> #func defined before this
> func_locals = {}
> @getlocals(func , func_locals)
> def probefunc(): pass
>
> which would make a probefunc function that would be identical to func
> except that before exiting, it would do func_locals.upd ate(locals()).
> (you might want func_locals to be a list and do[/color]
func_locals.app end(locals())[color=blue]
> in case func is recursive and you are interested in the all the[/color]
locals).

That's all good, at least if I knew how to poke with bytecodes ;-)
What's a good starting point to look at ?

By the way, the original problem was yet another property packager,
which I posted as recipe at the cookbook:
http://aspn.activestate.com/ASPN/Coo.../Recipe/410698.

George

[Kay Schluehr]

Re: Injecting code into a function


George Sakkis wrote:[color=blue][color=green]
> > Oh, I overlooked this. Then the solution becomes simple:
> >
> > sys._getframe() .f_trace
> >
> > Test:
> >[color=darkred]
> > >>> an = Analyzer()
> > >>> sys.settrace(an .trace_returns)
> > >>> sys._getframe() .f_trace[/color]
> > <bound method Analyzer.trace_ returns of <__main__.Analy zer instance[/color]
> at[color=green]
> > 0x010015D0>>[/color]
>
> Does this work for you non-interactively ? I tried running it from a
> script or importing it from a module but it returns None. Very
> strange...
>
> George[/color]

You are right. The expression was context-dependent :-/

I had not yet the time to analyze the output of the following function
but it returns stable values:

def gettracefunc():
import sys
i = 0
while 1:
try:
f_trace = sys._getframe(i ).f_trace
if f_trace:
return f_trace
i+=1
except ValueError:
break

Ciao,
Kay