Functions and code objects

**Fuzzyman** · Jul 27 '06, 12:45 PM

Re: Functions and code objects

Fuzzyman wrote:

Hello all,
>
I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.
>
The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.
>
How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?
>

Ok, so now I'm getting somewhere, without really knowing what I'm
doing. Using the CodeType I can create a new code object with identical
attributes, except an 'argcount' of 0.

It doesn't quite work, so I probably need to set some of the attributes
*differently*.

The code I use is :

>>def f(x):

... print x
...

>>c = f.func_code
>>CodeType = type(c)
>>a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
c.co_name,
... c.co_firstlinen o, c.co_lnotab, c.co_freevars, c.co_cellvars)

>>a

>>exec a

Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 2, in f
UnboundLocalErr or: local variable 'x' referenced before assignment

(the first argument, 0, becomes the 'arg_count' attribute)

So the code object is still creating a local scope, and knows that x is
a local variable. ('co_nlocals' and 'co_varnames' ?).

I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.

Anyone got any clues ?

The attributes mean (from http://pyref.infogami.com/type-code ) :

* co_name gives the function name
* co_argcount is the number of positional arguments (including
arguments with default values)
* co_nlocals is the number of local variables used by the function
(including arguments)
* co_varnames is a tuple containing the names of the local
variables (starting with the argument names)
* co_cellvars is a tuple containing the names of local variables
that are referenced by nested functions
* co_freevars is a tuple containing the names of free variables
* co_code is a string representing the sequence of bytecode
instructions
* co_consts is a tuple containing the literals used by the bytecode
* co_names is a tuple containing the names used by the bytecode
* co_filename is the filename from which the code was compiled
* co_firstlineno is the first line number of the function
* co_lnotab is a string encoding the mapping from byte code offsets
to line numbers (for details see the source code of the interpreter)
* co_stacksize is the required stack size (including local
variables)
* co_flags is an integer encoding a number of flags for the
interpreter.

In case anyone wonders, this is purely an experiment in creating
'annoymous code blocks'. :-)

What is a 'free variable' ?

'cell_vars' also looks interesting, but probably better not to mess
with it :-) (I guess it is only relevant for functions defined in the
code block)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

**Fuzzyman** · Jul 27 '06, 02:25 PM

Re: Functions and code objects

Fuzzyman wrote:

Fuzzyman wrote:

Hello all,

I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.

The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.

How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?

>
Ok, so now I'm getting somewhere, without really knowing what I'm
doing. Using the CodeType I can create a new code object with identical
attributes, except an 'argcount' of 0.
>
It doesn't quite work, so I probably need to set some of the attributes
*differently*.
>
The code I use is :
>

>def f(x):

... print x
...

>c = f.func_code
>CodeType = type(c)
>a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
c.co_name,
... c.co_firstlinen o, c.co_lnotab, c.co_freevars, c.co_cellvars)

>a

>exec a

Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 2, in f
UnboundLocalErr or: local variable 'x' referenced before assignment
>
(the first argument, 0, becomes the 'arg_count' attribute)
>
So the code object is still creating a local scope, and knows that x is
a local variable. ('co_nlocals' and 'co_varnames' ?).
>
I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.
>
Anyone got any clues ?
>

*Damn* I've extracted the code object and told it that it has no
arguments. Executing the code object results in the function object !

The code object is obviously the code object for the function
definition. *sigh*

I was hoping I could get to the code object for the *body* of the
function. Looks like that won't be possible without dis-assembling the
bytecode or other tricks even more hackish than what I've already done.

For the record, the code I was using was :

x = 3
def f(x):
print x

CodeType = type(f.func_cod e)

def convert_functio n(f):
code = f.func_code
nlocals = max(code.co_nlo cals - code.co_argcoun t, 0)
newCode = CodeType(0, nlocals, code.co_stacksi ze, code.co_flags,
code.co_code, code.co_consts, code.co_names,
code.co_varname s, code.co_filenam e,
code.co_name,
code.co_firstli neno, code.co_lnotab,
code.co_freevar s,
code.co_cellvar s)
return newCode

print convert_functio n(f)
exec convert_functio n(f)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

**Simon Forman** · Jul 27 '06, 02:35 PM

Re: Functions and code objects

Fuzzyman wrote:

Fuzzyman wrote:

Hello all,
>
I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.
>
The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.
>
How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?
>

Ok, so now I'm getting somewhere, without really knowing what I'm
doing. Using the CodeType I can create a new code object with identical
attributes, except an 'argcount' of 0.

It doesn't quite work, so I probably need to set some of the attributes
*differently*.

The code I use is :

>>def f(x):

... print x
...

>>c = f.func_code
>>CodeType = type(c)
>>a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
c.co_name,
... c.co_firstlinen o, c.co_lnotab, c.co_freevars, c.co_cellvars)

>>a

>>exec a

Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 2, in f
UnboundLocalErr or: local variable 'x' referenced before assignment

(the first argument, 0, becomes the 'arg_count' attribute)

So the code object is still creating a local scope, and knows that x is
a local variable. ('co_nlocals' and 'co_varnames' ?).

I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.

Anyone got any clues ?

>
*Damn* I've extracted the code object and told it that it has no
arguments. Executing the code object results in the function object !
>
The code object is obviously the code object for the function
definition. *sigh*
>
I was hoping I could get to the code object for the *body* of the
function. Looks like that won't be possible without dis-assembling the
bytecode or other tricks even more hackish than what I've already done.
>
For the record, the code I was using was :
>
x = 3
def f(x):
print x
>
CodeType = type(f.func_cod e)
>
def convert_functio n(f):
code = f.func_code
nlocals = max(code.co_nlo cals - code.co_argcoun t, 0)
newCode = CodeType(0, nlocals, code.co_stacksi ze, code.co_flags,
code.co_code, code.co_consts, code.co_names,
code.co_varname s, code.co_filenam e,
code.co_name,
code.co_firstli neno, code.co_lnotab,
code.co_freevar s,
code.co_cellvar s)
return newCode
>
print convert_functio n(f)
exec convert_functio n(f)
>
Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Out of curiosity, why are you doing this?

Peace,
~Simon

**Duncan Booth** · Jul 27 '06, 02:55 PM

Re: Functions and code objects

Fuzzyman wrote:

>I'd like to construct the code object so that it takes the parameters
>from the enclosing scope (the context I pass into exec I guess),
>without clobbering any local variables that may be defined in the code
>object.
>>
>Anyone got any clues ?

Does this do what you wanted? Instead of messing about with the code object
just work out which values from the namespace the function actually
expects.

>>def callfromnamespa ce(fn, namespace):

names = fn.func_code.co _varnames[:fn.func_code.c o_argcount]
fn(**dict((name , namespace[name])
for name in names if name in namespace))

>>def f(x, y=99):

z = 2
print x, y, z

>>x = 42
>>callfromnames pace(f, globals())

42 99 2

>>y = 37
>>callfromnames pace(f, globals())

42 37 2

>>def testme():

x = 3
callfromnamespa ce(f, vars())
y = 9
callfromnamespa ce(f, vars())

>>testme()

3 99 2
3 9 2

**Fuzzyman** · Jul 27 '06, 02:55 PM

Re: Functions and code objects

Simon Forman wrote:

Fuzzyman wrote:

[snip..]

I was hoping I could get to the code object for the *body* of the
function. Looks like that won't be possible without dis-assembling the
bytecode or other tricks even more hackish than what I've already done.

For the record, the code I was using was :

x = 3
def f(x):
print x

CodeType = type(f.func_cod e)

def convert_functio n(f):
code = f.func_code
nlocals = max(code.co_nlo cals - code.co_argcoun t, 0)
newCode = CodeType(0, nlocals, code.co_stacksi ze, code.co_flags,
code.co_code, code.co_consts, code.co_names,
code.co_varname s, code.co_filenam e,
code.co_name,
code.co_firstli neno, code.co_lnotab,
code.co_freevar s,
code.co_cellvar s)
return newCode

print convert_functio n(f)
exec convert_functio n(f)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

>
Out of curiosity, why are you doing this?
>

In Ruby anonymous code blocks (which take parameters and can presumably
return values) are executed within the scope in which they are used,
rather than the scope that they are defined.

I wanted to see how close to that I could get in Python. Obviously code
could be compiled from a string, but I thought extracting it from the
body of a function was closer.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Peace,
~Simon

**Fuzzyman** · Jul 27 '06, 04:15 PM

Re: Functions and code objects

Duncan Booth wrote:

Fuzzyman wrote:

[snip..]

>
Does this do what you wanted? Instead of messing about with the code object
just work out which values from the namespace the function actually
expects.
>

>def callfromnamespa ce(fn, namespace):

names = fn.func_code.co _varnames[:fn.func_code.c o_argcount]
fn(**dict((name , namespace[name])
for name in names if name in namespace))
>
>

>def f(x, y=99):

z = 2
print x, y, z
>
>

>x = 42
>callfromnamesp ace(f, globals())

42 99 2

>y = 37
>callfromnamesp ace(f, globals())

42 37 2

>def testme():

x = 3
callfromnamespa ce(f, vars())
y = 9
callfromnamespa ce(f, vars())
>
>

>testme()

3 99 2
3 9 2

Hmmm... it may do, thanks. :-)

I'll play around, and this may become a blog entry.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Functions and code objects

Functions and code objects

Comment

Comment

Comment

Comment

Comment

Comment