replacement of rexec?

[Huaiyu Zhu]

What is the prefered way to eval a string like "('a', 1)"? These strings
are representations of simple objects of type int, str, or dict, tuple or
list made of them. I do not want to use naked eval in case I run the
script on a wrong file that happens to contain Python code.

Previously I used the following

from rexec import RExec
r_eval = RExec().r_eval

After upgrading to 2.3 it no longer works:

File "/usr/local/lib/python2.3/rexec.py", line 184, in __init__
raise RuntimeError, "This code is not secure in Python 2.2 and 2.3"
RuntimeError: This code is not secure in Python 2.2 and 2.3

I understand that rexec is not secure against malicious attacks. However,
it is safer than eval for preventing accidents. What should I use in
place? The ideal thing I'm looking for is a function that can parse only
"non-active" python objects and raise exception on anything else.

Huaiyu

[Paul Clinch]

Re: replacement of rexec?

You could use the new compiler module to parse and inspect the result like this:

import compiler as cplr
mod = cplr.parseFile( "filetoload ")
for node in mod.node.nodes:
if 'Function' in str(node):
break

etc..
There are probably more accurate ways of determining that the file is what
you expect.

Otherwise you are back to manually parsing the file as text.

Hope this helps, Paul Clinch

[Huaiyu Zhu]

Re: replacement of rexec?

In article <8cf2994e.03102 31033.6a01871a@ posting.google. com>, Paul Clinch wrote:[color=blue]
> You could use the new compiler module to parse and inspect the result like this:
>
> import compiler as cplr
> mod = cplr.parseFile( "filetoload ")
> for node in mod.node.nodes:
> if 'Function' in str(node):
> break[/color]

Thanks for the suggestion. Is there existing code to assemble the parsed
tree back to the intended Python object? It's probably not too hard to
write one, but I wonder if this is already done?

Huaiyu

[Skip Montanaro]

Re: replacement of rexec?

[color=blue][color=green]
>> You could use the new compiler module to parse and inspect the result
>> like this:
>>
>> import compiler as cplr
>> mod = cplr.parseFile( "filetoload ")
>> for node in mod.node.nodes:
>> if 'Function' in str(node):
>> break[/color][/color]

Huaiyu> Thanks for the suggestion. Is there existing code to assemble
Huaiyu> the parsed tree back to the intended Python object? It's
Huaiyu> probably not too hard to write one, but I wonder if this is
Huaiyu> already done?

I'm not aware of anything. Assuming your inspection process shows the file
is okay, why not just call execfile()?

Skip

[Evan Simpson]

Re: replacement of rexec?

Skip Montanaro wrote:[color=blue][color=green][color=darkred]
> >> You could use the new compiler module to parse and inspect the result[/color][/color]
>
> Huaiyu> Thanks for the suggestion. Is there existing code to assemble
> Huaiyu> the parsed tree back to the intended Python object?
>
> I'm not aware of anything. Assuming your inspection process shows the file
> is okay, why not just call execfile()?[/color]

compiler.pycode gen and new.function or exec can be used to make a code
object and execute it. As Skip says, though, this is pointless unless
you are mutating the AST, as Zope's RestrictedPytho n package does.

Cheers,

Evan @ 4-am

[Jeremy Fincher]

Re: replacement of rexec?

Skip Montanaro <skip@pobox.com > wrote in message news:<mailman.2 42.1067527420.7 02.python-list@python.org >...[color=blue]
> I'm not aware of anything. Assuming your inspection process shows the file
> is okay, why not just call execfile()?[/color]


Probably because that won't get him a useful value :) He'd have to
use eval (but you probably knew that and just typo'ed execfile; I'm
mostly just saying it to try and prevent confusion on his part).

Jeremy

[Ian McMeans]

Re: replacement of rexec?

Are there plans to replace rexec? It seems like a useful library to
have (Zope is a good example of where it would be useful)

Huaiyu Zhu <huaiyu@gauss.a lmaden.ibm.com> wrote in message news:<slrnbpeal 0.fhh.huaiyu@ga uss.almaden.ibm .com>...[color=blue]
> What is the prefered way to eval a string like "('a', 1)"? These strings
> are representations of simple objects of type int, str, or dict, tuple or
> list made of them. I do not want to use naked eval in case I run the
> script on a wrong file that happens to contain Python code.
>
> Previously I used the following
>
> from rexec import RExec
> r_eval = RExec().r_eval
>
> After upgrading to 2.3 it no longer works:
>
> File "/usr/local/lib/python2.3/rexec.py", line 184, in __init__
> raise RuntimeError, "This code is not secure in Python 2.2 and 2.3"
> RuntimeError: This code is not secure in Python 2.2 and 2.3
>
> I understand that rexec is not secure against malicious attacks. However,
> it is safer than eval for preventing accidents. What should I use in
> place? The ideal thing I'm looking for is a function that can parse only
> "non-active" python objects and raise exception on anything else.
>
> Huaiyu[/color]

[Huaiyu Zhu]

Re: replacement of rexec?

In article <698f09f8.03103 01113.356fd637@ posting.google. com>, Jeremy
Fincher wrote:[color=blue]
> Skip Montanaro <skip@pobox.com > wrote in message news:<mailman.2 42.1067527420.7 02.python-list@python.org >...[color=green]
>> I'm not aware of anything. Assuming your inspection process shows the file
>> is okay, why not just call execfile()?[/color]
>
>
> Probably because that won't get him a useful value :) He'd have to
> use eval (but you probably knew that and just typo'ed execfile; I'm
> mostly just saying it to try and prevent confusion on his part).[/color]

That's the point - the file contains multiple strings to be converted to
python objects. I've written the following code over the weekend. It is
more complicated than I'd like. Hopefully someone can replace part or all
of it by some builtin functionality.

#-------------------------------------------------------
import compiler
def safe_eval(s):
"""
Evaluate strings that only contains the following structures:
const, tuple, list, dict
"""
stmts = compiler.parse( s).node.nodes
#print stmts
assert len(stmts) == 1
node = compiler.parse( s).node.nodes[0]
assert node.__class__ == compiler.ast.Di scard
nodes = node.getChildNo des()
assert len(nodes) == 1
return safe_assemble(n odes[0])

seq_types = {
compiler.ast.Tu ple: tuple,
compiler.ast.Li st: list,
}
map_types = {
compiler.ast.Di ct: dict,
}

def safe_assemble(n ode):
""" Recursively assemble parsed ast node """
cls = node.__class__
if cls == compiler.ast.Co nst:
return node.value
elif cls in seq_types:
nodes = node.nodes
args = map(safe_assemb le, nodes)
return seq_types[cls](args)
elif cls in map_types:
keys, values = zip(*node.items )
keys = map(safe_assemb le, keys)
values = map(safe_assemb le, values)
return map_types[cls](zip(keys, values))
else:
raise "Type not allowed", cls


if __name__ == '__main__':
tests = [
"3j",
"1, 2.5",
"['abc', 0xF7]",
"1, ['abc', [2,3]], {(4,5):[6.5, 8]}",
]
for s in tests:
print safe_eval(s)
#-------------------------------------------------------


Huaiyu

[John Roth]

Re: replacement of rexec?

This looks rather interesting. I frankly think we need it
as a built-in.

John Roth

"Huaiyu Zhu" <huaiyu@gauss.a lmaden.ibm.com> wrote in message
news:slrnbqgpsd .5m4.huaiyu@gau ss.almaden.ibm. com...[color=blue]
> In article <698f09f8.03103 01113.356fd637@ posting.google. com>, Jeremy
> Fincher wrote:[color=green]
> > Skip Montanaro <skip@pobox.com > wrote in message[/color][/color]
news:<mailman.2 42.1067527420.7 02.python-list@python.org >...[color=blue][color=green][color=darkred]
> >> I'm not aware of anything. Assuming your inspection process shows the[/color][/color][/color]
file[color=blue][color=green][color=darkred]
> >> is okay, why not just call execfile()?[/color]
> >
> >
> > Probably because that won't get him a useful value :) He'd have to
> > use eval (but you probably knew that and just typo'ed execfile; I'm
> > mostly just saying it to try and prevent confusion on his part).[/color]
>
> That's the point - the file contains multiple strings to be converted to
> python objects. I've written the following code over the weekend. It is
> more complicated than I'd like. Hopefully someone can replace part or all
> of it by some builtin functionality.
>
> #-------------------------------------------------------
> import compiler
> def safe_eval(s):
> """
> Evaluate strings that only contains the following structures:
> const, tuple, list, dict
> """
> stmts = compiler.parse( s).node.nodes
> #print stmts
> assert len(stmts) == 1
> node = compiler.parse( s).node.nodes[0]
> assert node.__class__ == compiler.ast.Di scard
> nodes = node.getChildNo des()
> assert len(nodes) == 1
> return safe_assemble(n odes[0])
>
> seq_types = {
> compiler.ast.Tu ple: tuple,
> compiler.ast.Li st: list,
> }
> map_types = {
> compiler.ast.Di ct: dict,
> }
>
> def safe_assemble(n ode):
> """ Recursively assemble parsed ast node """
> cls = node.__class__
> if cls == compiler.ast.Co nst:
> return node.value
> elif cls in seq_types:
> nodes = node.nodes
> args = map(safe_assemb le, nodes)
> return seq_types[cls](args)
> elif cls in map_types:
> keys, values = zip(*node.items )
> keys = map(safe_assemb le, keys)
> values = map(safe_assemb le, values)
> return map_types[cls](zip(keys, values))
> else:
> raise "Type not allowed", cls
>
>
> if __name__ == '__main__':
> tests = [
> "3j",
> "1, 2.5",
> "['abc', 0xF7]",
> "1, ['abc', [2,3]], {(4,5):[6.5, 8]}",
> ]
> for s in tests:
> print safe_eval(s)
> #-------------------------------------------------------
>
>
> Huaiyu[/color]

[John J. Lee]

Re: replacement of rexec?

"John Roth" <newsgroups@jhr othjr.com> writes:[color=blue]
> [Haiyu Zhu wrote:][color=green]
> > def safe_eval(s):
> > """
> > Evaluate strings that only contains the following structures:
> > const, tuple, list, dict
> > """[/color]
>
> This looks rather interesting. I frankly think we need it
> as a built-in.[/color]
[...]

Not a builtin, necessarily, but in the standard library would be nice.

I've written something like this before, only more ad-hoc, and wished
there was something like this available.


John