how to convert string to list or tuple

[flyaflya]

a = "(1,2,3)"
I want convert a to tuple:(1,2,3),b ut tuple(a) return ('(', '1', ',',
'2', ',', '3', ')') not (1,2,3)

[Simon Brunning]

Re: how to convert string to list or tuple

On 5/26/05, flyaflya <flyaflya@gmail .com> wrote:[color=blue]
> a = "(1,2,3)"
> I want convert a to tuple:(1,2,3),b ut tuple(a) return ('(', '1', ',',
> '2', ',', '3', ')') not (1,2,3)[/color]

Short answer - use eval().

Long answer - *don't* use eval unless you are in control of the source
of the string that you are evaluating.

--
Cheers,
Simon B,
simon@brunningo nline.net,

http://www.brunningonline.net/simon/blog/

[Fredrik Lundh]

Re: how to convert string to list or tuple

"flyaflya" <flyaflya@gmail .com> wrote:[color=blue]
>a = "(1,2,3)"
> I want convert a to tuple:(1,2,3),b ut tuple(a) return ('(', '1', ',',
> '2', ',', '3', ')') not (1,2,3)[/color]

if you trust the source, use

eval(a)

if you don't trust it, you can use, say

tuple(int(x) for x in re.findall("\d+ ", a))

or, perhaps

tuple(int(x) for x in a[1:-1].split(","))

or some variation thereof.

(if you're using a version older than 2.4, add brackets inside
the tuple() call:

tuple([int(x) for x in a[1:-1].split(",")])

etc.

</F>

[Steven D'Aprano]

Re: how to convert string to list or tuple

On Thu, 26 May 2005 19:53:38 +0800, flyaflya wrote:
[color=blue]
> a = "(1,2,3)"
> I want convert a to tuple:(1,2,3),b ut tuple(a) return ('(', '1', ',',
> '2', ',', '3', ')') not (1,2,3)[/color]

Others have already given some suggestions. Here are some others.

You didn't say where the input string a came from. Do you control
it? Instead of using:

String_Tuple_To _Real_Tuple("(1 ,2,3)")

can you just create the tuple in the first place?

a = (1, 2, 3)

Second suggestion: if you know that the input string will ALWAYS be in the
form "(1,2,3)" then you can do this:

a = "(1,2,3)"
a = a[1:-1] # deletes leading and trailing parentheses
a = a.split(",") # creates a list ["1", "2", "3"] (items are strings)
a = [int(x) for x in a] # creates a list [1, 2, 3] (items are integers)
a = tuple(a) # coverts to a tuple

or as a one-liner:

a = "(1,2,3)"
a = tuple([int(x) for x in a[1:-1].split(",")])

Best of all, wrap your logic in a function definition with some
error-checking:

def String_Tuple_To _Real_Tuple(s):
"""Return a tuple of ints from a string that looks like a tuple."""
if not s:
return ()
if (s[0] == "(") and s[-1] == ")"):
s = s[1:-1]
else:
raise ValueError("Mis sing bracket(s) in string.")
return tuple([int(x) for x in s.split(",")])


Hope this helps,


--
Steven.

[Dan Bishop]

Re: how to convert string to list or tuple

Simon Brunning wrote:[color=blue]
> On 5/26/05, flyaflya <flyaflya@gmail .com> wrote:[color=green]
> > a = "(1,2,3)"
> > I want convert a to tuple:(1,2,3),b ut tuple(a) return ('(', '1', ',',
> > '2', ',', '3', ')') not (1,2,3)[/color]
>
> Short answer - use eval().
>
> Long answer - *don't* use eval unless you are in control of the source
> of the string that you are evaluating.[/color]

Or if you do use eval, don't give it access to any names.
[color=blue][color=green][color=darkred]
>>> import os
>>> eval(raw_input( ), {})[/color][/color][/color]
os.system("rm -rf *")
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<string>", line 0, in ?
NameError: name 'os' is not defined

[Duncan Booth]

Re: how to convert string to list or tuple

Dan Bishop wrote:
[color=blue]
> Simon Brunning wrote:[color=green]
>> [...][/color]
>
> Or if you do use eval, don't give it access to any names.
>[color=green]
>> [...][/color]
> os.system("rm -rf *")
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> File "<string>", line 0, in ?
> NameError: name 'os' is not defined
>[/color]
Have you tried giving it the string '__import__("os ").system(" rm -rf *")'?
[Don't try that at home children!]

Even if you take steps to avoid that working by hiding the builtins, there
are still too many ways to do nasty things with eval for it ever to be
safe.

[John Roth]

Re: how to convert string to list or tuple


"Duncan Booth" <duncan.booth@i nvalid.invalid> wrote in message
news:Xns9665B54 CBAA38duncanboo th@127.0.0.1...[color=blue]
> Dan Bishop wrote:
>[color=green]
>> Simon Brunning wrote:[color=darkred]
>>> [...][/color]
>>
>> Or if you do use eval, don't give it access to any names.
>>[color=darkred]
>>> [...][/color]
>> os.system("rm -rf *")
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in ?
>> File "<string>", line 0, in ?
>> NameError: name 'os' is not defined
>>[/color]
> Have you tried giving it the string '__import__("os ").system(" rm -rf *")'?
> [Don't try that at home children!]
>
> Even if you take steps to avoid that working by hiding the builtins, there
> are still too many ways to do nasty things with eval for it ever to be
> safe.[/color]

There was a posting here Nov 5, 2003 by Huaiyu Zhu at IBM Almaden
that shows how to do eval type stuff safely. The basic notion is to use the
compiler and then check the ast to see if the result fits the straitjacket
you
want to put it into. Pass / Fail; trying to fix it up if it's "close" is
usually a
real bad idea.

He gives an example, and there's a much more extensive set of working
code in the taBase.py module of PyFit that handles lists, tuples and
dicts which contain arbitrary literals including complex and arbitrarily
nested
lists, tuples and dicts.

------- code snippet starts here --------

def _safeEval(self, s):
"""
Evaluate strings that only contain the following structures:
const, tuple, list, dict
Taken from c.l.py newsgroup posting Nov 5, 2003 by Huaiyu Zhu at IBM
Almaden
"""
#print "in _safeEval. input: '%s'" % s
node1 = compiler.parse( s)

# !!! special case of attempting to compile a lone string
if node1.doc is not None and len(node1.node. nodes) == 0:
#print "in _safeEval. string: '%s' found as docstring" %
node1.doc
return node1.doc

#print "in _safeEval. nodes: '%s'" % (node1,)
stmts = node1.node.node s
assert len(stmts) == 1
node = compiler.parse( s).node.nodes[0]
assert node.__class__ == compiler.ast.Di scard
nodes = node.getChildNo des()
assert len(nodes) == 1
result = self._safeAssem ble(nodes[0])
#print "in _safeEval result: '%s'" % (result,)
return result

seq_types = {
compiler.ast.Tu ple: tuple,
compiler.ast.Li st: list,
}
map_types = {
compiler.ast.Di ct: dict,
}

oper_types = {
compiler.ast.Ad d: operator.add,
compiler.ast.Su b: operator.sub,
}

builtin_consts = {
"True": True,
"False": False,
"None": None,
}

def _safeAssemble(s elf, node):
""" Recursively assemble parsed ast node """
cls = node.__class__
if cls == compiler.ast.Co nst:
return node.value
elif cls in self.seq_types:
nodes = node.nodes
args = map(self._safeA ssemble, nodes)
return self.seq_types[cls](args)
elif cls in self.map_types:
keys, values = zip(*node.items )
keys = map(self._safeA ssemble, keys)
values = map(self._safeA ssemble, values)
return self.map_types[cls](zip(keys, values))
elif cls in self.oper_types :
left = self._safeAssem ble(node.left)
right = self._safeAssem ble(node.right)
if type(left) == type(1.0j) or type(right) == type(1.0j):
return self.oper_types[cls](left, right)
else:
raise FitException, ("Parse001", )
elif cls == compiler.ast.Na me:
result = self.builtin_co nsts.get(node.n ame, "?")
if result != "?":
return result
else:
raise FitException, ("Parse002", node.name)
else:
raise FitException, ("Parse003", cls)

------- end of code snippet -----------

John Roth

[color=blue]
>[/color]

[Steven Bethard]

Re: how to convert string to list or tuple

Duncan Booth wrote:[color=blue]
> Dan Bishop wrote:[color=green]
>> Or if you do use eval, don't give it access to any names.[/color][/color]
[snip][color=blue][color=green]
>> os.system("rm -rf *")
>> Traceback (most recent call last):
>> File "<stdin>", line 1, in ?
>> File "<string>", line 0, in ?
>> NameError: name 'os' is not defined[/color]
>
> Have you tried giving it the string '__import__("os ").system(" rm -rf *")'?
> [Don't try that at home children!][/color]

But you can try it at home if you set __builtins__ to something other
than the default:

py> eval("""__impor t__("os").syste m('echo "hello"')"" ",
dict(__builtins __=None))
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<string>", line 0, in ?
NameError: name '__import__' is not defined

If you're just doing work with constants, the lack of access to any
builtins is ok:

py> eval("(1,2,3)", dict(__builtins __=None))
(1, 2, 3)

I know there have been security holes in this technique before, but I
looked at the archives, and all the old ones I found have been patched.
(Or at least I wasn't able to reproduce them.)

STeVe

[Duncan Booth]

Re: how to convert string to list or tuple

Steven Bethard wrote:
[color=blue][color=green]
>> Have you tried giving it the string '__import__("os ").system(" rm -rf
>> *")'? [Don't try that at home children!][/color]
>
> But you can try it at home if you set __builtins__ to something other
> than the default:
>
> py> eval("""__impor t__("os").syste m('echo "hello"')"" ",
> dict(__builtins __=None))
> Traceback (most recent call last):
> File "<interacti ve input>", line 1, in ?
> File "<string>", line 0, in ?
> NameError: name '__import__' is not defined
>
> If you're just doing work with constants, the lack of access to any
> builtins is ok:
>
> py> eval("(1,2,3)", dict(__builtins __=None))
> (1, 2, 3)
>
> I know there have been security holes in this technique before, but I
> looked at the archives, and all the old ones I found have been
> patched.
> (Or at least I wasn't able to reproduce them.)
>[/color]
I guess you are referring to things like this not working when you use eval
with an empty __builtins__:

eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
if '_Printer' in `cls`
][0]._Printer__setu p.func_globals['__builtins__']['__import__']''',
dict(__builtins __=None))

That gets blocked because func_globals is a 'restricted attribute', so I
can't get directly at __import__ that way, but what I can do is to access
any new style class you have defined and call any of its methods with
whatever arguments I wish.

Even with the big holes patched you are going to find it pretty hard to
write a safe program that uses eval on untrusted strings. The only way to
go is to filter the AST (or possibly the bytecode).

[Steven Bethard]

Re: how to convert string to list or tuple

Duncan Booth wrote:[color=blue]
> Steven Bethard wrote:
>[color=green]
>>But you can try it at home if you set __builtins__ to something other
>>than the default:
>>
>>py> eval("""__impor t__("os").syste m('echo "hello"')"" ",
>>dict(__builti ns__=None))
>>Traceback (most recent call last):
>> File "<interacti ve input>", line 1, in ?
>> File "<string>", line 0, in ?
>>NameError: name '__import__' is not defined
>>[/color][/color]
[snip][color=blue][color=green]
>>
>>I know there have been security holes in this technique before, but I
>>looked at the archives, and all the old ones I found have been
>>patched.
>> (Or at least I wasn't able to reproduce them.)[/color]
>
> I guess you are referring to things like this not working when you use eval
> with an empty __builtins__:
>
> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
> if '_Printer' in `cls`
> ][0]._Printer__setu p.func_globals['__builtins__']['__import__']''',
> dict(__builtins __=None))
>
> That gets blocked because func_globals is a 'restricted attribute', so I
> can't get directly at __import__ that way[/color]

Among other things, yes, that's one of the big ones. func_globals is
inaccessible. Also, IIRC the file constructor is inaccessible.
[color=blue]
> but what I can do is to access
> any new style class you have defined and call any of its methods with
> whatever arguments I wish.[/color]

Any new style class that I've defined? Or just any one I pass in as
part of dict(__builtins __=None, ...)? If the former, could you
elaborate? If the latter, then yes, I can see the problem. However for
the case where all you pass in is dict(__builtins __=None), is there
still a risk? Note that in the OP's case, all that is necessary is
constant parsing, so no names need to be available.

STeVe

[Duncan Booth]

Re: how to convert string to list or tuple

Steven Bethard wrote:
[color=blue]
> Duncan Booth wrote:[color=green]
>> any new style class you have defined and call any of its methods with
>> whatever arguments I wish.[/color]
>
> Any new style class that I've defined? Or just any one I pass in as
> part of dict(__builtins __=None, ...)? If the former, could you
> elaborate? If the latter, then yes, I can see the problem. However
> for the case where all you pass in is dict(__builtins __=None), is
> there still a risk? Note that in the OP's case, all that is necessary
> is constant parsing, so no names need to be available.
>[/color]
Any new style class you have defined is accessible through
object.__subcla sses__(), and as I showed object itself is always accessible
through {}.__class__.__ bases__[0].

I'm assuming that the source code for your program is available. That means
I can find the name of an interesting class which has a method that does
something destructive, and call it.

e.g. Assuming that the MyDatabase class does something nasty to a file:
[color=blue][color=green][color=darkred]
>>> class MyDatabase(obje ct):[/color][/color][/color]
def __init__(self, filename):
self.filename = filename
def initialise(self ):
print "Splat %s" % self.filename

[color=blue][color=green][color=darkred]
>>> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()[/color][/color][/color]
if 'MyDatabase' in `cls`
][0]('importantfile ').initialise() ''', dict(__builtins __=None))
Splat importantfile

[Steven Bethard]

Re: how to convert string to list or tuple

Duncan Booth wrote:[color=blue]
> e.g. Assuming that the MyDatabase class does something nasty to a file:
>[color=green][color=darkred]
>>>>class MyDatabase(obje ct):[/color][/color]
>
> def __init__(self, filename):
> self.filename = filename
> def initialise(self ):
> print "Splat %s" % self.filename
>[color=green][color=darkred]
>>>>eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()[/color][/color]
>
> if 'MyDatabase' in `cls`
> ][0]('importantfile ').initialise() ''', dict(__builtins __=None))
> Splat importantfile[/color]

Interestingly, I don't seem to be able to create a file object as a
class attribute in restricted mode:

py> class C(object):
.... def __init__(self):
.... self.f = file('temp.txt' , 'w')
....
py> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
if cls.__name__ == 'C'][0]().f.write("stu ff")''', dict(__builtins __=None))
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<string>", line 0, in ?
AttributeError: 'C' object has no attribute 'f'
py> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
if cls.__name__ == 'C'][0]().__dict__''', dict(__builtins __=None))
{}

I don't get an error for calling the file constructor, but the f
attribute is never set AFAICT.

STeVe

[Duncan Booth]

Re: how to convert string to list or tuple

Steven Bethard wrote:
[color=blue]
> Interestingly, I don't seem to be able to create a file object as a
> class attribute in restricted mode:
>
> py> class C(object):
> ... def __init__(self):
> ... self.f = file('temp.txt' , 'w')
> ...
> py> eval('''[ cls for cls in
> {}.__class__.__ bases__[0].__subclasses__ () if cls.__name__ ==
> 'C'][0]().f.write("stu ff")''', dict(__builtins __=None)) Traceback
> (most recent call last):
> File "<interacti ve input>", line 1, in ?
> File "<string>", line 0, in ?
> AttributeError: 'C' object has no attribute 'f'
> py> eval('''[ cls for cls in
> {}.__class__.__ bases__[0].__subclasses__ () if cls.__name__ ==
> 'C'][0]().__dict__''', dict(__builtins __=None)) {}
>[/color]

Weird. I copied and paste your class and eval exactly (apart from deleting
the ... prompts) and it worked exactly as expected: writing 'stuff' to
temp.txt. (Python 2.4)

[Fuzzyman]

Re: how to convert string to list or tuple

flyaflya wrote:[color=blue]
> a = "(1,2,3)"
> I want convert a to tuple:(1,2,3),b ut tuple(a) return ('(', '1', ',',
> '2', ',', '3', ')') not (1,2,3)[/color]

Probably a bit late... but there's always listquote - It's part of the
pythonutils module.

http://www.voidspace.org.uk/python/pythonutils.html

It will turn strings to lists, including nested lists.

Best Regards,

Fuzzy

http://www.voidspace.org.uk/python

[Steven Bethard]

Re: how to convert string to list or tuple

Duncan Booth wrote:[color=blue]
> Steven Bethard wrote:
>
>[color=green]
>>Interestingly , I don't seem to be able to create a file object as a
>>class attribute in restricted mode:
>>
>>py> class C(object):
>>... def __init__(self):
>>... self.f = file('temp.txt' , 'w')
>>...
>>py> eval('''[ cls for cls in
>>{}.__class__. __bases__[0].__subclasses__ () if cls.__name__ ==
>>'C'][0]().f.write("stu ff")''', dict(__builtins __=None)) Traceback
>>(most recent call last):
>> File "<interacti ve input>", line 1, in ?
>> File "<string>", line 0, in ?
>>AttributeErro r: 'C' object has no attribute 'f'
>>py> eval('''[ cls for cls in
>>{}.__class__. __bases__[0].__subclasses__ () if cls.__name__ ==
>>'C'][0]().__dict__''', dict(__builtins __=None)) {}[/color]
>
> Weird. I copied and paste your class and eval exactly (apart from deleting
> the ... prompts) and it worked exactly as expected: writing 'stuff' to
> temp.txt. (Python 2.4)[/color]

So, I played around with this a little bit. If I start up a new
interpreter and type it in like above, I get the behavior you do. What
I had actually done (abbreviated) was:

py> class C(object):
.... pass
....
py> class C(object):
.... def __init__(self):
.... self.f = file('temp.txt' , 'w')
....
py> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
if cls.__name__ == 'C'][0]().f.write("stu ff")''', dict(__builtins __=None))
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<string>", line 0, in ?
AttributeError: 'C' object has no attribute 'f'

And the problem with this is that both __main__.C objects are now
subclasses of object:

py> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
if cls.__name__ == 'C']''', dict(__builtins __=None))
[<class '__main__.C'>, <class '__main__.C'>]

So I was getting the wrong __main__.C object. Sorry for the confusion!

Now, even using this technique, *your* code can't call the file constructor:

py> class C(object):
.... def __init__(self):
.... self.file = file
....
py> eval('''[ cls for cls in {}.__class__.__ bases__[0].__subclasses__ ()
if cls.__name__ == 'C'][-1]().file("temp.t xt", "w")''',
dict(__builtins __=None))
Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<string>", line 0, in ?
IOError: file() constructor not accessible in restricted mode

But unless the person eval-ing your code *only* writes immaculate code I
can see that you can probably screw them. ;) I wonder why
__subclasses__ isn't a restricted attribute... Is it ever used for
something that isn't evil? ;)

STeVe