Loading a Python collection from an text-file

[Ilias Lazaridis]

within a python script, I like to create a collection which I fill with
values from an external text-file (user editable).

How is this accomplished the easiest way (if possible without the need
of libraries which are not part of the standard distribution)?

something like:

text-file:
{peter, 16},
{anton, 21}

-

within code:

users.load(text-file.txt)

for user in users
user.name
user.age

..

--

http://lazaridis.com

[James Stroud]

Re: Loading a Python collection from an text-file

Ilias Lazaridis wrote:[color=blue]
> within a python script, I like to create a collection which I fill with
> values from an external text-file (user editable).
>
> How is this accomplished the easiest way (if possible without the need
> of libraries which are not part of the standard distribution)?
>
> something like:
>
> text-file:
> {peter, 16},
> {anton, 21}
>
> -
>
> within code:
>
> users.load(text-file.txt)
>
> for user in users
> user.name
> user.age
>
> .
>[/color]

This is specific for the text above. You will have to re-craft a regex
if the actual file is different.

import re

def get_names(afile ):
regex = re.compile(r'{([^,]*),\s*([^}]*)}')
names = []
for aline in afile:
m = regex.search(al ine)
names.append(m. groups())
return names

def test():
import cStringIO
afile = cStringIO.Strin gIO("{peter, 16},\n{anton, 21}\n")
print get_names(afile )

test()

[Ken Starks]

Re: Loading a Python collection from an text-file

Ilias Lazaridis wrote:
[color=blue]
> within a python script, I like to create a collection which I fill with
> values from an external text-file (user editable).
>
> How is this accomplished the easiest way (if possible without the need
> of libraries which are not part of the standard distribution)?
>
> something like:
>
> text-file:
> {peter, 16},
> {anton, 21}
>
> -
>
> within code:
>
> users.load(text-file.txt)
>
> for user in users
> user.name
> user.age
>
> .
>[/color]
"""
What I do for this kind of work is to use a gnumeric spreadsheet
which saves the data in a simple xml format. xml is much less
error-prone than plain text.
Google for, and study 'The gnumeric file format' by David Gilbert.
You need to know how to unzip the file, and how to write a SAX parser.


If you want to use a plain text format, keep it simple. I would
separate the two fields with tab (thus permit a comma within a field)
and allow 'comment' lines that start with a hash.
You don't need the braces, or the end-of-line comma you included.

# snip 'text-file.txt'
# name and age on one line separated by tab
Jonny 8
Mary 87
Moses 449


# end-snip 'text-file.txt'
Then:
"""

import string

class user:
def __init__(self,n ame,age):
self.name=name
self.age=int(ag e) # or a float, or a time-interval, or date-of-birth

def show(self):
print "%s is aged %s" % (self.name, self.age)

if __name__=="__ma in__":
users=[]
filename="text-file.txt"
fieldsep="\t"
F=open(filename ,"r")
Lines=F.readlin es()
for L0 in Lines:
L1=string.strip (L0)
if not L1.startswith(" #"):
Record=string.s plit(L1,fieldse p)
# insert error handling/validation here
users.append(us er(Record[0],Record[1]))

F.close()
for user in users:
user.show()

[jschull@gmail.com]

Re: Loading a Python collection from an text-file

another approach (probably frowned upon, but it has worked for me) is
to use python syntax (a dictionary, say, or a list) and just import (or
reload) the file

[Larry Bates]

Re: Loading a Python collection from an text-file

Take a look at ConfigParser module. The format of the file would be
something like:

[members]
peter=16
anton=21

People are accustomed to this format file (windows .ini format).

-Larry


Ilias Lazaridis wrote:[color=blue]
> within a python script, I like to create a collection which I fill with
> values from an external text-file (user editable).
>
> How is this accomplished the easiest way (if possible without the need
> of libraries which are not part of the standard distribution)?
>
> something like:
>
> text-file:
> {peter, 16},
> {anton, 21}
>
> -
>
> within code:
>
> users.load(text-file.txt)
>
> for user in users
> user.name
> user.age
>
> .
>[/color]

[Ilias Lazaridis]

Re: Loading a Python collection from an text-file

jschull@gmail.c om wrote:[color=blue]
> another approach (probably frowned upon, but it has worked for me) is
> to use python syntax (a dictionary, say, or a list) and just import (or
> reload) the file
>[/color]

this sounds good.

can I import a whole collection of instances this way?

-

(thanks for all the other answers within this thread).

..

--

http://lazaridis.com

[Ido Yehieli]

Re: Loading a Python collection from an text-file

perhapse consider using the pickle module?

http://docs.python.org/lib/module-pickle.html

[Bengt Richter]

Re: Loading a Python collection from an text-file

On Mon, 23 Jan 2006 21:00:55 +0200, Ilias Lazaridis <ilias@lazaridi s.com> wrote:
[color=blue]
>within a python script, I like to create a collection which I fill with
>values from an external text-file (user editable).
>
>How is this accomplished the easiest way (if possible without the need
>of libraries which are not part of the standard distribution)?
>
>something like:
>
>text-file:
>{peter, 16},
>{anton, 21}
>
>-
>
>within code:
>
>users.load(tex t-file.txt)
>
>for user in users
> user.name
> user.age
>
>.
>
>--
>http://lazaridis.com[/color]

I'd use a CSV text file, maybe something like (only tested as far as you see!):

----< for_ilias_lazar idis.py >----------------------------------------------
import csv, types

class Fields(object):
def __init__(self, kvpairs): self.__dict__.u pdate(kvpairs)

class Users(object):
def __init__(self):
self.userlist=[]
def load(self, lineiter):
if isinstance(line iter, basestring):
lineiter = open(lineiter) # assume it's a file path
csvit = csv.reader(line iter)
self.colnames = colnames = csvit.next()
typenames = csvit.next()
self.coltypes =coltypes = [getattr(types, name.capitalize ()+'Type')
for name in typenames]
for row in csvit:
self.userlist.a ppend(Fields(zi p(colnames, (t(s) for t,s in zip(coltypes, row)))))
def __iter__(self): return iter(self.userl ist)

def test():
import StringIO
f = StringIO.String IO("""\
name,age
String,Int
peter,16
anton,21
""")
users = Users()
users.load(f)
for user in users:
print user.name, user.age
for user in users:
for name in users.colnames:
print '%s=%s,'%(name, getattr(user, name)),
print

if __name__ == '__main__': test()
-----------------------------------------------------------------------

Output:

[ 4:47] C:\pywk\clp>py2 4 for_ilias_lazar idis.py
peter 16
anton 21
name=peter, age=16,
name=anton, age=21,

(the first for user in users loop presumes knowledge of the field names name and age.
The second gets them automatically from the names loaded in the load method from
the first line of the text file. The second line expects type names as you see
in the types module, except without the "Type" suffix.

Perhaps you can adapt for your purposes.

Regards,
Bengt Richter

[Fuzzyman]

Re: Loading a Python collection from an text-file

Seeing as we're suggesting alternatives, ConfigObj is great for hand
readable/writable data persistence.

You can use validate and ConfigPersist for automatic type conversion.

You can persist (basically) all the standard datatypes using this. The
syntax is usually more 'familiar' than Yaml, but it's not as flexible.

http://www.voidspace.org.uk/python/configobj.html

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Magnus Lycka]

Re: Loading a Python collection from an text-file

Ilias Lazaridis wrote:[color=blue]
> jschull@gmail.c om wrote:
>[color=green]
>> another approach (probably frowned upon, but it has worked for me) is
>> to use python syntax (a dictionary, say, or a list) and just import (or
>> reload) the file
>>[/color]
>
> this sounds good.
>
> can I import a whole collection of instances this way?[/color]

Sure, it's just a Python module with variables in it.

I wouldn't try to teach my users Python syntax though.

If you really need this kind of data structure freedom,
I'd lean towards YAML or possibly XML. (XML isn't too
bad if you provide good tools. It's not a good idea
with just a text editor.)

If a spreadsheet like layout is enough, I (still)
recommend csv.

[Magnus Lycka]

Re: Loading a Python collection from an text-file

Ilias Lazaridis wrote:[color=blue]
> within a python script, I like to create a collection which I fill with
> values from an external text-file (user editable).[/color]
If a spreadsheet like layout fits, use the csv module and
a plain comma separated file.

Then the end user can also use e.g. Excel to edit the data.

[Magnus Lycka]

Re: Loading a Python collection from an text-file

Ido Yehieli wrote:[color=blue]
> perhapse consider using the pickle module?
> http://docs.python.org/lib/module-pickle.html[/color]

User editable? We should be kind to our users!
[color=blue][color=green][color=darkred]
>>> d = {'peter':14, 'paul':23}
>>> pickle.dumps(d)[/color][/color][/color]
"(dp0\nS'paul'\ np1\nI23\nsS'pe ter'\np2\nI14\n s."

[Ido Yehieli]

Re: Loading a Python collection from an text-file

>>Sure, it's just a Python module with variables in it.[color=blue][color=green]
>>
>>I wouldn't try to teach my users Python syntax though.[/color][/color]

not to mention the security risks

[Fredrik Lundh]

Re: Loading a Python collection from an text-file

Ido Yehieli wrote:
[color=blue][color=green][color=darkred]
> >>Sure, it's just a Python module with variables in it.
> >>
> >>I wouldn't try to teach my users Python syntax though.[/color][/color]
>
> not to mention the security risks[/color]

you mean all the things they can do from inside python that they
cannot do from the command line ?

</F>

[Ilias Lazaridis]

Re: Loading a Python collection from an text-file

Ken Starks wrote:[color=blue]
> Ilias Lazaridis wrote:
>[color=green]
>>within a python script, I like to create a collection which I fill with
>>values from an external text-file (user editable).
>>
>>How is this accomplished the easiest way (if possible without the need
>>of libraries which are not part of the standard distribution)?
>>
>>something like:
>>
>>text-file:
>>{peter, 16},
>>{anton, 21}
>>
>>-
>>
>>within code:
>>
>>users.load(te xt-file.txt)
>>
>>for user in users
>> user.name
>> user.age[/color][/color]
[...]

the solutions below seems to be the most compact one.

this, or the suggested CSV module within the other messages.

thank's to everyone for the feedback.

[...][color=blue]
> If you want to use a plain text format, keep it simple. I would
> separate the two fields with tab (thus permit a comma within a field)
> and allow 'comment' lines that start with a hash.
> You don't need the braces, or the end-of-line comma you included.
>
> # snip 'text-file.txt'
> # name and age on one line separated by tab
> Jonny 8
> Mary 87
> Moses 449
>
>
> # end-snip 'text-file.txt'
> Then:
> """
>
> import string
>
> class user:
> def __init__(self,n ame,age):
> self.name=name
> self.age=int(ag e) # or a float, or a time-interval, or date-of-birth
>
> def show(self):
> print "%s is aged %s" % (self.name, self.age)
>
> if __name__=="__ma in__":
> users=[]
> filename="text-file.txt"
> fieldsep="\t"
> F=open(filename ,"r")
> Lines=F.readlin es()
> for L0 in Lines:
> L1=string.strip (L0)
> if not L1.startswith(" #"):
> Record=string.s plit(L1,fieldse p)
> # insert error handling/validation here
> users.append(us er(Record[0],Record[1]))
>
> F.close()
> for user in users:
> user.show()[/color]

..

--

http://lazaridis.com