lightweight human-readable config?

[Maxim Khesin]

I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)

can you guys suggest some format for this? thanks,
max

[John Roth]

Re: lightweight human-readable config?


"Maxim Khesin" <max@cNvOiSsiPo AnMtech.com> wrote in message
news:p4Xnb.1736 7$Gq.5475490@tw ister.nyc.rr.co m...[color=blue]
> I want to have a config file with my python proggie, satisfying the
> following requirements:
> 1) support key->(value, default)
> 2) simple and intuitive to read and edit
> 3) easyly readable into a python datastructure (a dictionary?)
> 4) not requiring any heavy libraries needed (I am distributing my
> proggie as a py2exe executable and do not want to bloat the size)
>
> can you guys suggest some format for this? thanks,
> max[/color]

A text file containing lines formatted as:

key: value

for line in inFile.readline s():
key, value = line.split(":")
configdict[key] = value

I usually like to add a statement that filters out lines
that start with a "#" character.

Elaborate to taste.

John Roth

[color=blue]
>[/color]

[Noen]

Re: lightweight human-readable config?

John Roth wrote:

[color=blue]
> "Maxim Khesin" <max@cNvOiSsiPo AnMtech.com> wrote in message
> news:p4Xnb.1736 7$Gq.5475490@tw ister.nyc.rr.co m...
>[color=green]
>>I want to have a config file with my python proggie, satisfying the
>>following requirements:
>>1) support key->(value, default)
>>2) simple and intuitive to read and edit
>>3) easyly readable into a python datastructure (a dictionary?)
>>4) not requiring any heavy libraries needed (I am distributing my
>>proggie as a py2exe executable and do not want to bloat the size)
>>
>>can you guys suggest some format for this? thanks,
>>max[/color]
>
>
> A text file containing lines formatted as:
>
> key: value
>
> for line in inFile.readline s():
> key, value = line.split(":")
> configdict[key] = value
>
> I usually like to add a statement that filters out lines
> that start with a "#" character.
>
> Elaborate to taste.
>
> John Roth[/color]
You could try out
This format is already avaiable as a python module following the .ini
format (RFC 822).
The format is like this
[SECTION]
varname1: var
varname2: var
varname3: var
varname4: var
....
Compiled using the -OO, its size is.
30.10.2003 00:33 13 483 ConfigParser.py o

Enjoy -- Noen[color=blue]
>
>
>
>
>[/color]

[Heather Coppersmith]

Re: lightweight human-readable config?

On Wed, 29 Oct 2003 22:22:13 GMT,
Maxim Khesin <max@cNvOiSsiPo AnMtech.com> wrote:
[color=blue]
> I want to have a config file with my python proggie, satisfying the
> following requirements:
> 1) support key->(value, default)
> 2) simple and intuitive to read and edit
> 3) easyly readable into a python datastructure (a dictionary?)
> 4) not requiring any heavy libraries needed (I am distributing my
> proggie as a py2exe executable and do not want to bloat the size)[/color]
[color=blue]
> can you guys suggest some format for this? thanks,[/color]

Will the standard library ConfigParser work? Point your web
browser here:

http://www.python.org/doc/current/lib/module-ConfigParser.html

Regards,
Heather

--
Heather Coppersmith
That's not right; that's not even wrong. -- Wolfgang Pauli

[Ville Vainio]

Re: lightweight human-readable config?

Maxim Khesin <max@cNvOiSsiPo AnMtech.com> writes:
[color=blue]
> I want to have a config file with my python proggie, satisfying the
> following requirements:[/color]

I usually use plain old python modules for configuration. If you don't
want to use 'import' and a name ending with .py, just exec the module
in some namaspace. You can do "default value" behaviour by execing
some default value module in the same namespace (often you might want
to parse /etc/fooconfig before ~/.fooconfig). Then you just harvest
the namespace for the config data.

--
Ville Vainio http://www.students.tut.fi/~vainio24

[Anthony Briggs]

Re: lightweight human-readable config?

At 10:22 PM +0000 29/10/03, Maxim Khesin wrote:[color=blue]
>I want to have a config file with my python proggie, satisfying the
>following requirements:
>1) support key->(value, default)
>2) simple and intuitive to read and edit
>3) easyly readable into a python datastructure (a dictionary?)
>4) not requiring any heavy libraries needed (I am distributing my
>proggie as a py2exe executable and do not want to bloat the size)[/color]

It's probably reinventing the wheel, but here's one I wrote a while
ago. It might serve as a starting point, if nothing else. The actual
code's quite short - it's all those pesky comments and unit tests
that take up all the room ;)

It's also possible to have .py files imported (I forget how) from a
py2exe program, so you can also go down the python formatted config
file route.

Anthony


--

#!/usr/local/bin/python

# A stab at writing a config script
# We'll assume that # and ; are comment characters
# unless they're in quotes

# to run the unit tests, simply execute this script:
# config.py -v

import sys, os, string
import unittest

class config:
""" A class to store configuration values in a text file, and
read them out.

To use this class, simply declare it:

myconfig = config("config. txt")

it will automatically read in all of the values from config.txt,
and myconfig will then behave roughly like a dictionary:

myconfig['name'] # corresponds to name=... in config.txt

There are also the following functions:

myconfig.keys() # a list of all of the attributes
myconfig.delete ('wibble') # delete attribute wibble
myconfig.readco nfig() # read in the config file again
myconfig.writec onfig([filename]) # write the config file
# filename is optional
"""

def __init__(self, configfile):
"""Set up the class, and read in all of the values."""
self._info={}
self._file=conf igfile
if os.access(self. _file, os.F_OK) == 1:
# we can read in the config file
self.readconfig ()


# A couple of simple functions to make using the class easier
# ie. you can just say configclassinst ance['blah']
def __getitem__(sel f, name):
return self._info[name]

def __setitem__(sel f, name, value):
self._info[name] = value

def keys(self):
"""return all of the config values."""
return self._info.keys ()

def delete(self, item):
"""Delete a key from the config directory."""
if item not in self._info.keys ():
raise IndexError, 'Item '+str(item)+' not found in the config.'
else:
del self._info[item]


def readconfig(self ):
"""Read in the config file, line by line."""
# First, clear out the old config:
self._info = {}

# Now, open it and read in each line, calling _readconfigline
to parse it
file = open(self._file , 'r')
for line in file.readlines( ):
self._readconfi gline(line)
file.close()

return

def _readconfigline (self,theline):
"""Given a line, parse it and put it into the self.info dictionary."""

theline = string.strip(th eline)
if theline=='' or theline[0]=='#' or theline[0]==';':
return # this line is empty or a comment

attribute, value = string.split(th eline, '=', 1)

attribute = string.strip(at tribute)
value = string.strip(va lue)

if value[0] == '=':
raise ValueError, "Too many equal signs in "+theline

# First we check for quotes, just to be sure that
# there aren't any nested comment chars in them
value = string.strip(va lue)
if '"' in value:
if value[0]=='"' and string.count(va lue, '"') == 2:
value, ignored = string.split(va lue[1:], '"', 1)
else: # they're playing funny buggers, and gave us
# something like attr=blah"blah"
raise ValueError, "Quotes misplaced in "+theline

else: # no quotes - cut sick!
if '#' in value:
value, ignored = string.split(va lue, '#', 1)
value = string.strip(va lue)
if ';' in value:
value, ignored = string.split(va lue, ';', 1)
value = string.strip(va lue)

self._info[attribute] = value


def writeconfig(sel f, filename=""):
"""Write out all of the values in the config to a file."""

if filename=="":
filename = self._file

if os.access(self. _file, os.F_OK) == 1:
# We might want to raise an error at this point...
raise IOError, "File "+self._fil e+" already exists!"

file = open(filename, 'w')
for item in self._info.keys ():
file.write(item +'="'+self._inf o[item]+'"\n')
file.close()



class testItems(unitt est.TestCase):
"""Test that config works correctly."""

configtest = config("testcon fig.txt")

goodconfiglines = [ # line, attribute, value
['womble=blah', 'womble', 'blah'],
['womble2="blah2 "', 'womble2', 'blah2'],

['blah2="wibble" #comment', 'blah2', 'wibble'],
['blah3="wibble2 " ;comment', 'blah3', 'wibble2'],
['blah6="####" # A comment', 'blah6', '####'],
['blah7=";;;;" ; A comment', 'blah7', ';;;;'],
['blah10=quotes a go go! ; # # ; ; ##; ;#',
'blah10', 'quotes a go go!'],

['blah5 = "womble5"', 'blah5', 'womble5'],
['blah9=womble72 ; comment!', 'blah9', 'womble72'],
['blah11=" spaces should be preserved! "', 'blah11',
" spaces should be preserved! "],

["blah8=This won't break!", 'blah8', "This
won't break!"],

]

badconfiglines = [ # these config lines should throw a ValueError
'blah11="broken "quotes"',
'blah12="more"b roken"quotes"',
'blah13=broken" quotes?"',
'strangequals== "blah"',
'strangeequals2 ="wibble"="blah "',
]

def testAddLines(se lf):
"""Test that lines can be added to the config object."""
# First, clear out self.testconfig :
for item in self.configtest .keys():
self.configtest .delete(item)

for item in self.goodconfig lines:
self.configtest ._readconfiglin e(item[0])
self.assertEqua l(item[2], self.configtest[item[1]])

def testDelete(self ):
"""Test that delete works."""
# First, clear out self.testconfig :
for item in self.configtest .keys():
self.configtest .delete(item)

# Now add and delete
for item in self.goodconfig lines:
self.configtest[item[1]]=item[2]
self.failUnless (self.configtes t[item[1]]==item[2])

self.configtest .delete(item[1])
self.failIf(ite m[2] in self.configtest .keys())

def testConfigKeys( self):
"""Test that items added appear in config.keys()." ""
# First, clear out self.testconfig :
for item in self.configtest .keys():
self.configtest .delete(item)
self.failUnless (self.configtes t.keys()==[])

# Now add and delete
for item in self.goodconfig lines:
self.configtest[item[1]]=item[2]
self.failUnless (item[1] in self.configtest .keys())


def testConfigLines (self):
"""Test that all of the types of config lines can be read
from a file."""
for item in self.goodconfig lines:
#write the line to a config file
file = open("testconfi g.txt", "w")
file.write(item[0])
file.close()

# Now check that it can be read from the file
self.configtest .readconfig()
self.assertEqua l(item[2], self.configtest[item[1]])

def testSanityCheck (self):
"""For each item in goodconfiglines , test that it can be
written to a file
and read back unchanged."""
# First, clear out self.testconfig :
for item in self.configtest .keys():
self.configtest .delete(item)

# Now test!
for item in self.goodconfig lines:
self.configtest[item[1]]=item[2]
os.unlink("test config.txt")
self.configtest .writeconfig("" )
self.configtest .readconfig()
self.assertEqua l(item[2],self.configtes t[item[1]])

def testBadConfigLi nes(self):
"""Test that dodgy config lines will throw an exception."""
for item in self.badconfigl ines:
self.assertRais es(ValueError,
self.configtest ._readconfiglin e, theline=item)



if __name__ == "__main__":
unittest.main()

--
----------------------------------------------------
HyPEraCtiVE? HeY, WhO aRE YoU cALliNg HypERaCtIve?!
aBRiGgS@wEStNeT .cOm.aU
----------------------------------------------------

[anton muhin]

Re: lightweight human-readable config?

Maxim Khesin wrote:[color=blue]
> I want to have a config file with my python proggie, satisfying the
> following requirements:
> 1) support key->(value, default)
> 2) simple and intuitive to read and edit
> 3) easyly readable into a python datastructure (a dictionary?)
> 4) not requiring any heavy libraries needed (I am distributing my
> proggie as a py2exe executable and do not want to bloat the size)
>
> can you guys suggest some format for this? thanks,
> max
>[/color]
I like YAML (www.yaml.org), although the only Python implementation I'm
aware of is a bit buggy.

hth,
anton.

[Nick Vargish]

Re: lightweight human-readable config?

"John Roth" <newsgroups@jhr othjr.com> writes:
[color=blue]
> for line in inFile.readline s():
> key, value = line.split(":")
> configdict[key] = value[/color]

for line in inFile.readline s():
key, value = line.split(':', 1) # only split once, values could
# have ':' in them
key = key.rstrip() # remove trailing spaces from key
value = value.lstrip() # remove leading spaces from value
configdict[key] = value

You could reduce the last three lines to:

configdict[key.rstrip()] = value.lstrip()

Nick

--
# sigmask || 0.2 || 20030107 || public domain || feed this to a python
print reduce(lambda x,y:x+chr(ord(y )-1),' Ojdl!Wbshjti!=o bwAcboefstobudi/psh?')

[Maxim Khesin]

Re: lightweight human-readable config?

> Will the standard library ConfigParser work? Point your web[color=blue]
> browser here:
>
> http://www.python.org/doc/current/li...figParser.html
>
> Regards,
> Heather[/color]

Thanks for the pointer. It is not clear how this supports defaults
(without understanding windows .ini (I cannot find official docs for
that) of RFC 822 (I found too much documentation for this :) )
m

[Skip Montanaro]

Re: lightweight human-readable config?

[color=blue][color=green]
>> http://www.python.org/doc/current/li...figParser.html[/color][/color]

Maxim> Thanks for the pointer. It is not clear how this supports
Maxim> defaults

Take a look at the SpamBayes Options.py file.

Skip

[Maxim Khesin]

Re: lightweight human-readable config?


Nick Vargish wrote:[color=blue]
> "John Roth" <newsgroups@jhr othjr.com> writes:
>
>[color=green]
>>for line in inFile.readline s():
>> key, value = line.split(":")
>> configdict[key] = value[/color]
>
>
> for line in inFile.readline s():
> key, value = line.split(':', 1) # only split once, values could[/color]

Just one comment: using ':' for separator is not a great idea if Windows
paths are come of the potential values.
m

[John Roth]

Re: lightweight human-readable config?


"Maxim Khesin" <max@cNvOiSsiPo AnMtech.com> wrote in message
news:1qvob.4688 1$ri.7441396@tw ister.nyc.rr.co m...[color=blue]
>
> Nick Vargish wrote:[color=green]
> > "John Roth" <newsgroups@jhr othjr.com> writes:
> >
> >[color=darkred]
> >>for line in inFile.readline s():
> >> key, value = line.split(":")
> >> configdict[key] = value[/color]
> >
> >
> > for line in inFile.readline s():
> > key, value = line.split(':', 1) # only split once, values could[/color]
>
> Just one comment: using ':' for separator is not a great idea if Windows
> paths are come of the potential values.[/color]

That's why the "1" as the second parameter. It makes sure that
split doesn't do more than one split.

John Roth[color=blue]
> m
>[/color]

[Corey Coughlin]

Re: lightweight human-readable config?

I actually have a whole set of programs that use a format like this,
it's actually parsed on spaces and keywords, so you just say:

keyword1 value1 value2 value3

keyword2 value1

keyword3
value1
value2

and so on. It's pretty free format, and relatively easy to parse, all
you need to set up is a list of keywords, and in return from these
functions you get a dictionary with the value lists for each keyword.
You can also add comments by adding '#' signs, and the rest of the
line is a comment. Here are the functions:

def ParseCommandFil e(commfn, commkeys):
commwords = ReadCommandFile (commfn)
return ParseList(commw ords, commkeys)

def ReadCommandFile (commfn):
if not (commfn and os.path.exists( commfn):
raise OSError, "Can't find this path: %s" % fname
commf = open(commfn,'r' )
commtext = ''
for line in commf:
if '#' in line:
line = line[:string.find(li ne,'#')]
commtext = commtext + ' ' + line
commf.close()
commwords = string.split(co mmtext)
return commwords

def ParseList(comml ist, commkeys):
commvals = {}
for ck in commkeys:
commvals[ck] = []
currkey = ''
currval = []
foundkeys = commkeys[:]
for cword in commlist:
if currkey:
if cword in foundkeys:
commvals[currkey] = currval
currkey = cword
foundkeys.remov e(currkey)
currval = []
else:
currval.append( cword)
else:
if cword in foundkeys:
currkey = cword
foundkeys.remov e(currkey)
if currkey:
commvals[currkey] = currval
return commvals

There it is, maybe not the most efficient code, but it lets you write
a command file parsed on spaces, so there's no need to worry about
extra syntax characters, which keeps it nice and clean.

Maxim Khesin <max@cNvOiSsiPo AnMtech.com> wrote in message news:<p4Xnb.173 67$Gq.5475490@t wister.nyc.rr.c om>...[color=blue]
> I want to have a config file with my python proggie, satisfying the
> following requirements:
> 1) support key->(value, default)
> 2) simple and intuitive to read and edit
> 3) easyly readable into a python datastructure (a dictionary?)
> 4) not requiring any heavy libraries needed (I am distributing my
> proggie as a py2exe executable and do not want to bloat the size)
>
> can you guys suggest some format for this? thanks,
> max[/color]

[Larry Bates]

Re: lightweight human-readable config?

You can have defaults in configparser .INI file if you like.

import ConfigParser
ini_filename='p rogram.ini'
defaults={'key1 ':'value1','key 2':'value2')
#
# This will initialize the ConfigParser with dictionary
# of key,value defaults.
INI=ConfigParse r.ConfigParser( defaults)
#
# Make sure .INI file exists
#
if not os.path.exists:
print "Unable to locate .INI filename=%s" % ini_filename
sys.exit(2)
#
# Now read the .INI file's contents
#
INI.read(ini_fi lename)

section='INIT'
option='debug'
try: debug=INI.getbo olean(section, option)
except: debug=0

option='trace'
try: trace=INI.getbo olean(section, option)
except: trace=0

section='RUNTIM E'
option='first_i nvoice'
try: first_invoice=I NI.get(section, option)
except:
print ".INI file missing first_invoice key"
sys.exit(2)

option='last_in voice'
try: last_invoice=IN I.get(section, option)
except:
print ".INI file missing last_invoice key"
sys.exit(2)

..
.. remainder of program
..



..INI files have following format

[SECTION1]
KEY1=
KEY2=
KEY3=

[SECTION2]
KEY1=
KEY2=
KEY3=

You can have as many sections as you like and keynames
are unique under a section. Normally I do something like:

[INIT]
debug=1
trace=1
logfile='progra m.log'

[RUNTIME]
first_invoice=1 2345
last_invoice=99 999

This way I can turn on/off debug and trace modes and
have the program pick up RUNTIME parameters for processing.

Hope information helps.
-Larry






"Maxim Khesin" <max@cNvOiSsiPo AnMtech.com> wrote in message
news:U1fob.2314 0$Gq.6423269@tw ister.nyc.rr.co m...[color=blue][color=green]
> > Will the standard library ConfigParser work? Point your web
> > browser here:
> >
> > http://www.python.org/doc/current/li...figParser.html
> >
> > Regards,
> > Heather[/color]
>
> Thanks for the pointer. It is not clear how this supports defaults
> (without understanding windows .ini (I cannot find official docs for
> that) of RFC 822 (I found too much documentation for this :) )
> m
>[/color]