Creating referenceable objects from XML

[Michael Williams]

Hi All,

I'm looking for a quality Python XML implementation. All of the DOM
and SAX implementations I've come across so far are rather
convoluted. Are there any quality implementations that will (after
parsing the XML) return an object that is accessible by name? Such as
the following:




xml = """
<book>
<title>MyBook </title>
<author>the author</author>
</book>
"""





And after parsing the XML allow me to access it as so:

book.title

I need it to somehow convert my XML to intuitively referenceable
object. Any ideas? I could even do it myself if I knew the
mechanism by which python classes do this (create variables on the fly).

Thanks in advance!

[rurpy@yahoo.com]

Re: Creating referenceable objects from XML


Michael Williams wrote:[color=blue]
> Hi All,
>
> I'm looking for a quality Python XML implementation. All of the DOM
> and SAX implementations I've come across so far are rather
> convoluted. Are there any quality implementations that will (after
> parsing the XML) return an object that is accessible by name? Such as
> the following:
>
>
> xml = """
> <book>
> <title>MyBook </title>
> <author>the author</author>
> </book>
> """
>
> And after parsing the XML allow me to access it as so:
>
> book.title
>
> I need it to somehow convert my XML to intuitively referenceable
> object. Any ideas? I could even do it myself if I knew the
> mechanism by which python classes do this (create variables on the fly).
>
> Thanks in advance![/color]

You might want to take a look at Fredrik Lundh's ElementTree
(and cElementTree) modules:

http://effbot.org/zone/element-index.htm

[Diez B. Roggisch]

Re: Creating referenceable objects from XML

Michael Williams wrote:[color=blue]
> I'm looking for a quality Python XML implementation. All of the DOM
> and SAX implementations I've come across so far are rather
> convoluted.[/color]

Welcome to the wonderful world of XML.
[color=blue]
> I need it to somehow convert my XML to intuitively referenceable
> object. Any ideas? I could even do it myself if I knew the
> mechanism by which python classes do this (create variables on the fly).[/color]

You've been given the advice to use ElementTree - I can only second
that.

But if for whatever reason you do want to do it yourself (or for future
use), the

getattr/setattr

functions are what you are looking for. Look them up in TFM.

Regards,

Diez

[Gerard Flanagan]

Re: Creating referenceable objects from XML


Michael Williams wrote:[color=blue]
> Hi All,
>
> I'm looking for a quality Python XML implementation. All of the DOM
> and SAX implementations I've come across so far are rather
> convoluted. Are there any quality implementations that will (after
> parsing the XML) return an object that is accessible by name? Such as
> the following:
>
>
>
>
> xml = """
> <book>
> <title>MyBook </title>
> <author>the author</author>
> </book>
> """
>
>
>
>
>
> And after parsing the XML allow me to access it as so:
>
> book.title
>
> I need it to somehow convert my XML to intuitively referenceable
> object. Any ideas? I could even do it myself if I knew the
> mechanism by which python classes do this (create variables on the fly).
>
> Thanks in advance![/color]

Michael

Here's an approach to ElementTree that worked for me. It's not generic
or anything and a bit brittle (eg. it won't handle missing nodes) but
maybe for a simple, flat schema or for a prototype?

All the best

Gerard

(TOY CODE - NOT TESTED MUCH)

from elementtree import ElementTree

class ElementWrapper( object):

def __tostring(self ):
return ElementTree.tos tring(self.elem ent)

def __fromstring(se lf, xml):
self.element = ElementTree.fro mstring(xml)

xml = property( __tostring, __fromstring )

def __init__(self, element=None):
self.element = element

def __str__(self):
return self.xml

def parse(self, infile):
tree = ElementTree.par se(infile)
self.element = tree.getroot()

def write(self, outfile):
ElementTree.Ele mentTree(self.e lement).write(o utfile)

###########


from elementtree.Ele mentTree import Element
from elementwrapper import ElementWrapper

xmlns = 'http://schemas/email/0.1'


class MailDocument(El ementWrapper):

def __build_element (self):
root = Element('{%s}Ma il' % xmlns)
root.append( Element('{%s}Da te' % xmlns) )
root.append( Element('{%s}Fr om' % xmlns) )
root.append( Element('{%s}Su bject' % xmlns) )
root.append( Element('{%s}To ' % xmlns) )
root.append( Element('{%s}Cc ' % xmlns) )
root.append( Element('{%s}Bo dy' % xmlns) )
root.append( Element('{%s}At tachments' % xmlns) )
self.element = root

############### ############### ############### ########
# Properties
#
def __get_uid(self) :
return self.element.ge t('id')

def __set_uid(self, id=''):
self.element.se t('id', id)

def __get_date(self ):
return self.element[0].text

def __set_date(self , value=''):
self.element[0].text = value

def __get_from(self ):
addr = self.element[1].get('address')
nm = self.element[1].get('name')
return addr, nm

def __get_subject(s elf):
return self.element[2].text

def __set_subject(s elf, value=''):
self.element[2].text = value

def __get_body(self ):
return self.element[5].text

def __set_body(self , value=''):
self.element[5].text = value

uid = property( __get_uid, __set_uid )
From = property( __get_from)
subject = property( __get_subject, __set_subject )
date = property( __get_date, __set_date )
body = property( __get_body, __set_body )

def set_from_header (self, address='', name=''):
self.element[1].set('address', address)
self.element[1].set('name', name)
#
# End Properties
############### ############### ############### ########

############### ############### ############### ########
# Lists
#
def add_to_header(s elf, address='', name=''):
self.__add_mail to( self.element[3], address, name )

def remove_to_heade r(self, index):
elem = self.element[3][index]
self.element[3].remove(elem)

def add_cc_header(s elf, address='', name=''):
self.__add_mail to( self.element[4], address, name )

def remove_cc_heade r(self, index):
elem = self.element[4][index]
self.element[4].remove(elem)

def add_attachment( self, filename='', fileuri='', filetype=''):
elem = Element("{%s}Ur i" % xmlns, value=fileuri, type=filetype
)
elem.text = filename
self.element[6].append( elem )

def remove_attachme nt(self, index):
elem = self.element[6][index]
self.element[6].remove(elem)

def __add_mailto(se lf, element, Address='', Name=''):
element.append( Element("{%s}ma ilto" % xmlns, address=Address ,
name=Name ) )

def get_to_headers( self):
hdrs = []
for item in self.element[3]:
hdrs.append( ( item.get('addre ss'), item.get('name' ) ) )
return hdrs

def get_cc_headers( self):
hdrs = []
for item in self.element[4]:
hdrs.append( (item.get('addr ess'), item.get('name' ) ) )
return hdrs

def get_attachments (self):
ret = []
for item in self.__element[6]:
hdrs.append( (item.text, item.get('value '),
item.get('type' ) ) )
return hdrs
#
# End Lists
############### ############### ############### ###########

############### ############### ############### ###########
# Initialise
#
def __init__(self):
self.__build_el ement()
self.__set_uid( )
self.__set_date ()
self.__set_subj ect()
self.set_from_h eader()
self.__set_body ()
#
# End Initialise
############### ############### ############### ###########

xml_test ='''
<mail:Mail xmlns:mail="htt p://schemas/email/0.1">
<mail:Date>10/10/05</mail:Date>
<mail:From address='me@org .org' name='Mr. Jones'/>
<mail:Subject>j ust a note</mail:Subject>
<mail:To>
<mail:mailto address='you@or g.org' name='Mrs Jones' />
<mail:mailto address='a.noth er@org.org' name='Alan Nother' />
</mail:To>
<mail:Cc></mail:Cc>
<mail:Body>hi there,
just a note to say hi there!</mail:Body>
<mail:Attachmen ts></mail:Attachment s>
</mail:Mail>
'''
if __name__ == '__main__':
mail = MailDocument()
mail.xml = xml_test
#mail.parse('te st/data/test.xml')
print 'From: ' + mail.From[0]
print 'Subject: ' + mail.subject
mail.set_from_h eader('new@new. com')
print 'From: ' + mail.From[0]
mail.add_to_hea der('aaa.bbb@cc c', 'aaaaaa')
mail.add_to_hea der('fff.ggg@hh h', 'ffffff')
print 'To:'
for hdr in mail.get_to_hea ders():
print hdr
mail.remove_to_ header(1)
print 'To:'
for hdr in mail.get_to_hea ders():
print hdr
#mail.write('te st_copy.xml')

[Laurent Pointal]

Re: Creating referenceable objects from XML

Michael Williams wrote:[color=blue]
> Hi All,
>
> I'm looking for a quality Python XML implementation. All of the DOM
> and SAX implementations I've come across so far are rather convoluted.
> Are there any quality implementations that will (after parsing the XML)
> return an object that is accessible by name? Such as the following:
>
>
>
>
> xml = """
> <book>
> <title>MyBook </title>
> <author>the author</author>
> </book>
> """
>
>
>
>
>
> And after parsing the XML allow me to access it as so:
>
> book.title
>
> I need it to somehow convert my XML to intuitively referenceable
> object. Any ideas? I could even do it myself if I knew the mechanism
> by which python classes do this (create variables on the fly).
>
> Thanks in advance![/color]

Another tool (ElementsTree already quoted): Amara
( http://uche.ogbuji.net/uche.ogbuji.n.../4suite/amara/ )

[never tested but bookmarked as it seem interresting]

A+

Laurent.

[uche.ogbuji@gmail.com]

Re: Creating referenceable objects from XML

Michael Williams wrote:[color=blue]
> Hi All,[/color]
[color=blue]
> I'm looking for a quality Python XML implementation. All of the DOM
> and SAX implementations I've come across so far are rather
> convoluted. Are there any quality implementations that will (after
> parsing the XML) return an object that is accessible by name? Such as
> the following:[/color]
[color=blue]
> xml = """
> <book>
> <title>MyBook </title>
> <author>the author</author>
> </book>
> """[/color]
[color=blue]
> And after parsing the XML allow me to access it as so:[/color]
[color=blue]
> book.title[/color]
[color=blue]
> I need it to somehow convert my XML to intuitively referenceable
> object. Any ideas? I could even do it myself if I knew the
> mechanism by which python classes do this (create variables on the fly).[/color]

Looks as if MIchael is working with Amara now, but I did want to note
for the record that APIs that allow one to access a node in the
"book.title " fashion are what I call Python data bindings.

Python data bindings I usually point out are:

Amara Bindery: http://www.xml.com/pub/a/2005/01/19/amara.html
Gnosis: http://www.xml.com/pub/a/2003/07/02/py-xml.html
generateDS: http://www.xml.com/pub/a/2003/06/11/py-xml.html

Based on updates to EaseXML in response to my article another entry
might be:

EaseXML: http://www.xml.com/pub/a/2005/07/27/py-xml.html

ElementTree ( http://www.xml.com/pub/a/2003/02/12/py-xml.html ) is a
Python InfoSet rather than a Python data binding. You access nodes
using generic names related to the node type rather than the node name.
Whether data bindings or Infosets are your preference is a matter of
taste, but it's a useful distinction to make between the approaches.
It looks as if Gerald Flanagan has constructed a little specialized
binding tool on top of ElementTree, and that's one possible hybrid
approach.

xmltramp ( http://www.aaronsw.com/2002/xmltramp/ ) is another
interesting hybrid.

--
Uche Ogbuji Fourthought, Inc.
http://uche.ogbuji.net http://fourthought.com
http://copia.ogbuji.net http://4Suite.org
Articles: http://uche.ogbuji.net/tech/publications/

[Alan Kennedy]

Re: Creating referenceable objects from XML

[Michael Williams][color=blue]
> I need it to somehow convert my XML to intuitively referenceable
> object. Any ideas? I could even do it myself if I knew the mechanism
> by which python classes do this (create variables on the fly).[/color]

You seem to already have a fair idea what kind of model you need, and to
know that there is a simple way for you to create one. I encourage you
to progress on this path: it will increase the depth of your understanding.

One mistake I think that some people make about XML is relying on other
peoples interpretations of the subject, rather than forming their own
opinions.

The multitude of document models provided by everyone and his mother all
make assumptions about how the components of the model will be accessed,
in what order those components will be accessed, how often and when, how
memory efficient the model is, etc, etc.

To really understand the trade-offs and strengths of all the different
models, it is a good exercise to build your own object model. It's a
simple exercise, due to pythons highly dynamic nature. Understanding
your own model will help you understand what the other models do and do
not provide. You can then evaluate other off-the-shelf models for your
specific applications: I always find different XML tools suit different
situations.

See this post of mine from a couple years back about different ways of
building your own document/data models.

http://groups.google.com/group/comp.lang.python/msg/e2a4a1c35395ffec

I think the reference to the ActiveState recipe will be of particular
interest, since you could have a running example very quickly indeed.

See also my tutorial post on extracting document content from a SAX
stream. I gave the example of a simple stack-based xpath-style
expression matcher.

http://groups.google.com/group/comp.lang.python/msg/6853bddbb9326948

Also contained in that thread is an illuminating and productive
discussion between the effbot and myself about how wonderfully simple
ElementTree makes this, not to mention unbeatably efficient.

this-week-i-ave-been-mostly-using-kid-for-templating-ly'yrs,

--
alan kennedy
------------------------------------------------------
email alan: http://xhaus.com/contact/alan

[Fredrik Lundh]

Re: Creating referenceable objects from XML

uche.ogbuji@gma il.com wrote:
[color=blue]
> ElementTree ( http://www.xml.com/pub/a/2003/02/12/py-xml.html ) is a
> Python InfoSet rather than a Python data binding. You access nodes
> using generic names related to the node type rather than the node name.
> Whether data bindings or Infosets are your preference is a matter of
> taste, but it's a useful distinction to make between the approaches.
> It looks as if Gerald Flanagan has constructed a little specialized
> binding tool on top of ElementTree, and that's one possible hybrid
> approach.[/color]

in my experience, it's hard to make a python/xml mapping that's well suited for all
possible use cases (many bindings suffer from issues with namespaces, collisions
between tags/attribute names and python names, etc), but it's usually trivial to write
a custom wrapper for a specific case.

for most normal use, manual infoset navigation is often the easiest way to pull out
data from the infoset (find, get, findtext, int, float, etc).

for certain cases, creating wrappers on demand can be quite efficient; e.g.

http://online.effbot.org/2003_07_01_archive.htm#element-tricks

and for highly regular cases, incremental parsing/conversion up front is often the
fastest and most efficient way to deal with data; e.g.

http://effbot.org/zone/element-iterparse.htm#plist

</F>