XML parsing per record

[Fredrik Lundh]

Re: XML parsing per record

Willem Ligtenberg wrote:
[color=blue]
> By the way, I know about findall, but when I iterate thruogh it like:
> for x in function:
> print 'function', x
>
> I get:
> function <Element 'Prot-ref_name_E' at 0xb7d10cf8>
> function <Element 'Prot-ref_name_E' at 0xb7d10d10>
>
> But ofcourse I want the information in there...[/color]

for x in function:
print 'function', x.text

</F>

[William Park]

Re: XML parsing per record

Willem Ligtenberg <WLigtenberg@gm ail.com> wrote:
....[color=blue]
> ID --> <Gene-track_geneid>32 0632</Gene-track_geneid>[/color]
....[color=blue]
> Product-type --> <Entrezgene_typ e value="protein-coding">6</Entrezgene_type >[/color]
....[color=blue]
> EC --> <Prot-ref_ec>
> <Prot-ref_ec_E>1.5.1. 5</Prot-ref_ec_E>
> <Prot-ref_ec_E>3.5.4. 9</Prot-ref_ec_E>
> </Prot-ref_ec>[/color]
....[color=blue]
>
> Some can happen more than once in a record.[/color]

Since all your data are contained in unique tags on individual lines,
you can tackle this so many different ways. Okey, that's your input
format. What is your output format?

--
William Park <opengeometry@y ahoo.ca>, Toronto, Canada
Slackware Linux -- because it works.

[Willem Ligtenberg]

Re: XML parsing per record

Is there an easy way, to couple data together. Because I have discoverd an
irritating feature in the xml file.
Sometimes this is a database reference:
<Dbtag>
<Dbtag_db>UCS C</Dbtag_db>
<Dbtag_tag>
<Object-id>
<Object-id_str>1234</Object-id_str>
</Object-id>
</Dbtag_tag>
</Dbtag>

And sometimes:

<Dbtag>
<Dbtag_db>UCS C</Dbtag_db>
<Dbtag_tag>
<Object-id>
<Object-id_id>1234</Object-id_id>
</Object-id>
</Dbtag_tag>
</Dbtag>

So I get a list database names and two! lists of ID's
And those two are in no way related. Is there an easy way to create a
dictionary like this DBname --> ID
If not, I still might need to revert to SAX... :(

On Fri, 22 Apr 2005 15:56:29 +0200, Willem Ligtenberg wrote:
[color=blue]
> As you can read in the other post of mine, my problem was with the
> iterating through the list. didn't know that you should do. e.text. I did
> only print e, not print e.text
> Did read documentation, but must admit not everything.
>
> Anyway, thank you very much!
>
> On Fri, 22 Apr 2005 15:47:08 +0200, Fredrik Lundh wrote:
>[color=green]
>> Willem Ligtenberg wrote:
>>[color=darkred]
>>> As I'm trying to write the code using cElementTree.
>>> I stumble across one problem. Sometimes there are multiple values to
>>> retrieve from one record for the same element. Like this:
>>> <Prot-ref_name_E>ATP-binding cassette, subfamily G, member 1</Prot-ref_name_E>
>>> <Prot-ref_name_E>ATP-binding cassette 8</Prot-ref_name_E>
>>>
>>> How do you get not only the first, but the rest as well, so that I can
>>> store it in a list.[/color]
>>
>> findall returns a list of matching elements. if "elem" is the paretnt element,
>> this gives you a list of the text inside all Prot-ref_name_E child elements:
>>
>> [e.text for e in elem.findall("P rot-ref_name_E")]
>>
>> (you have read the elementtree documentation, I hope?)
>>
>> </F>[/color][/color]

[Fredrik Lundh]

Re: XML parsing per record

Willem Ligtenberg wrote:
[color=blue]
> So I get a list database names and two! lists of ID's
> And those two are in no way related. Is there an easy way to create a
> dictionary like this DBname --> ID[/color]

why not just check for both alternatives?

text = elem.findtext(" Object-id_str")
if text is None:
text = elem.findtext(" Object-id_id")

(or you can loop over the child elements and map elem.tag through a
dictionary...)
[color=blue]
> If not, I still might need to revert to SAX... :([/color]

you still have to check for both alternatives...

(if you find a parsing problem that you cannot solve with a light-weight
DOM, SAX won't help you...)

</F>

[Willem Ligtenberg]

Re: XML parsing per record

Since there are more than one database references possible per record you
should get per record a list of database names, database strings and
databases ids. (where the strings and the id's are really the same thing...)
So per record you check for both alternatives but since there could be
more than one, you do findall and get a (unsorted) list back. And now you
don't know which ID belonged to which database...
See my problem?

Cheers,

Willem

On Fri, 22 Apr 2005 19:38:03 +0200, Fredrik Lundh wrote:
[color=blue]
> Willem Ligtenberg wrote:
>[color=green]
>> So I get a list database names and two! lists of ID's
>> And those two are in no way related. Is there an easy way to create a
>> dictionary like this DBname --> ID[/color]
>
> why not just check for both alternatives?
>
> text = elem.findtext(" Object-id_str")
> if text is None:
> text = elem.findtext(" Object-id_id")
>
> (or you can loop over the child elements and map elem.tag through a
> dictionary...)
>[color=green]
>> If not, I still might need to revert to SAX... :([/color]
>
> you still have to check for both alternatives...
>
> (if you find a parsing problem that you cannot solve with a light-weight
> DOM, SAX won't help you...)
>
> </F>[/color]

[Fredrik Lundh]

Re: XML parsing per record

Willem Ligtenberg wrote:
[color=blue]
> Since there are more than one database references possible per record you
> should get per record a list of database names, database strings and
> databases ids. (where the strings and the id's are really the same thing...)
> So per record you check for both alternatives but since there could be
> more than one, you do findall and get a (unsorted) list back.[/color]

findall returns matching elements in document order.
[color=blue]
> And now you don't know which ID belonged to which database...[/color]

why not? by looking at each database separately, surely you must be
able to figure out if the subelement holds an ID or a string? sure, if you
do document.findal l(".//Object-id_id"), you'll get all IDs in document
order. but if you do record.findall( ".//Dbtag"), you get a list of all Dbtag
elements, and can then look inside them to see what they contain.
[color=blue]
> See my problem?[/color]

I'm afraid not. the document seems to have a clear structure; for some
reason, you don't seem to take that into account in your program.

</F>

[Fredrik Lundh]

Re: XML parsing per record

[color=blue]
> order. but if you do record.findall( ".//Dbtag"), you get a list of all Dbtag
> elements[/color]

make that "you get a list of all Dbtag elements in that record"

</F>

[Kent Johnson]

Re: XML parsing per record

Willem Ligtenberg wrote:[color=blue]
> Is there an easy way, to couple data together. Because I have discoverd an
> irritating feature in the xml file.
> Sometimes this is a database reference:
> <Dbtag>
> <Dbtag_db>UCS C</Dbtag_db>
> <Dbtag_tag>
> <Object-id>
> <Object-id_str>1234</Object-id_str>
> </Object-id>
> </Dbtag_tag>
> </Dbtag>
>
> And sometimes:
>
> <Dbtag>
> <Dbtag_db>UCS C</Dbtag_db>
> <Dbtag_tag>
> <Object-id>
> <Object-id_id>1234</Object-id_id>
> </Object-id>
> </Dbtag_tag>
> </Dbtag>
>
> So I get a list database names and two! lists of ID's
> And those two are in no way related. Is there an easy way to create a
> dictionary like this DBname --> ID
> If not, I still might need to revert to SAX... :([/color]

None of your requirements sound particularly difficult to implement. If you would post a complete
example of the data you want to parse and the data you would like to end up it would be easier to
help you. The sample data you posted originally does not have many of the fields you want to extract
and your example of what you want to end up with is not too clear either.

If you are having trouble with ElementTree I expect you will be completely lost with SAX,
ElementTree is much easier to work with and cElementTree is very fast.

Kent