Extract information from HTML table

**placid** · Apr 1 '07, 12:55 PM

Re: Extract information from HTML table

On Apr 1, 10:13 pm, "Ulysse" <maxim...@gmail .comwrote:

Hello,
>
I'm trying to extract the data from HTML table. Here is the part of
the HTML source :
"""
<tr>
<td class="tdn" valign="top">
<input name="x44553130 " value="y"
type="checkbox" ></td>
<td class="tdn" valign="top" width="30%">
Sat, 31.03.2007 - 20:24:00</td>
<td class="tdn">
<a href="http://s2.bitefight.fr/bite/
bericht.php?q=0 1bf0ba7258ad976 d890379f987d444 e&beid=2628 033">Vous
avez tendu une embuscade à votre victime !</a></td>
</tr>
<tr>
<td class="tdn" valign="top">
<input name="x44553032 " value="y"
type="checkbox" ></td>
<td class="tdn" valign="top" width="30%">
Sat, 31.03.2007 - 20:14:35</td>
<td class="tdn">
<a href="http://s2.bitefight.fr/bite/
bericht.php?q=0 1bf0ba7258ad976 d890379f987d444 e&beid=2628 007">Vous
avez tendu une embuscade à votre victime !</a></td>
</tr>
<tr>
<td class="tdn" valign="top">
<input name="x44552991 " value="y"
type="checkbox" ></td>
<td class="tdn" valign="top" width="30%">
Sat, 31.03.2007 - 20:11:39</td>
<td class="tdn"Vous avez bien accompli votre
tâche de Gardien de Cimetière et vous vous
voyez remis votre salaire comme récompense.
Vous recevez 320
<img src="messages-bite_fichiers/res2.gif"
alt="Or" align="absmiddl e" border="0">
et collectez 3 d'expérience !</td>
</tr>
"""
>
I would like to transform this in following thing :
>
Date : Sat, 31.03.2007 - 20:24:00
ContainType : Link
LinkText : Vous avez tendu une embuscade à votre victime !
LinkURL :http://s2.bitefight.fr/bite/bericht....976d890379f987...
>
Date : Sat, 31.03.2007 - 20:14:35
ContainType : Link
LinkText : Vous avez tendu une embuscade à votre victime !
LinkURL :http://s2.bitefight.fr/bite/bericht....976d890379f987...
>
Date : Sat, 31.03.2007 - 20:14:35
ContainType : Text
Contain : Vous avez bien accompli votre tâche de Gardien de Cimetière
et vous vous
voyez remis votre salaire comme récompense.
Vous recevez 320 et collectez 3 d'expérience !
>
....
>
Do you know the way to do it ?

You can use Beautiful Soup http://www.crummy.com/software/BeautifulSoup/

see this page to see how you can search for tags, then retrieve the
contents

Beautiful Soup documentation

http://www.crummy.com/software/BeautifulSoup/documentation.html#Searching%20Within%20the%20Parse%20Tree

Cheers

**irstas@gmail.com** · Apr 1 '07, 12:55 PM

Re: Extract information from HTML table

On Apr 1, 3:13 pm, "Ulysse" <maxim...@gmail .comwrote:

Hello,
>
I'm trying to extract the data from HTML table. Here is the part of
the HTML source :
>
....
>
Do you know the way to do it ?

Beautiful Soup is an easy way to parse HTML (that may be broken).

Beautiful Soup: We called him Tortoise because he taught us.

http://www.crummy.com/software/BeautifulSoup/

Here's a start of a parser for your HTML:

soup = BeautifulSoup(t xt)
for tr in soup('tr'):
dateTd, textTd = tr('td')[1:]
print 'Date :', dateTd.contents[0].strip()
print textTd #element still needs parsing

where txt is the string in your message.

**Ulysse** · Apr 1 '07, 03:05 PM

Re: Extract information from HTML table

On Apr 1, 2:52 pm, irs...@gmail.co m wrote:

On Apr 1, 3:13 pm, "Ulysse" <maxim...@gmail .comwrote:
>

Hello,

>

I'm trying to extract the data from HTML table. Here is the part of
the HTML source :

>

....

>

Do you know the way to do it ?

>
Beautiful Soup is an easy way to parse HTML (that may be broken).http://www.crummy.com/software/BeautifulSoup/
>
Here's a start of a parser for your HTML:
>
soup = BeautifulSoup(t xt)
for tr in soup('tr'):
dateTd, textTd = tr('td')[1:]
print 'Date :', dateTd.contents[0].strip()
print textTd #element still needs parsing
>
where txt is the string in your message.

I have seen the Beautiful Soup online help and tried to apply that to
my problem. But it seems to be a little bit hard. I will rather try to
do this with regular expressions...

**Dotan Cohen** · Apr 1 '07, 07:55 PM

Re: Extract information from HTML table

On 1 Apr 2007 07:56:04 -0700, Ulysse <maxime.p@gmail .comwrote:

I have seen the Beautiful Soup online help and tried to apply that to
my problem. But it seems to be a little bit hard. I will rather try to
do this with regular expressions...
>

If you think that Beautiful Soup is difficult than wait till you try
to do this with regexes. Granted you know the exact format of the HTML
you are scraping will help, if you ever need to parse HTML from an
unknown source than Beautiful Soup is the only way to go. Not all HTML
authors close their td and tr tags, and sometimes there are attributes
to those tags. If you plan on ever reusing the code or the format of
the HTML may change, then you are best off sticking with Beautiful
Soup.

Dotan Cohen

http://lyricslist.com/

Recent Questions: - Questions recently asked on What-Is-What.com

http://what-is-what.com/

Questions recently asked on What-Is-What.com

**anjesh** · Apr 2 '07, 08:45 AM

Re: Extract information from HTML table

On Apr 2, 12:54 am, "Dotan Cohen" <dotanco...@gma il.comwrote:

On 1 Apr 2007 07:56:04 -0700, Ulysse <maxim...@gmail .comwrote:
>

I have seen the Beautiful Soup online help and tried to apply that to
my problem. But it seems to be a little bit hard. I will rather try to
do this with regular expressions...

>
If you think that Beautiful Soup is difficult than wait till you try
to do this with regexes. Granted you know the exact format of the HTML
you are scraping will help, if you ever need to parse HTML from an
unknown source than Beautiful Soup is the only way to go. Not all HTML
authors close their td and tr tags, and sometimes there are attributes
to those tags. If you plan on ever reusing the code or the format of
the HTML may change, then you are best off sticking with Beautiful
Soup.
>
Dotan Cohen
>
http://lyricslist.com/http://what-is-what.com/

Have you tried HTMLParser. It can do the task you want to perform

Page not found — Python 3.14.7 documentation

http://docs.python.org/lib/module-HTMLParser.html

-anjesh

**Cameron Laird** · Apr 2 '07, 08:15 PM

Re: Extract information from HTML table

In article <1175503135.234 560.51730@n59g2 000hsh.googlegr oups.com>,
anjesh <anjeshtuladhar @gmail.comwrote :

>On Apr 2, 12:54 am, "Dotan Cohen" <dotanco...@gma il.comwrote:

>On 1 Apr 2007 07:56:04 -0700, Ulysse <maxim...@gmail .comwrote:
>>

I have seen the Beautiful Soup online help and tried to apply that to
my problem. But it seems to be a little bit hard. I will rather try to
do this with regular expressions...

>>
>If you think that Beautiful Soup is difficult than wait till you try
>to do this with regexes. Granted you know the exact format of the HTML
>you are scraping will help, if you ever need to parse HTML from an
>unknown source than Beautiful Soup is the only way to go. Not all HTML
>authors close their td and tr tags, and sometimes there are attributes
>to those tags. If you plan on ever reusing the code or the format of
>the HTML may change, then you are best off sticking with Beautiful
>Soup.
>>
>Dotan Cohen
>>
>http://lyricslist.com/http://what-is-what.com/

>
>
>Have you tried HTMLParser. It can do the task you want to perform
>http://docs.python.org/lib/module-HTMLParser.html
>
>-anjesh
>

Yes, except that these last two follow-ups UNDERstate the difficulty--in
fact, the impossibility--of achieving adequate results on this problem
with regular expressions. We'll help with the documentation for HTMLParser
and BeautifulSoup. REs are an invitation to madness.

<URL: http://www.unixreview.com/documents/s=10121/ur0702e/ might amuse
those who want to think more about REs.

**Ulysse** · Apr 2 '07, 09:15 PM

Re: Extract information from HTML table

On Apr 2, 9:28 pm, cla...@lairds.u s (Cameron Laird) wrote:

In article <1175503135.234 560.51...@n59g2 000hsh.googlegr oups.com>,
>
>
>
anjesh <anjeshtulad... @gmail.comwrote :

On Apr 2, 12:54 am, "Dotan Cohen" <dotanco...@gma il.comwrote:

On 1 Apr 2007 07:56:04 -0700, Ulysse <maxim...@gmail .comwrote:

>

I have seen the Beautiful Soup online help and tried to apply that to
my problem. But it seems to be a little bit hard. I will rather try to
do this with regular expressions...

>

If you think that Beautiful Soup is difficult than wait till you try
to do this with regexes. Granted you know the exact format of the HTML
you are scraping will help, if you ever need to parse HTML from an
unknown source than Beautiful Soup is the only way to go. Not all HTML
authors close their td and tr tags, and sometimes there are attributes
to those tags. If you plan on ever reusing the code or the format of
the HTML may change, then you are best off sticking with Beautiful
Soup.

>

Dotan Cohen

>

>http://lyricslist.com/http://what-is-what.com/

>

Have you tried HTMLParser. It can do the task you want to perform
http://docs.python.org/lib/module-HTMLParser.html

>

-anjesh

>
Yes, except that these last two follow-ups UNDERstate the difficulty--in
fact, the impossibility--of achieving adequate results on this problem
with regular expressions. We'll help with the documentation for HTMLParser
and BeautifulSoup. REs are an invitation to madness.
>
<URL:http://www.unixreview. com/documents/s=10121/ur0702e/might amuse
those who want to think more about REs.

r'(\d{2}\.\d{2} \.\d{4} - \d{2}:\d{2}:\d{ 2})</td>\W*?<td class="tdn">
\W*?<a href="(.*?)">(. *?)</a>.*?</td>'

r'(\d{2}\.\d{2} \.\d{4} - \d{2}:\d{2}:\d{ 2}).*?player\.p hp.*?>(.*?)</
a>.*?<textarea. *?>(.*?)</textarea>'

r'(\d{2}\.\d{2} \.\d{4} - \d{2}:\d{2}:\d{ 2})</td>\W*?<td class="tdn">
\W*?Message au clan de :([a-zA-Z0-9_\-]+?)\W*<br>(.*?) </th>'

These three REs extract all data I need. That not exactly apply to the
given string.
I read the article but I didn't understood why REs are invitation to
madness...

Extract information from HTML table

Extract information from HTML table

Comment

Comment

Comment

Comment

Comment

Comment

Comment