Help with regular expressions

[dmbkiwi]

I have a problem. I have written a python based theme for a linux app
called superkaramba, which is effectively an engine for desktop applets
that utilises python as its theming language. The script basically parses
weather websites, and displays the info in a visually appealing way.

A couple of other people have contributed code to this project,
particularly relating to the parsing of the websites. Unfortunately, it
is not parsing one particular part of the website properly. This is
because it is expecting the data to be in a certain form, and occasionally
it is in a different form. Unfortunately this causes the entire script to
fail to run.

Unfortunately, I know very little about regular expressions and can't get
hold of the person who wrote this part of the script. The other issue I am
struggling with is that there are no error messages as to what's going
wrong, which makes it more difficult to code round the issue.

The issue comes down to a couple of lines in the html for the web page.
The following lines parse correctly:

<TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo1> Wind:</TD>
<TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo2> From the Northeast at 6&nbsp;mph</TD>

these don't:

<TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo1> Wind:</TD>
<TD ALIGN=LEFT VALIGN=TOP CLASS=obsInfo2> calm&nbsp;</TD>

the relevant portion of the python script is as follows:

print '============== =============== =============== ======='
p_current = r'''(?isx) # Ignore case, Dot matches all, Verbose
wxicons/52/(?P<icon>\d*?)\ .gif # Icon
..*?obsTempText A>(?P<temp>\d*? )&deg; # Temp
..*?obsTextA>(? P<sky>.*?)</b> # Sky
..*?Feels\sLike <br>(?P<heat>.* ?)&deg; # Heat
..*?UV\sIndex:. *?Info2>(?P<uv> .*?)&nbsp
..*?Dew\sPoint: .*?Info2>(?P<de w>.*?)&deg;
..*?Humidity:.* ?Info2>(?P<hum> \d+)
..*?Visibility: .+?Info2>(?P<vi s>.*?)</td>
..*?Pressure:.+ ?Info2>(?P<baro >.*?)\sinches\s and\s(?P<change >.*?)</td>
..*?Wind:.+?Inf o2>(?P<wind>.*? )\sat\s(?P<spee d>\d*?)&nbsp;
'''


match = re.search(p_cur rent, data1)
if match:
now.icon(match. group('icon'))
now.temperature (match.group('t emp'), 'F')
now.relative_he at(match.group( 'heat'), 'F')
now.sky(match.g roup('sky'))
now.uv(match.gr oup('uv'))
now.dewpoint(ma tch.group('dew' ), 'F')
now.humidity(ma tch.group('dew' ))
now.visibility( match.group('vi s'))
now.pressure(ma tch.group('baro '), 'inHg')
now.pressure_ch ange(match.grou p('change'))
mywind = match.group('wi nd')
now.wind(mywind .replace('From the ', ''))
now.wind_speed( match.group('sp eed'), 'mph')

Obviously the issue is that the regular expression expects "at", and in
the second line of the html that doesn't parse, there is no at.

The question I have, is how do I go about fixing this. What I want is to
test to see if the line does or doesn't contain an "at", and if not,
change it to contain an "at". I'm just not sure how to code the RE in
python to do this.

Any help would be appreciated.

Matt

[Sybren Stuvel]

Re: Help with regular expressions

dmbkiwi enlightened us with:[color=blue]
> A couple of other people have contributed code to this project,
> particularly relating to the parsing of the websites.
> Unfortunately, it is not parsing one particular part of the website
> properly. This is because it is expecting the data to be in a
> certain form, and occasionally it is in a different form.
> Unfortunately this causes the entire script to fail to run.[/color]

You seem to expect old HTML. Why not use XHTML only ('tidy' can
convert between them) and use a regular XML parser? Much, much, much
easier! And you won't have to be afraid of messing up your regular
expressions ;-)

Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?

[dmbkiwi]

Re: Help with regular expressions

On Tue, 26 Aug 2003 08:47:33 +0000, Sybren Stuvel wrote:
[color=blue]
> dmbkiwi enlightened us with:[color=green]
>> A couple of other people have contributed code to this project,
>> particularly relating to the parsing of the websites.
>> Unfortunately, it is not parsing one particular part of the website
>> properly. This is because it is expecting the data to be in a
>> certain form, and occasionally it is in a different form.
>> Unfortunately this causes the entire script to fail to run.[/color]
>
> You seem to expect old HTML. Why not use XHTML only ('tidy' can
> convert between them) and use a regular XML parser? Much, much, much
> easier! And you won't have to be afraid of messing up your regular
> expressions ;-)
>
> Sybren[/color]

XML would be nice, but unfortunately I have no choice as to the markup
language used by the site. It's a website on the world wide web, not a
site overwhich I have any control. My regular expressions are at the
mercy of the developers of that site.

Any other suggestions?

Matt

[John J. Lee]

Re: Help with regular expressions

dmbkiwi <dmbkiwi@yahoo. com> writes:[color=blue]
> On Tue, 26 Aug 2003 08:47:33 +0000, Sybren Stuvel wrote:[/color]
[...][color=blue][color=green]
> > You seem to expect old HTML. Why not use XHTML only ('tidy' can
> > convert between them) and use a regular XML parser? Much, much, much
> > easier! And you won't have to be afraid of messing up your regular
> > expressions ;-)
> >
> > Sybren[/color]
>
> XML would be nice, but unfortunately I have no choice as to the markup
> language used by the site. It's a website on the world wide web, not a
> site overwhich I have any control. My regular expressions are at the
> mercy of the developers of that site.[/color]

You misunderstand. HTMLTidy (or its descendant, tidylib) reads ugly,
non-conformant HTML and spits out clean, conformant XHTML (or HTML).

uTidylib is a ctypes wrapper of tidylib.

import tidy
from cStringIO import StringIO
tidydoc = tidy.parseStrin g(html)
s = StringIO()
tidydoc.write(s )
tidied_html = s.getvalue()


mxTidy is a wrapper of a shared-library-ized HTMLTidy.

from mx.Tidy import tidy
tidied_html = tidy(html)[2]


John