Parsing HTML

[mtuller]

I am trying to parse a webpage and extract information. I am trying to
use pyparser. Here is what I have:

from pyparsing import *
import urllib

# define basic text pattern
spanStart = Literal('<span class=\"hpPageT ext\">')

spanEnd = Literal('</span></td>')

printCount = spanStart + SkipTo(spanEnd) + spanEnd

# get printer addresses
printerURL = "http://printer.mydomai n.com/hp/device/this.LCDispatch er?
nav=hp.Usage"
printerListPage = urllib.urlopen( printerURL)
printerListHTML = printerListPage .read()
printerListPage .close

for srvrtokens,star tloc,endloc in
printCount.scan String(printerL istHTML): print srvrtokens

print printCount


I have the last print statement to check what is being sent because I
am getting nothing back. What it sends is:
{"<span class="hpPageTe xt">" SkipTo:("</span></td>") "</span></td>"}

If I pull out the "hpPageText " I get results back, but more than what
I want. I know it has something to do with escaping the quotation
marks, but I am puzzled as to how to do it.


Thanks,

Mike

[metaperl]

Re: Parsing HTML

On Feb 8, 2:38 pm, "mtuller" <mitul...@gmail .comwrote:

I am trying to parse a webpage and extract information.

BeautifulSoup is a great Python module for this purpose:

http://www.crummy.com/software/BeautifulSoup/

Here's an article on screen scraping using it:

http://iwiwdsmi.blogspot.com/2007/01/how-to-use-python-and-beautiful-soup-to.html

[mtuller]

Re: Parsing HTML

I was asking how to escape the quotation marks. I have everything
working in pyparser except for that. I don't want to drop everything
and go to a different parser.

Can someone else help?

>

I am trying to parse a webpage and extract information.

>
BeautifulSoup is a great Python module for this purpose:
>

http://www.crummy.com/software/BeautifulSoup/

>
Here's an article on screen scraping using it:
>
http://iwiwdsmi.blogspot.com/2007/01...and-beautiful-...

[Paul McGuire]

Re: Parsing HTML

On Feb 8, 4:15 pm, "mtuller" <mitul...@gmail .comwrote:

I was asking how to escape the quotation marks. I have everything
working in pyparser except for that. I don't want to drop everything
and go to a different parser.
>
Can someone else help?
>
>

Mike -

pyparsing includes a helper for constructing HTML tags called
makeHTMLTags. This method does more than just wrap the given tag text
within <>'s, but also comprehends attributes, upper/lower case, and
various styles of quoted strings. To use it, replace your Literal
definitions for spanStart and spanEnd with:

spanStart, spanEnd = makeHTMLTags('s pan')

If you don't want to match just *any* <spantag, but say, you only
want those with the class = "hpPageText ", then add this parse action
to spanStart:

def onlyAcceptWithT agAttr(attrname ,attrval):
def action(tagAttrs ):
if not(attrname in tagAttrs and tagAttrs[attrname]==attrval):
raise ParseException( "",0,"")
return action

spanStart.setPa rseAction(onlyA cceptWithTagAtt r("class","hpPa geText"))


-- Paul

[sofeng]

Re: Parsing HTML

On Feb 8, 11:43 am, "metaperl" <metap...@gmail .comwrote:

On Feb 8, 2:38 pm, "mtuller" <mitul...@gmail .comwrote:
>

I am trying to parse a webpage and extract information.

>
BeautifulSoup is a great Python module for this purpose:
>

http://www.crummy.com/software/BeautifulSoup/

>
Here's an article on screen scraping using it:
>
http://iwiwdsmi.blogspot.com/2007/01...and-beautiful-...

This article has moved to http://iwiwdsmp.blogspot.com/2007/02...l-soup-to.html

[John Nagle]

Re: Parsing HTML

BeautifulSoup does parse HTML well, but there are a few issues:

1. It's rather slow; it can take seconds of CPU time to parse
some larger web pages.

2. There's no error reporting. It tries to do the right thing,
but when it doesn't, you have no idea what went wrong.

BeautifulSoup would be a good test case for the PyPy crowd to
work on. It really needs the speedup.

John Nagle

sofeng wrote:

On Feb 8, 11:43 am, "metaperl" <metap...@gmail .comwrote:

>>On Feb 8, 2:38 pm, "mtuller" <mitul...@gmail .comwrote:

>>>I am trying to parse a webpage and extract information.

>>BeautifulSo up is a great Python module for this purpose: