Text over multiple lines

[Rigga]

Hi,

I am using the HTMLParser to parse a web page, part of the routine I need
to write (I am new to Python) involves looking for a particular tag and
once I know the start and the end of the tag then to assign all the data
in between the tags to a variable, this is easy if the tag starts and ends
on the same line however how would I go about doing it if its split over
two or more lines?

Thanks

R

[Pawel Kraszewski]

Re: Text over multiple lines

Rigga wrote:
[color=blue]
> I am using the HTMLParser to parse a web page, part of the routine I need
> to write (I am new to Python) involves looking for a particular tag and
> once I know the start and the end of the tag then to assign all the data
> in between the tags to a variable, this is easy if the tag starts and ends
> on the same line however how would I go about doing it if its split over
> two or more lines?[/color]

Perhaps you should glue the whole text as a one long line?

--
Pawel Kraszewski FreeBSD/Linux

E-Mail/Jabber Phone ICQ GG
Pawel_Kraszewsk i@wp.pl +48 604 777447 45615564 69381

[Nigel Rowe]

Re: Text over multiple lines

Rigga wrote:
[color=blue]
> Hi,
>
> I am using the HTMLParser to parse a web page, part of the routine I need
> to write (I am new to Python) involves looking for a particular tag and
> once I know the start and the end of the tag then to assign all the data
> in between the tags to a variable, this is easy if the tag starts and ends
> on the same line however how would I go about doing it if its split over
> two or more lines?
>
> Thanks
>
> R[/color]

Don't re-invent the wheel,

http://www.crummy.com/software/BeautifulSoup/

--
Nigel Rowe
A pox upon the spammers that make me write my address like..
rho (snail) swiftdsl (stop) com (stop) au

[Rigga]

Re: Text over multiple lines

On Sun, 20 Jun 2004 21:03:34 +1000, Nigel Rowe wrote:
[color=blue]
> Rigga wrote:
>[color=green]
>> Hi,
>>
>> I am using the HTMLParser to parse a web page, part of the routine I need
>> to write (I am new to Python) involves looking for a particular tag and
>> once I know the start and the end of the tag then to assign all the data
>> in between the tags to a variable, this is easy if the tag starts and ends
>> on the same line however how would I go about doing it if its split over
>> two or more lines?
>>
>> Thanks
>>
>> R[/color]
>
> Don't re-invent the wheel,
> http://www.crummy.com/software/BeautifulSoup/[/color]
I want to do it manually as it will help with my understanding of Python,
any ideas how I go about it?

[Diez B. Roggisch]

Re: Text over multiple lines

Rigga wrote:
[color=blue]
> Hi,
>
> I am using the HTMLParser to parse a web page, part of the routine I need
> to write (I am new to Python) involves looking for a particular tag and
> once I know the start and the end of the tag then to assign all the data
> in between the tags to a variable, this is easy if the tag starts and ends
> on the same line however how would I go about doing it if its split over
> two or more lines?[/color]


What difference does it make that the text is spread over more than one
line? Just collect the data in handle_data.

--
Regards,

Diez B. Roggisch

[Peter Hansen]

Re: Text over multiple lines

Rigga wrote:
[color=blue]
> On Sun, 20 Jun 2004 21:03:34 +1000, Nigel Rowe wrote:[color=green]
>>Don't re-invent the wheel,
>>http://www.crummy.com/software/BeautifulSoup/[/color]
>
> I want to do it manually as it will help with my understanding of Python,
> any ideas how I go about it?[/color]

Wouldn't it help you a lot more then, to figure it out on your
own? ;-)

(If you are really looking for help improving your understanding
of Python, reading the source for BeautifulSoup and figuring out
how *it* does it will probably get you farther than figuring it
out yourself. If, on the other hand, it's a better understanding
of *programming* that you are after, then doing it yourself is
the best bet... IMHO )

-Peter

[John Roth]

Re: Text over multiple lines

"Rigga" <Rigga@hasnomai l.com> wrote in message
news:pan.2004.0 6.20.09.56.28.5 59657@hasnomail .com...[color=blue]
> Hi,
>
> I am using the HTMLParser to parse a web page, part of the routine I need
> to write (I am new to Python) involves looking for a particular tag and
> once I know the start and the end of the tag then to assign all the data
> in between the tags to a variable, this is easy if the tag starts and ends
> on the same line however how would I go about doing it if its split over
> two or more lines?
>
> Thanks[/color]

Depending on exactly what I want to do, I frequently use <file>.read()
to pick up the entire file in one string, rather than <file>.readline s() to
create a list of strings. It works quite well when what I need to do
can be served by regexs (which is not always the case.)

John Roth[color=blue]
>
> R[/color]

[Nelson Minar]

Re: Text over multiple lines

Rigga <Rigga@hasnomai l.com> writes:[color=blue]
> I am using the HTMLParser to parse a web page, part of the routine I need
> to write (I am new to Python) involves looking for a particular tag and
> once I know the start and the end of the tag then to assign all the data
> in between the tags to a variable, this is easy if the tag starts and ends
> on the same line however how would I go about doing it if its split over
> two or more lines?[/color]

I often have variants of this problem too. The simplest way to make it
work is to read all the HTML in at once with a single call to
file.read(), and then use a regular expression. Note that you probably
don't need re.MULTILINE, although you should take a look at what it
means in the docs just to know.

This works fine as long as you expect your files to be relatively
small (under a meg or so).

[Rigga]

Re: Text over multiple lines

On Sun, 20 Jun 2004 17:22:53 +0000, Nelson Minar wrote:
[color=blue]
> Rigga <Rigga@hasnomai l.com> writes:[color=green]
>> I am using the HTMLParser to parse a web page, part of the routine I need
>> to write (I am new to Python) involves looking for a particular tag and
>> once I know the start and the end of the tag then to assign all the data
>> in between the tags to a variable, this is easy if the tag starts and ends
>> on the same line however how would I go about doing it if its split over
>> two or more lines?[/color]
>
> I often have variants of this problem too. The simplest way to make it
> work is to read all the HTML in at once with a single call to
> file.read(), and then use a regular expression. Note that you probably
> don't need re.MULTILINE, although you should take a look at what it
> means in the docs just to know.
>
> This works fine as long as you expect your files to be relatively
> small (under a meg or so).[/color]

Im reading the entire file in to a variable at the moment and passing it
through HTMLParser. I have ran in to another problem that I am having a
hard time working out, my data is in this format:

<TD><SPAN class=qv id=EmployeeNo
title="Employee Number">123456</SPAN></TD></TR>

Some times the data is spread over 3 lines like:

<TD><SPAN class=qv id=BusinessName
title="Business Name">Some Shady Business
Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>

The data I need to get is the data enclosed in quotes after the word
title= and data after the > and before the </SPAN, in the case aove would
be: Some Shady Business
Group Ltd.

Running the file through HTMLParser I discovered that the title= part
and the data part I need is contained in a list therefore I have done this:

snippet of my code:

class MyHTMLParser(HT MLParser):

def handle_starttag (self, tag, attrs):
print "Encountere d the beginning of a %s tag" % tag

def handle_data(sel f, data):
if "title=" in data:
print "found title"

However I can not work out how to search through the data (which is in a
list) to pull out the data I need.

Sorry if this is a dumb question but hey I am learning!

Many thanks

Rigga

[William Park]

Re: Text over multiple lines

Rigga <Rigga@hasnomai l.com> wrote:[color=blue]
> On Sun, 20 Jun 2004 17:22:53 +0000, Nelson Minar wrote:
>[color=green]
> > Rigga <Rigga@hasnomai l.com> writes:[color=darkred]
> >> I am using the HTMLParser to parse a web page, part of the routine
> >> I need to write (I am new to Python) involves looking for a
> >> particular tag and once I know the start and the end of the tag
> >> then to assign all the data in between the tags to a variable, this
> >> is easy if the tag starts and ends on the same line however how
> >> would I go about doing it if its split over two or more lines?[/color]
> >
> > I often have variants of this problem too. The simplest way to make
> > it work is to read all the HTML in at once with a single call to
> > file.read(), and then use a regular expression. Note that you
> > probably don't need re.MULTILINE, although you should take a look at
> > what it means in the docs just to know.
> >
> > This works fine as long as you expect your files to be relatively
> > small (under a meg or so).[/color]
>
> Im reading the entire file in to a variable at the moment and passing
> it through HTMLParser. I have ran in to another problem that I am
> having a hard time working out, my data is in this format:
>
> <TD><SPAN class=qv id=EmployeeNo
> title="Employee Number">123456</SPAN></TD></TR>
>
> Some times the data is spread over 3 lines like:
>
> <TD><SPAN class=qv id=BusinessName
> title="Business Name">Some Shady Business
> Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>
>
> The data I need to get is the data enclosed in quotes after the word
> title= and data after the > and before the </SPAN, in the case aove
> would be: Some Shady Business Group Ltd.[/color]

Approach:

1. Extract '<SPAN ([^>]*)>([^<]*)</SPAN>' which is

<SPAN class=qv id=BusinessName
title="Business Name">Some Shady Business
Group Ltd.</SPAN>

with parenthized groups giving

submatch[1]='class=qv id=BusinessName \ntitle="Busine ss Name"'
submatch[2]='Some Shady Business\nGroup Ltd.'

2. Split submatch[1] into

class=qv
id=BusinessName
title="Business Name"

Homework:

Write a Python script.

Bash solution:

First, you need my patched Bash which can be found at

http://freshmeat.net/projects/bashdiff/

You need to patch the Bash shell, and compile. It has many Python
features, particularly regex and array. Shell solution is

text='<TD><SPAN class=qv id=BusinessName
title="Business Name">Some Shady Business
Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>'

newf () { # Usage: newf match submatch1 submatch2
eval $2 # --> class, id, title
echo $title > title
echo $3 > name
}
x=()
array -e '<SPAN ([^>]*)>([^<]*)</SPAN>' -E newf x "$text"
cat title
cat name

I can explain the steps, that it's rather long. :-)

--
William Park, Open Geometry Consulting, <opengeometry@y ahoo.ca>
No, I will not fix your computer! I'll reformat your harddisk, though.

[Rigga]

Re: Text over multiple lines

On Mon, 21 Jun 2004 05:06:50 +0000, William Park wrote:
[color=blue]
> Rigga <Rigga@hasnomai l.com> wrote:[color=green]
>> On Sun, 20 Jun 2004 17:22:53 +0000, Nelson Minar wrote:
>>[color=darkred]
>> > Rigga <Rigga@hasnomai l.com> writes:
>> >> I am using the HTMLParser to parse a web page, part of the routine
>> >> I need to write (I am new to Python) involves looking for a
>> >> particular tag and once I know the start and the end of the tag
>> >> then to assign all the data in between the tags to a variable, this
>> >> is easy if the tag starts and ends on the same line however how
>> >> would I go about doing it if its split over two or more lines?
>> >
>> > I often have variants of this problem too. The simplest way to make
>> > it work is to read all the HTML in at once with a single call to
>> > file.read(), and then use a regular expression. Note that you
>> > probably don't need re.MULTILINE, although you should take a look at
>> > what it means in the docs just to know.
>> >
>> > This works fine as long as you expect your files to be relatively
>> > small (under a meg or so).[/color]
>>
>> Im reading the entire file in to a variable at the moment and passing
>> it through HTMLParser. I have ran in to another problem that I am
>> having a hard time working out, my data is in this format:
>>
>> <TD><SPAN class=qv id=EmployeeNo
>> title="Employee Number">123456</SPAN></TD></TR>
>>
>> Some times the data is spread over 3 lines like:
>>
>> <TD><SPAN class=qv id=BusinessName
>> title="Business Name">Some Shady Business
>> Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>
>>
>> The data I need to get is the data enclosed in quotes after the word
>> title= and data after the > and before the </SPAN, in the case aove
>> would be: Some Shady Business Group Ltd.[/color]
>
> Approach:
>
> 1. Extract '<SPAN ([^>]*)>([^<]*)</SPAN>' which is
>
> <SPAN class=qv id=BusinessName
> title="Business Name">Some Shady Business
> Group Ltd.</SPAN>
>
> with parenthized groups giving
>
> submatch[1]='class=qv id=BusinessName \ntitle="Busine ss Name"'
> submatch[2]='Some Shady Business\nGroup Ltd.'
>
> 2. Split submatch[1] into
>
> class=qv
> id=BusinessName
> title="Business Name"
>
> Homework:
>
> Write a Python script.
>
> Bash solution:
>
> First, you need my patched Bash which can be found at
>
> http://freshmeat.net/projects/bashdiff/
>
> You need to patch the Bash shell, and compile. It has many Python
> features, particularly regex and array. Shell solution is
>
> text='<TD><SPAN class=qv id=BusinessName
> title="Business Name">Some Shady Business
> Group Ltd.</SPAN></TD></TR></TBODY></TABLE></TD></TR>'
>
> newf () { # Usage: newf match submatch1 submatch2
> eval $2 # --> class, id, title
> echo $title > title
> echo $3 > name
> }
> x=()
> array -e '<SPAN ([^>]*)>([^<]*)</SPAN>' -E newf x "$text"
> cat title
> cat name
>
> I can explain the steps, that it's rather long. :-)[/color]

Thanks for everyones help, I have now worked out a way that works for me
, your input has helped me immensley.

many thanks

R