Combining Regular Expressions

[Andrew Dixon - Depictions.net]

Hi Everyone.

I have been working on some code that strips the HTML code out of an HTML
page leaving just the text on the page. At the moment this is what I have:

// Strip all tags
replacePattern = "<(.|\n)+?> ";
pageHTML = pageHTML.replac eAll(replacePat tern,"");

//Remove any HTML specific characters (e.g. &quot; or &amp;)
replacePattern = "&(.|\n)+?; ";
pageHTML = pageHTML.replac eAll(replacePat tern,"");

// Remove whitespace
replacePattern = "\\s{2,}";
pageHTML = pageHTML.replac eAll(replacePat tern," ");

Is there a way I can combine all four patterns into one expression so I can
make the code more efficient? I've not really worked with RegEx so any
advice would be most welcome. Can I do something like:

replacePattern = "[<(.|\n)+?>][&(.|\n)+?;][\\s{2,}]";
pageHTML = pageHTML.replac eAll(replacePat tern,"");

Thanks.
--

Best Regards
[color=blue][color=green][color=darkred]
>>> Andrew Dixon[/color][/color][/color]

[Chris]

Re: Combining Regular Expressions

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Dixon - Depictions.net wrote:
[color=blue]
> Hi Everyone.
>
> I have been working on some code that strips the HTML code out of an
> HTML page leaving just the text on the page. At the moment this is
> what I have:
>
> // Strip all tags
> replacePattern = "<(.|\n)+?> ";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> //Remove any HTML specific characters (e.g. &quot; or &amp;)
> replacePattern = "&(.|\n)+?; ";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> // Remove whitespace
> replacePattern = "\\s{2,}";
> pageHTML = pageHTML.replac eAll(replacePat tern," ");
>
> Is there a way I can combine all four patterns into one expression
> so I can make the code more efficient? I've not really worked with
> RegEx so any advice would be most welcome. Can I do something like:
>
> replacePattern = "[<(.|\n)+?>][&(.|\n)+?;][\\s{2,}]";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> Thanks.[/color]

Hi,
I'm not that familiar with regular expressions myself, but according
to the documentation, you should be able to use something like this:

(<(.|\n)+?>)|(& (.|\n)+?;)

to match either of the first two items in your list (tags and
entities). The whitespace thing, I would recommend keeping a separate
operation, since you really want to replace each block of whitespace
with " ", but you want to replace tags and entities with "". Also,
isn't it easier to use "\\s+" than "\\s{2,}" for whitespace? Finally,
another point to ponder: using your whitespace replacement system
will put the entire output on one line. I'd think about changing your
expression if you want to keep linebreaks where they are instead of
turning them into spaces.

- --
Chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/8xNbwxczzJRavJY RAlHQAJ9kY1USFt v36iInWnR0v6hqT L0PzwCZAYZ2
gILykD4bpg2T8Io/eZJ+M1Q=
=DX+T
-----END PGP SIGNATURE-----

[hiwa]

Re: Combining Regular Expressions

"Andrew Dixon - Depictions.net" <andrew.dixon@N OREPLY.depictio ns.net> wrote in message news:<V_DIb.290 8$nB3.27114715@ news-text.cableinet. net>...[color=blue]
> Hi Everyone.
>
> I have been working on some code that strips the HTML code out of an HTML
> page leaving just the text on the page. At the moment this is what I have:
>
> // Strip all tags
> replacePattern = "<(.|\n)+?> ";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> //Remove any HTML specific characters (e.g. &quot; or &amp;)
> replacePattern = "&(.|\n)+?; ";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> // Remove whitespace
> replacePattern = "\\s{2,}";
> pageHTML = pageHTML.replac eAll(replacePat tern," ");
>
> Is there a way I can combine all four patterns into one expression so I can
> make the code more efficient? I've not really worked with RegEx so any
> advice would be most welcome. Can I do something like:
>
> replacePattern = "[<(.|\n)+?>][&(.|\n)+?;][\\s{2,}]";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> Thanks.[/color]

Java regular expressions can be combined with the '|' operator. But if
your objective is retrieving text from html documents, you can
effectively use HTMLEditorKit.P arserCallback#h andleText() method.

[Tony Morris]

Re: Combining Regular Expressions

You might want to look at http://htmlparser.sourceforge.net

--
Tony Morris
(BInfTech, Cert 3 I.T., SCJP[1.4], SCJD)
Software Engineer
IBM Australia - Tivoli Security Software

"Andrew Dixon - Depictions.net" <andrew.dixon@N OREPLY.depictio ns.net> wrote
in message news:V_DIb.2908 $nB3.27114715@n ews-text.cableinet. net...[color=blue]
> Hi Everyone.
>
> I have been working on some code that strips the HTML code out of an HTML
> page leaving just the text on the page. At the moment this is what I have:
>
> // Strip all tags
> replacePattern = "<(.|\n)+?> ";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> file://Remove any HTML specific characters (e.g. &quot; or &amp;)
> replacePattern = "&(.|\n)+?; ";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> // Remove whitespace
> replacePattern = "\\s{2,}";
> pageHTML = pageHTML.replac eAll(replacePat tern," ");
>
> Is there a way I can combine all four patterns into one expression so I[/color]
can[color=blue]
> make the code more efficient? I've not really worked with RegEx so any
> advice would be most welcome. Can I do something like:
>
> replacePattern = "[<(.|\n)+?>][&(.|\n)+?;][\\s{2,}]";
> pageHTML = pageHTML.replac eAll(replacePat tern,"");
>
> Thanks.
> --
>
> Best Regards
>[color=green][color=darkred]
> >>> Andrew Dixon[/color][/color]
>
>[/color]

[Andrew Dixon - Depictions.net]

Re: Combining Regular Expressions

Hi Chris.

Thanks. The reason I used \\s{2,} is so that it only replaces where there is
2 spaces or greater so that it doesn't all end up as one great long string
with any spaces which is what \\s+ would do.

--

Best Regards
[color=blue][color=green][color=darkred]
>>> Andrew Dixon[/color][/color][/color]

www.depictions.net - Sell your photographs online and set your own price.
"Chris" <chris2k01@hotm ail.com> wrote in message
news:b4HIb.1053 01$ss5.27559@cl grps13...[color=blue]
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Andrew Dixon - Depictions.net wrote:
>[color=green]
> > Hi Everyone.
> >
> > I have been working on some code that strips the HTML code out of an
> > HTML page leaving just the text on the page. At the moment this is
> > what I have:
> >
> > // Strip all tags
> > replacePattern = "<(.|\n)+?> ";
> > pageHTML = pageHTML.replac eAll(replacePat tern,"");
> >
> > //Remove any HTML specific characters (e.g. &quot; or &amp;)
> > replacePattern = "&(.|\n)+?; ";
> > pageHTML = pageHTML.replac eAll(replacePat tern,"");
> >
> > // Remove whitespace
> > replacePattern = "\\s{2,}";
> > pageHTML = pageHTML.replac eAll(replacePat tern," ");
> >
> > Is there a way I can combine all four patterns into one expression
> > so I can make the code more efficient? I've not really worked with
> > RegEx so any advice would be most welcome. Can I do something like:
> >
> > replacePattern = "[<(.|\n)+?>][&(.|\n)+?;][\\s{2,}]";
> > pageHTML = pageHTML.replac eAll(replacePat tern,"");
> >
> > Thanks.[/color]
>
> Hi,
> I'm not that familiar with regular expressions myself, but according
> to the documentation, you should be able to use something like this:
>
> (<(.|\n)+?>)|(& (.|\n)+?;)
>
> to match either of the first two items in your list (tags and
> entities). The whitespace thing, I would recommend keeping a separate
> operation, since you really want to replace each block of whitespace
> with " ", but you want to replace tags and entities with "". Also,
> isn't it easier to use "\\s+" than "\\s{2,}" for whitespace? Finally,
> another point to ponder: using your whitespace replacement system
> will put the entire output on one line. I'd think about changing your
> expression if you want to keep linebreaks where they are instead of
> turning them into spaces.
>
> - --
> Chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.2 (GNU/Linux)
>
> iD8DBQE/8xNbwxczzJRavJY RAlHQAJ9kY1USFt v36iInWnR0v6hqT L0PzwCZAYZ2
> gILykD4bpg2T8Io/eZJ+M1Q=
> =DX+T
> -----END PGP SIGNATURE-----[/color]

[Chris]

Re: Combining Regular Expressions

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Dixon - Depictions.net wrote:
[color=blue]
> Hi Chris.
>
> Thanks. The reason I used \\s{2,} is so that it only replaces where
> there is 2 spaces or greater so that it doesn't all end up as one
> great long string with any spaces which is what \\s+ would do.
>[/color]

Hi,
First, I have to assume you mean "without any spaces" rather than
"with any spaces".

If one of these is not true, then I'm afraid I don't quite understand
the question.

If they are, however, then I think we misunderstand each other. My
idea was to replace "\\s+" with " ". This would take any string of
one space or more and replace it with one space - which makes sense.
Replacing one space with one space is still one space. I think you
thought I meant to replace it with "", which would indeed remove all
spaces. However, *your* example is also slightly flawed, assuming
your replacement string is "", in that where there are 2 or more
spaces, it will take all of them and remove them entirely, rather
than replacing them with one space. If, on the other hand, your
replacement string is " ", then both examples work equally well, but
my regexp is shorter than yours :) Largely academic at this point
though.

- --
Chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/82s7wxczzJRavJY RAiseAJ9ah2iajb VZRGYQ6szYmkNNA isAHgCgmUm0
Fwp7qzP8SFWauv/EH3kxH6U=
=suPx
-----END PGP SIGNATURE-----