Read webpage with VB app

[KC]

Is there a simple way to read (and parse) a webpage with a VB.net
application. Right now I can sort of read a page but it's a framed page and
the server thinks the app is a browser that can't handle frames...so I get a
message to the effect in the response.

Ken

[Herfried K. Wagner [MVP]]

Re: Read webpage with VB app

* "KC" <you@dontneed.t his> scripsit:[color=blue]
> Is there a simple way to read (and parse) a webpage with a VB.net
> application. Right now I can sort of read a page but it's a framed page and
> the server thinks the app is a browser that can't handle frames...so I get a
> message to the effect in the response.[/color]

You can load each frame's HTML page. This will require you to know the
filename or parse the frameset. Alternatively, you can place a
WebBrowser control on your form and use this code:

\\\
Me.WebBrowser1. Navigate("http://www.over-the-moon.org/dollz")
///

In the WebBrowser's 'DocumentComple te' event handler:

\\\
Dim i As Integer
For i = 0 To Me.WebBrowser1. Document.frames .length - 1
MsgBox(Me.WebBr owser1.Document .frames(i).Docu ment.documentEl ement.innerText )
Next i
///

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[Herfried K. Wagner [MVP]]

Re: Read webpage with VB app

* "KC" <you@dontneed.t his> scripsit:[color=blue]
> Is there a simple way to read (and parse) a webpage with a VB.net
> application. Right now I can sort of read a page but it's a framed page and
> the server thinks the app is a browser that can't handle frames...so I get a
> message to the effect in the response.[/color]

You can load each frame's HTML page. This will require you to know the
filename or parse the frameset. Alternatively, you can place a
WebBrowser control on your form and use this code:

\\\
Me.WebBrowser1. Navigate("http://www.over-the-moon.org/dollz")
///

In the WebBrowser's 'DocumentComple te' event handler:

\\\
Dim i As Integer
For i = 0 To Me.WebBrowser1. Document.frames .length - 1
MsgBox(Me.WebBr owser1.Document .frames(i).Docu ment.documentEl ement.innerText )
Next i
///

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[Cor Ligthert]

Re: Read webpage with VB app

Hi KC,

In addition to Herfried,

A little example,

Open a new windows application project

In the toolbox rightclick and select add/Remove items

In the customize toolbox select Com and in that Microsoft Webbrowser

When that is in the toolbox drag it to your form
Drag also a button to your form.

Then this code and you have a mini Webbrowser.

Private Sub Button1_Click(B yVal sender As System.Object, ByVal e As
System.EventArg s) Handles Button1.Click
Me.AxWebBrowser 1.Navigate2("ww w.google.com")
End Sub

webbrowser

http://support.microsoft.com/?kbid=311303

mshtml

http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/hosting/hosting.asp

I hope this helps a little bit?

Cor

[Chad Z. Hower aka Kudzu]

Re: Read webpage with VB app

"KC" <you@dontneed.t his> wrote in
news:#7l5f1pPEH A.3304@TK2MSFTN GP12.phx.gbl:[color=blue]
> Is there a simple way to read (and parse) a webpage with a VB.net
> application. Right now I can sort of read a page but it's a framed page
> and the server thinks the app is a browser that can't handle frames...so
> I get a message to the effect in the response.[/color]

It sounds like you just want to transfer it, and not display it? If so the
webbrower control is overkill and in fact can interfere. You should look at
just using straight HTTP.


--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Empower ASP.NET with IntraWeb

http://www.atozed.com/IntraWeb/

[Cor Ligthert]

Re: Read webpage with VB app

Hi Kudzu,
[color=blue]
> It sounds like you just want to transfer it, and not display it? If so the
> webbrower control is overkill and in fact can interfere. You should look[/color]
at[color=blue]
> just using straight HTTP.
>[/color]
Altough I do not disagree, are we not talking about ants on the Titanic. ;)

Cor

[Herfried K. Wagner [MVP]]

Re: Read webpage with VB app

* "Chad Z. Hower aka Kudzu" <cpub@hower.org > scripsit:[color=blue][color=green]
>> Is there a simple way to read (and parse) a webpage with a VB.net
>> application. Right now I can sort of read a page but it's a framed page
>> and the server thinks the app is a browser that can't handle frames...so
>> I get a message to the effect in the response.[/color]
>
> It sounds like you just want to transfer it, and not display it? If so the
> webbrower control is overkill and in fact can interfere. You should look at
> just using straight HTTP.[/color]

I agree, but then you will have to parse the frameset to get the URLs of
the pages displayed in the frames.

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[Chad Z. Hower aka Kudzu]

Re: Read webpage with VB app

"Cor Ligthert" <notfirstname@p lanet.nl> wrote in news:O6bHww9PEH A.640
@TK2MSFTNGP09.p hx.gbl:[color=blue][color=green]
>> It sounds like you just want to transfer it, and not display it? If so the
>> webbrower control is overkill and in fact can interfere. You should look[/color]
> at[color=green]
>> just using straight HTTP.
>>[/color]
> Altough I do not disagree, are we not talking about ants on the Titanic. ;)[/color]

Not in this case. WebBrowser doesnt just do HTTP, or simple parsing. It loads
TONS of stuff to render etc just to do this. We are not talking about ants on
the Titanic, but rather Ants VERSUS the Titanic. :)


--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Empower ASP.NET with IntraWeb

http://www.atozed.com/IntraWeb/

[Chad Z. Hower aka Kudzu]

Re: Read webpage with VB app

hirf-spam-me-here@gmx.at (Herfried K. Wagner [MVP]) wrote in
news:enWwHN$PEH A.3708@TK2MSFTN GP10.phx.gbl:[color=blue]
> I agree, but then you will have to parse the frameset to get the URLs of
> the pages displayed in the frames.[/color]

Yes, but thats quite a trivial task.


--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Make your ASP.NET applications run faster

http://www.atozed.com/IntraWeb/

[Cor Ligthert]

Re: Read webpage with VB app

Hi Kudzu,
[color=blue][color=green]
> > I agree, but then you will have to parse the frameset to get the URLs of
> > the pages displayed in the frames.[/color]
>
> Yes, but thats quite a trivial task.
>[/color]
That should be very easy when you use the logic from Mshmtl.

Something simple as let say (I did not test it, however it is not much
different))

If tagname = "iframe" Then
Url = DirectCast( DirectCast(iDoc ument.all.item( i), mshtml.IHTMLEle ment)
mshtml.IHTMLFra meBase).src.ToS tring
End If

:-))

Cor

[Herfried K. Wagner [MVP]]

Re: Read webpage with VB app

* "Chad Z. Hower aka Kudzu" <cpub@hower.org > scripsit:[color=blue][color=green]
>> I agree, but then you will have to parse the frameset to get the URLs of
>> the pages displayed in the frames.[/color]
>
> Yes, but thats quite a trivial task.[/color]

Mhm... but why implement it if it's already available through MSHTML?

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[Chad Z. Hower aka Kudzu]

Re: Read webpage with VB app

hirf-spam-me-here@gmx.at (Herfried K. Wagner [MVP]) wrote in
news:uCraQOAQEH A.2216@TK2MSFTN GP12.phx.gbl:[color=blue][color=green]
>> Yes, but thats quite a trivial task.[/color]
>
> Mhm... but why implement it if it's already available through MSHTML?[/color]

Because unless you are implementing just one instance in a client app,
dragging in WebBrowser will us significant resources. Its like loading up a
747 to go to the corner store for a loaf of bread.


--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Develop ASP.NET applications easier and in less time:

http://www.atozed.com/IntraWeb/

[Cor Ligthert]

Re: Read webpage with VB app

Hi Kudzu,

It becomes smaller, first the Titanic and now it is a 747.

:-)

Although I agree with you when the OP wants to integrate it in his program.
When he wants an basic approach he can use the webbrowser because it is so
easy to navigate with that.

Do not mix up MSHTML with the webbrowser, in my opinion that are different
things.
MSDN is not real clear about MSHTML.

You can load a doc in MSHTML and than process it using the DOM.
See for that the sample I showed in the other thread in this message.

Cor

[Herfried K. Wagner [MVP]]

Re: Read webpage with VB app

* "Chad Z. Hower aka Kudzu" <cpub@hower.org > scripsit:[color=blue][color=green]
>> Mhm... but why implement it if it's already available through MSHTML?[/color]
>
> Because unless you are implementing just one instance in a client app,
> dragging in WebBrowser will us significant resources. Its like loading up a
> 747 to go to the corner store for a loaf of bread.[/color]

Full ACK. Depends on if you are lazy or not...

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>

[Chad Z. Hower aka Kudzu]

Re: Read webpage with VB app

"Cor Ligthert" <notfirstname@p lanet.nl> wrote in news:uVfXAMCQEH A.1644
@TK2MSFTNGP09.p hx.gbl:[color=blue]
> It becomes smaller, first the Titanic and now it is a 747.[/color]

Different examples. :)
[color=blue]
> Although I agree with you when the OP wants to integrate it in his program.
> When he wants an basic approach he can use the webbrowser because it is so
> easy to navigate with that.[/color]

If he doesnt care about weight, or for that matter dependencies. In a client
app he's probably ok.
[color=blue]
> Do not mix up MSHTML with the webbrowser, in my opinion that are different
> things.
> MSDN is not real clear about MSHTML.[/color]

Yes, but they are quite interreliant, and he spoke of using WebBrowser to do
the fetching.
[color=blue]
> You can load a doc in MSHTML and than process it using the DOM.[/color]

But he will get it using WebBrowser. :)



--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programmin g is an art form that fights back"

Get your ASP.NET in gear with IntraWeb!

http://www.atozed.com/IntraWeb/