view page source or save after load

[zephron2000]

Hey,

I need to either:

1. View the page source of a webpage after it loads

or

2. Save the webpage to my computer after it loads (same as File Save
Page As)

urllib is not sufficient (using urlopen or something else in urllib
isn't going to do the trick)

Any ideas?

Thanks,
Lara

[James Stroud]

Re: view page source or save after load

zephron2000 wrote:

Hey,
>
I need to either:
>
1. View the page source of a webpage after it loads
>
or
>
2. Save the webpage to my computer after it loads (same as File Save
Page As)
>
urllib is not sufficient (using urlopen or something else in urllib
isn't going to do the trick)
>
Any ideas?
>
Thanks,
Lara
>
>
>

I happen to be tweaking a module that does this as your question came
in. The relevant lines are:

fetchparams = urllib.urlencod e(fetchparams)
wwwf = urllib.urlopen( "?".join([baseurl, fetchparams]))
afile = open(filename, "w")
afile.write(www f.read())
afile.close()

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/

[alex23]

Re: view page source or save after load

zephron2000 wrote:

I need to either:
1. View the page source of a webpage after it loads
or
2. Save the webpage to my computer after it loads (same as File Save
Page As)
urllib is not sufficient (using urlopen or something else in urllib
isn't going to do the trick)

You don't really say _why_ urllib.urlopen "isn't going to do the
trick". The following does what you've described:

import urllib
page = urllib.urlopen( 'http://some.address')
open('saved_pag e.txt','w').wri te(page).close( )

If you're needing to use a browser directly and you're running under
Windows, try the Internet Explorer Controller library, IEC:

import IEC
ie = IEC.IEControlle r()
ie.Navigate('ht tp://some.address')
page = ie.GetDocumentH TML()
open('saved_pag e.txt','w').wri te(page.encode( 'iso-8859-1')).close()

(You can grab IEC from http://www.mayukhbose.com/python/IEC/index.php)

Hope this helps.

-alex23

[Gabriel Genellina]

Re: view page source or save after load

At Thursday 21/9/2006 02:26, alex23 wrote:

>page = urllib.urlopen( 'http://some.address')

add .read() at the end

>open('saved_pa ge.txt','w').wr ite(page).close ()

write() does not return the file object, so this won't work; you have
to bind the file to a temporary variable to be able to close it.



Gabriel Genellina
Softlab SRL





_______________ _______________ _______________ _____
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!

http://www.yahoo.com.ar/respuestas

[alex23]

Re: view page source or save after load


Gabriel Genellina wrote:
<fixes for my stupidity>

Thanks for the corrections, Gabriel. I really need to learn to
cut&paste working code :)

Cheers.

-alex23

[James Stroud]

Re: view page source or save after load

Gabriel Genellina wrote:

At Thursday 21/9/2006 02:26, alex23 wrote:
>

>page = urllib.urlopen( 'http://some.address')

>
add .read() at the end
>

>open('saved_pa ge.txt','w').wr ite(page).close ()

>
write() does not return the file object, so this won't work; you have to
bind the file to a temporary variable to be able to close it.

Strictly speaking, "have to" is not perfectly correct. The ".close()"
part can simply be eliminated as the file should close via garbage
collection once leaving the local namespace.

James