I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.
I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):
Set-Cookie: JSESSIONID=M1LH YKX2DVXRMCQBAFM L3UQ; domain=.tvland. com; path=/
Set-Cookie: JSESSIONID=0000 CSPVV3Q5TXU2BUP IIDEWOCY:ulnfn1 uq;Path=/
my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.
Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?
Thanks!
Hal
--------------------------------------------------------------------
Program listing:
use LWP::UserAgent;
use HTTP::Cookies;
our $domain = "tvland.com ";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozil la/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTT P::Cookies->new(file =>"lwpcookies.t xt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type(" application/x-www-form-urlencoded");
$req->header('Accept ' => 'text/html');
$res = $ua->request($req );
print "Cookie: ".$res->status_line."\ n";
$page = $res->as_string;
# print "Page: $page\n";
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.
I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):
Set-Cookie: JSESSIONID=M1LH YKX2DVXRMCQBAFM L3UQ; domain=.tvland. com; path=/
Set-Cookie: JSESSIONID=0000 CSPVV3Q5TXU2BUP IIDEWOCY:ulnfn1 uq;Path=/
my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.
Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?
Thanks!
Hal
--------------------------------------------------------------------
Program listing:
use LWP::UserAgent;
use HTTP::Cookies;
our $domain = "tvland.com ";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozil la/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTT P::Cookies->new(file =>"lwpcookies.t xt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type(" application/x-www-form-urlencoded");
$req->header('Accept ' => 'text/html');
$res = $ua->request($req );
print "Cookie: ".$res->status_line."\ n";
$page = $res->as_string;
# print "Page: $page\n";
Comment