Managing multiple HTTP requests on same session

[idoha]

Hi,

I am trying to write a script that simulate a nevigation on a particular web site.
Before I am getting the page that contains the information I need, I have to go through a series of preceding pages.

The page flow is as follow: login page -> page1 -> page2 ..etc.

The problem is that once I am doing the login to the webside and requesting for the next page, the web server doesn't recongnize me anymore. The response I get is that the login has expired. Therefore I am not getting the actual html on this page and I cannot move forward to the next page in the flow.

I guess that the web server doesn't recognize two subsequent requests as coming from the same session.

I tried to use a few packages of perl: LWP:UserAgent, POE:Component:C lient:HTTP. Unfortunately, I couldn't solve that.

Do you have any idea ?

Thanks,
Ido.

[KevinADC]

I am pretty sure WWW::Mechanize can do this as long as there is no javascript invloved. Read the documentation and read any FAQS or Cookbooks that are also posted/linked to on CPAN. I can not help with WWW::Mechanize usage as I have no experience with the module.

[Kelicula]

Are you saving the cookies, that the remote site is sending you, and sending them back?
More than likely the remote site has sent you a cookie after the "login" page, which you're script will have to save and send back to the server, on all future request, just like a regular browser.

Here is an example server/client comm. with cookies.


[code=text]
client:
GET www.somepage.co m HTTP/1.1
Host: localhost
Accept: image/gif, image/x-bitmap, image/jpg, */*
Connection: Keep-Alive

Server:
HTTP/1.1 200 OK
Date: Sat, 18 Mar 2008 20:35:35 GMT
Server: Apache/1.3.9 (Unix)
Set-Cookie: user_id=123456; path=/; expires=Sat, 18-Mar-05 19:06:19 GMT


Then on all future request, client should echo:
GET /somepage.html HTTP1.1
Host: localhost
Accept: image/gif, image/jpeg, */*
Connection: Keep-Alive
Cookie: user_id=123456


[/code]