Monitoring/intercepting http traffic for a particular site

[deepthi82]

Hi,

I'm new to Python and here is the task I want to execute using python:

I'm trying to monitor the http requests send by a client to a particular server
say xxx.com.
I'm interested in a particular URL which the client sends.
I don't want to make any explicit requests to the server but just want to
watch what all being is sent. so I want to establish a connection to the server
and then monitor/intercept the requests/response to/from the server over a certain amount of time. In a way trying to sniff over the network.
How can I use httplib methods to acheive this? Any help is appreciated.

[bartonc]

Sound like malicious mischief to me. Any other opinions or counter points?

[Motoma]

Hi,

I'm new to Python and here is the task I want to execute using python:

I'm trying to monitor the http requests send by a client to a particular server
say xxx.com.
I'm interested in a particular URL which the client sends.
I don't want to make any explicit requests to the server but just want to
watch what all being is sent. so I want to establish a connection to the server
and then monitor/intercept the requests/response to/from the server over a certain amount of time. In a way trying to sniff over the network.
How can I use httplib methods to acheive this? Any help is appreciated.

There are already programs to do this. Could you explain why you want to do it in Python?

[deepthi82]

There are already programs to do this. Could you explain why you want to do it in Python?

I need the script/program to work on all three platforms which our product supports: windows/linux and mac
so I wanted to do it in python.

[deepthi82]

I need the script/program to work on all three platforms which our product supports: windows/linux and mac
so I wanted to do it in python.

As of now, I'm using network sniffers to do this, but that requires manual intervention and automating this would releive me from doing this atleast twice a day across all three platforms.

[Motoma]

As of now, I'm using network sniffers to do this, but that requires manual intervention and automating this would releive me from doing this atleast twice a day across all three platforms.

tethereal was not sufficient for scripting with?

[deepthi82]

tethereal was not sufficient for scripting with?

I'm using fiddler to do this, but I have never did scripting with these tools
and I ideally want to do this just with Python (using the default libraries or some extension modules.) so its easy to maintain by any python-familiar person in the team. so I'm wondering if this can be done using urllib,httplib and socket modules?

[Motoma]

I'm using fiddler to do this, but I have never did scripting with these tools
and I ideally want to do this just with Python (using the default libraries or some extension modules.) so its easy to maintain by any python-familiar person in the team. so I'm wondering if this can be done using urllib,httplib and socket modules?

Yes, but you will need to use the pycap libraries as well.

Honestly though, tethereal provides amazing access (which could easily be scripted in python) to all of the sorts of things you are trying to do. Please do a little research before choosing the tool you use (we will have to answer your future questions after all ;))