Re: urllib accept-language doesn't have any effect

[Martin Bachwerk]

Hmm, thanks for the ideas,

I've checked the requests in Firefox one more time after deleting all
the cookies and both google.com and gizmodo.com do indeed forward me to
the German site without caring about the browser settings.

wget shows me that the server does a 302 redirect straight away.. soo..

Is there a way to pretend I'm from the US and not from wherever else? I
tried setting the X_FORWARDED_FOR header, but that doesn't seem to
work.. There must be some way to override the client IP, or?

Thanks again.
Martin

>
On Oct 15, 2008, at 9:50 AM, Martin Bachwerk wrote:
>

>Hello,
>>
>I'm trying to load a couple of pages using the urllib2 module. The
>problem is that I live in Germany and some sites seem to look at the
>IP of the client and forward him to a localized page.. Here's an
>example of the code, how I want to access google.com main english
>page, but get German instead. (For those of you who live in US, you
>will probably get correct results.. try emulating with 'fr' in
>accepted languages or something)
>>
>opener = urllib2.build_o pener()
>opener.addhead ers = [('Host', 'www.google.com '),
>('Accept-Language','en-gb,en;q=0.5'), ('User-agent', 'Mozilla/5.0
>(Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.1) Gecko/2008070208
>Firefox/3.0.1')]
>webfile = opener.open(url )

>
Martin,
It looks to me like what you're sending is correct. Debugging
suggestions --
>
- Set up a Web server on 127.0.0.1 and see what that server receives
when your Python code connects to it. Maybe you're not sending quite
what you think.
- Try emulating your Python code with wget or a similar command line
tool that lets you set headers.
- Sniff the conversation you're having with google using Wireshark.
Maybe you're getting redirected by the remote server.
>
Good luck
Philip
>

[Diez B. Roggisch]

Re: urllib accept-language doesn't have any effect

Martin Bachwerk wrote:

Hmm, thanks for the ideas,
>
I've checked the requests in Firefox one more time after deleting all
the cookies and both google.com and gizmodo.com do indeed forward me to
the German site without caring about the browser settings.
>
wget shows me that the server does a 302 redirect straight away.. soo..
>
Is there a way to pretend I'm from the US and not from wherever else? I
tried setting the X_FORWARDED_FOR header, but that doesn't seem to
work.. There must be some way to override the client IP, or?

No. Not within application code, that is. You can of course try and use e.g.
TOR or other anonymization proxies, or brush up your m4d hacking sk1llz and
learn a bit or two about IP-spoofing.

Diez