Urllib2 upgrade is breaking urlopen()

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Jay Davis

    Urllib2 upgrade is breaking urlopen()

    We have a lot of simple code using 'urllib', that basically
    just goes out and opens and read()'s some http url. Very basic,
    ten line scripts, and they work fine with urllib.

    We want to use urllib2 for everything, though, but when we
    do we get:

    page=urllib2.ur lopen("http://www.something.c om/whatever.php")
    ....
    File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
    File "/usr/lib/python2.3/urllib2.py", line 463, in http_error_302
    self.inf_msg + msg, headers, fp)
    urllib2.HTTPErr or: HTTP Error 302: The HTTP server returned a redirect
    error that would lead to an infinite loop.
    The last 30x error message was:
    Found


    These redirect errors just don't happen with urrlib.urlopen( ), and I
    can
    only find a few posts on deja that even mention this problem. Is
    there a known workaround?

    Thanks,

    J.D.
  • John J. Lee

    #2
    Re: Urllib2 upgrade is breaking urlopen()

    dj00302003@yaho o.com (Jay Davis) writes:
    [color=blue]
    > We have a lot of simple code using 'urllib', that basically
    > just goes out and opens and read()'s some http url. Very basic,
    > ten line scripts, and they work fine with urllib.
    >
    > We want to use urllib2 for everything, though, but when we
    > do we get:[/color]
    [...][color=blue]
    > urllib2.HTTPErr or: HTTP Error 302: The HTTP server returned a redirect
    > error that would lead to an infinite loop.
    > The last 30x error message was:
    > Found[/color]

    Which version of Python are you using?

    It might also help if you can give a publicly-accessible URL that
    triggers the problem (mail it to me privately if you prefer).


    John

    Comment

    • John J. Lee

      #3
      Re: Urllib2 upgrade is breaking urlopen()

      jjl@pobox.com (John J. Lee) writes:
      [color=blue]
      > dj00302003@yaho o.com (Jay Davis) writes:
      >[color=green]
      > > We have a lot of simple code using 'urllib', that basically
      > > just goes out and opens and read()'s some http url. Very basic,
      > > ten line scripts, and they work fine with urllib.
      > >
      > > We want to use urllib2 for everything, though, but when we
      > > do we get:[/color]
      > [...][color=green]
      > > urllib2.HTTPErr or: HTTP Error 302: The HTTP server returned a redirect
      > > error that would lead to an infinite loop.
      > > The last 30x error message was:
      > > Found[/color]
      >
      > Which version of Python are you using?
      >
      > It might also help if you can give a publicly-accessible URL that
      > triggers the problem (mail it to me privately if you prefer).[/color]

      Jay sent me a URL, and it turns out both urllib and urllib2 are
      functioning as intended. There's a 302 loop caused by the server
      expecting cookies to get returned. urllib goes into a loop until it
      notices it's redirected 10 times, then just returns the 302 as a
      normal response (which doesn't seem very nice, but appears to be what
      was intended). urllib2 detects the loop and returns the 302 as an
      HTTPError exception (which can be caught and .read() if necessary).


      John

      Comment

      Working...