Python Behind a Squid Corporate Proxy on Windows

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Larry Hale

    Python Behind a Squid Corporate Proxy on Windows

    Greetings, Pythonistas!

    My employer has a Squid Proxy between my Python programs and The
    Internet.

    I've searched high-and-low, and can only find examples online of how
    to do basic authentication to connect TO an external Web Server, but
    not how to go THROUGH a (corporate) Proxy, thus my question here.
    I've tried all sorts of permutations based on my findings, but
    (obviously) nothing's working for me. I've come across conflicting
    information, including that urllib2 can't/won't do what I'm needing,
    though this statement was a couple years old.

    I'm running Python 2.5 on Windows XP SP2 64-bit. The Proxy is Squid
    (don't know what version, but can find out if this is relevant).

    urllib2 grabs the Proxy info fine: 'http': 'http://webfilter.xyz.l ocal:
    3128' (.getproxies()) . There's the obvious necessity of telling
    "something" my ID/pass *somehow*.

    An example of one iteration:

    ############### #######

    import urllib2

    proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
    myusername:@web filter.xyz.loca l:3128" } )
    opener = urllib2.build_o pener( proxy_handler )

    urllib2.install _opener( opener )

    response = opener.open( "http://python.org" )

    print response

    ############### #######

    I've tried situations using HTTPPasswordMgr WithDefaultReal m and its
    add_password method, also to no avail. (Again, THIS sort of thing
    seems related to the -external- Web Server, _not_ going *through* a
    Proxy. 'Course, I could be wrong. But if I am, I couldn't get this
    to work, either. ;)

    I've looked at what's happening with Wireshark; Py/my program/urllib2
    makes the initial page request, Proxy replies HTTP Error 407: Proxy
    Authentication Required, and Py/my program/urllib2 simply stops at
    that, reporting the message/error.

    Please forgive me; I've been programming in Python for a few years
    now, but never had to deal with Web Proxies. I *have* tried to RTFMs,
    but the "documentat ion" for non-basic stuffs with urllib2 is... uh...
    well, I'm sure ye olde pro's know... :)


    Cheers, and *any* help would be appreciated!
    -Larry
  • Larry Hale

    #2
    Re: Python Behind a Squid Corporate Proxy on Windows

    Err, the line above should be:

    proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
    myusername:pass word@webfilter. xyz.local:3128" } )

    (Sorry! :)

    Comment

    • Chris

      #3
      Re: Python Behind a Squid Corporate Proxy on Windows

      On Jul 17, 6:40 pm, Larry Hale <larz...@hotmai l.comwrote:
      Err, the line above should be:
      >
      proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
      myusername:pass w...@webfilter. xyz.local:3128" } )
      >
      (Sorry!  :)
      some old code I wrote to download public domain info for our company,
      also through a squid proxy and this still works for me

      import urllib2, urllib

      proxy = urllib2.ProxyHa ndler({'http': 'http://
      username:passwo rd@proxy_url:pr oxy_port'})
      auth = urllib2.HTTPBas icAuthHandler()
      opener = urllib2.build_o pener(proxy, auth, urllib2.HTTPHan dler)
      urllib2.install _opener(opener)

      conn = urllib2.urlopen ('http://python.org')
      return_str = conn.read()

      fp = open('python.ht ml','w').write( return_str)

      Comment

      • Larry Hale

        #4
        Re: Python Behind a Squid Corporate Proxy on Windows

        Thank you so much for the reply, but alas: I get the same results.
        (urllib2 makes the initial request, Squid replies "Error 407: Proxy
        Authentication Required", which urllib2 dutifully fails on/raises said
        error...)

        I do appreciate your time, though! :)
        -Larry


        On Jul 18, 3:08 am, Chris <cwi...@gmail.c omwrote:
        On Jul 17, 6:40 pm, Larry Hale <larz...@hotmai l.comwrote:
        >
        Err, the line above should be:
        >
        proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
        myusername:pass w...@webfilter. xyz.local:3128" } )
        >
        (Sorry!  :)
        >
        some old code I wrote to download public domain info for our company,
        also through a squid proxy and this still works for me
        >
        import urllib2, urllib
        >
        proxy = urllib2.ProxyHa ndler({'http': 'http://
        username:passwo rd@proxy_url:pr oxy_port'})
        auth = urllib2.HTTPBas icAuthHandler()
        opener = urllib2.build_o pener(proxy, auth, urllib2.HTTPHan dler)
        urllib2.install _opener(opener)
        >
        conn = urllib2.urlopen ('http://python.org')
        return_str = conn.read()
        >
        fp = open('python.ht ml','w').write( return_str)

        Comment

        • Larry Hale

          #5
          Re: Python Behind a Squid Corporate Proxy on Windows

          Thank you so much for the reply, but alas: I get the same results.
          (urllib2 makes the initial request, Squid replies "Error 407: Proxy
          Authentication Required", which urllib2 dutifully fails on/raises said
          error...)

          I do appreciate your time, though! :)
          -Larry


          On Jul 18, 3:08 am, Chris <cwi...@gmail.c omwrote:
          On Jul 17, 6:40 pm, Larry Hale <larz...@hotmai l.comwrote:
          >
          Err, the line above should be:
          >
          proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
          myusername:pass w...@webfilter. xyz.local:3128" } )
          >
          (Sorry!  :)
          >
          some old code I wrote to download public domain info for our company,
          also through a squid proxy and this still works for me
          >
          import urllib2, urllib
          >
          proxy = urllib2.ProxyHa ndler({'http': 'http://
          username:passwo rd@proxy_url:pr oxy_port'})
          auth = urllib2.HTTPBas icAuthHandler()
          opener = urllib2.build_o pener(proxy, auth, urllib2.HTTPHan dler)
          urllib2.install _opener(opener)
          >
          conn = urllib2.urlopen ('http://python.org')
          return_str = conn.read()
          >
          fp = open('python.ht ml','w').write( return_str)

          Comment

          • Larry Hale

            #6
            Re: Python Behind a Squid Corporate Proxy on Windows

            Thank you so much for the reply, but alas: I get the same results.
            (urllib2 makes the initial request, Squid replies "Error 407: Proxy
            Authentication Required", which urllib2 dutifully fails on/raises said
            error...)

            I do appreciate your time, though! :)
            -Larry


            On Jul 18, 3:08 am, Chris <cwi...@gmail.c omwrote:
            On Jul 17, 6:40 pm, Larry Hale <larz...@hotmai l.comwrote:
            >
            Err, the line above should be:
            >
            proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
            myusername:pass w...@webfilter. xyz.local:3128" } )
            >
            (Sorry!  :)
            >
            some old code I wrote to download public domain info for our company,
            also through a squid proxy and this still works for me
            >
            import urllib2, urllib
            >
            proxy = urllib2.ProxyHa ndler({'http': 'http://
            username:passwo rd@proxy_url:pr oxy_port'})
            auth = urllib2.HTTPBas icAuthHandler()
            opener = urllib2.build_o pener(proxy, auth, urllib2.HTTPHan dler)
            urllib2.install _opener(opener)
            >
            conn = urllib2.urlopen ('http://python.org')
            return_str = conn.read()
            >
            fp = open('python.ht ml','w').write( return_str)

            Comment

            • Danilo

              #7
              Re: Python Behind a Squid Corporate Proxy on Windows

              Hi,

              do you manage to go through the proxy? I'm having the same problem
              with error 407. I've tried many solutions from the web but it always
              show the same error. So, if you solved your problem, please, post it
              here.

              Thanks
              Danilo

              On 18 jul, 15:43, Larry Hale <larz...@hotmai l.comwrote:
              Thank you so much for the reply, but alas: I get the same results.
              (urllib2 makes the initial request,Squidre plies "Error 407:Proxy
              Authentication Required", which urllib2 dutifully fails on/raises said
              error...)
              >
              I do appreciate your time, though! :)
              -Larry
              >
              On Jul 18, 3:08 am, Chris <cwi...@gmail.c omwrote:
              >
              On Jul 17, 6:40 pm, Larry Hale <larz...@hotmai l.comwrote:
              >
              Err, the line above should be:
              >
              proxy_handler = urllib2.ProxyHa ndler( { "http": "http://
              myusername:pass w...@webfilter. xyz.local:3128" } )
              >
              (Sorry! :)
              >
              some old code I wrote to download public domain info for our company,
              also through asquidproxyand this still works for me
              >
              import urllib2, urllib
              >
              proxy= urllib2.ProxyHa ndler({'http': 'http://
              username:passwo rd@proxy_url:pr oxy_port'})
              auth = urllib2.HTTPBas icAuthHandler()
              opener = urllib2.build_o pener(proxy, auth, urllib2.HTTPHan dler)
              urllib2.install _opener(opener)
              >
              conn = urllib2.urlopen ('http://python.org')
              return_str = conn.read()
              >
              fp = open('python.ht ml','w').write( return_str)

              Comment

              Working...