Client side network programming

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Roysun_rohit
    #1

    Client side network programming

    I am interested in making a search engine which takes web sites
    iteratively, and downloads the web page or has to perform some search
    across the web pages.
    I am unsucessful to do so. My machine works through a proxy server and the
    internet connection is through 24 hour lease line. when ever i try the
    code it gives :

    Traceback (innermost last):
    File "http-getfile-urllib2.py", line 19, in ?
    urllib.urlretri eve(remoteaddr, localname)
    File "/usr/lib/python1.5/urllib.py", line 66, in urlretrieve
    return _urlopener.retr ieve(url, filename, reporthook)
    File "/usr/lib/python1.5/urllib.py", line 186, in retrieve
    fp = self.open(url)
    File "/usr/lib/python1.5/urllib.py", line 159, in open
    return getattr(self, name)(url)
    File "/usr/lib/python1.5/urllib.py", line 260, in open_http
    h = httplib.HTTP(ho st)
    File "/usr/lib/python1.5/httplib.py", line 53, in __init__
    if host: self.connect(ho st, port)
    File "/usr/lib/python1.5/httplib.py", line 81, in connect
    self.sock.conne ct(host, port)
    IOError: [Errno socket error] (101, 'Network is unreachable')
    =============== =============== =============== ===============
    The code is like this:-
    #!/usr/bin/env python
    import os, sys, urllib, urlparse, socket
    showlines = 6
    try:
    servername, filename = sys.argv[1:3]
    except:
    servername, filename = 'www.igib.res.i n', '/sarsanalysis.ht ml'

    remoteaddr = 'http://%s%s' % (servername, filename)
    if len(sys.argv) == 4:
    localname = sys.argv[3]
    else:
    (scheme, server, path, parms, query, frag) =
    urlparse.urlpar se(remoteaddr)
    localname = os.path.split(p ath)[1]

    print remoteaddr, localname
    urllib.urlretri eve(remoteaddr, localname)
    remotedata = open(localname) .readlines()
    for line in remotedata[:showlines]: print line,
    =============== =============== =============== ===============
    I am new to the internet programming as well as python. please guide me,
    how to solve this one.

    Tags: None
    • fishboy
      #2
      Re: Client side network programming

      On Wed, 02 Jun 2004 06:23:20 -0400, "Roysun_roh it"
      <roysun_rohit@r ediffmail.com> wrote:
      [color=blue]
      >I am interested in making a search engine which takes web sites
      >iteratively, and downloads the web page or has to perform some search
      >across the web pages.
      >I am unsucessful to do so. My machine works through a proxy server and the
      >internet connection is through 24 hour lease line. when ever i try the
      >code it gives :
      >
      >Traceback (innermost last):
      > File "http-getfile-urllib2.py", line 19, in ?
      > urllib.urlretri eve(remoteaddr, localname)
      > File "/usr/lib/python1.5/urllib.py", line 66, in urlretrieve
      > return _urlopener.retr ieve(url, filename, reporthook)
      > File "/usr/lib/python1.5/urllib.py", line 186, in retrieve
      > fp = self.open(url)
      > File "/usr/lib/python1.5/urllib.py", line 159, in open
      > return getattr(self, name)(url)
      > File "/usr/lib/python1.5/urllib.py", line 260, in open_http
      > h = httplib.HTTP(ho st)
      > File "/usr/lib/python1.5/httplib.py", line 53, in __init__
      > if host: self.connect(ho st, port)
      > File "/usr/lib/python1.5/httplib.py", line 81, in connect
      > self.sock.conne ct(host, port)
      >IOError: [Errno socket error] (101, 'Network is unreachable')
      >============== =============== =============== =============== =
      >The code is like this:-
      >#!/usr/bin/env python
      >import os, sys, urllib, urlparse, socket
      >showlines = 6
      >try:
      > servername, filename = sys.argv[1:3]
      >except:
      > servername, filename = 'www.igib.res.i n', '/sarsanalysis.ht ml'
      >
      >remoteaddr = 'http://%s%s' % (servername, filename)
      >if len(sys.argv) == 4:
      > localname = sys.argv[3]
      >else:
      > (scheme, server, path, parms, query, frag) =
      >urlparse.urlpa rse(remoteaddr)
      > localname = os.path.split(p ath)[1]
      >
      >print remoteaddr, localname
      >urllib.urlretr ieve(remoteaddr , localname)
      >remotedata = open(localname) .readlines()
      >for line in remotedata[:showlines]: print line,
      >============== =============== =============== =============== =
      >I am new to the internet programming as well as python. please guide me,
      >how to solve this one.[/color]

      urllib will work with proxies. Just set your environment to point at
      it before you start python. Like this (copied from the urllib doc):

      % http_proxy="htt p://www.someproxy.c om:3128"
      % export http_proxy
      % python

      hth,[color=blue]
      ><{{{*>[/color]

        Comment

        • Anand Pillai
          #3
          Re: Client side network programming

          Try downloading HarvestMan from http://harvestman.freezope.org .

          -Anand


          fishboy <fishboy@spamsp amspam.com> wrote in message news:<qslrb0h9b stsob5056jfdnkv cqk7c8p0go@4ax. com>...[color=blue]
          > On Wed, 02 Jun 2004 06:23:20 -0400, "Roysun_roh it"
          > <roysun_rohit@r ediffmail.com> wrote:
          >[color=green]
          > >I am interested in making a search engine which takes web sites
          > >iteratively, and downloads the web page or has to perform some search
          > >across the web pages.
          > >I am unsucessful to do so. My machine works through a proxy server and the
          > >internet connection is through 24 hour lease line. when ever i try the
          > >code it gives :
          > >
          > >Traceback (innermost last):
          > > File "http-getfile-urllib2.py", line 19, in ?
          > > urllib.urlretri eve(remoteaddr, localname)
          > > File "/usr/lib/python1.5/urllib.py", line 66, in urlretrieve
          > > return _urlopener.retr ieve(url, filename, reporthook)
          > > File "/usr/lib/python1.5/urllib.py", line 186, in retrieve
          > > fp = self.open(url)
          > > File "/usr/lib/python1.5/urllib.py", line 159, in open
          > > return getattr(self, name)(url)
          > > File "/usr/lib/python1.5/urllib.py", line 260, in open_http
          > > h = httplib.HTTP(ho st)
          > > File "/usr/lib/python1.5/httplib.py", line 53, in __init__
          > > if host: self.connect(ho st, port)
          > > File "/usr/lib/python1.5/httplib.py", line 81, in connect
          > > self.sock.conne ct(host, port)
          > >IOError: [Errno socket error] (101, 'Network is unreachable')
          > >============== =============== =============== =============== =
          > >The code is like this:-
          > >#!/usr/bin/env python
          > >import os, sys, urllib, urlparse, socket
          > >showlines = 6
          > >try:
          > > servername, filename = sys.argv[1:3]
          > >except:
          > > servername, filename = 'www.igib.res.i n', '/sarsanalysis.ht ml'
          > >
          > >remoteaddr = 'http://%s%s' % (servername, filename)
          > >if len(sys.argv) == 4:
          > > localname = sys.argv[3]
          > >else:
          > > (scheme, server, path, parms, query, frag) =
          > >urlparse.urlpa rse(remoteaddr)
          > > localname = os.path.split(p ath)[1]
          > >
          > >print remoteaddr, localname
          > >urllib.urlretr ieve(remoteaddr , localname)
          > >remotedata = open(localname) .readlines()
          > >for line in remotedata[:showlines]: print line,
          > >============== =============== =============== =============== =
          > >I am new to the internet programming as well as python. please guide me,
          > >how to solve this one.[/color]
          >
          > urllib will work with proxies. Just set your environment to point at
          > it before you start python. Like this (copied from the urllib doc):
          >
          > % http_proxy="htt p://www.someproxy.c om:3128"
          > % export http_proxy
          > % python
          >
          > hth,[color=green]
          > ><{{{*>[/color][/color]

            Comment

            Previous template Next
            Working...