TRY: and Error Handling

**bartonc** · Mar 11 '07, 02:40 AM

Originally posted by jaybee

I've written a program that queries a remote site to download lots of snippets of data from a public server - one every 3 seconds or so. The basic script works, and sometimes it runs for hours. But at other times, it locks up and gives an error message like this:

Code:

  File "C:\Python25\lib\urllib.py", line 325, in open_http
    h.endheaders()
  File "C:\Python25\lib\httplib.py", line 856, in endheaders
    self._send_output()
  File "C:\Python25\lib\httplib.py", line 728, in _send_output
    self.send(msg)
  File "C:\Python25\lib\httplib.py", line 695, in send
    self.connect()
  File "C:\Python25\lib\httplib.py", line 679, in connect
    raise socket.error, msg
IOError: [Errno socket error] (10060, 'Operation timed out')

I am tinkering with try: to keep the program from crashing - ideally, it would try again, but even if it skips that query and goes on to the next snippet, that would be ok. I can't get the try/except to work.

Any ideas on what the the try/except block shold look like? I have tried various things (except error:, except socket.error, etc) - the program runs, but it wont handle the error gracefully.

Thanks, J.

It will really help if you post a code snippet, but lacking that:

Code:

try:
    whatever code of yours that invokes endheaders()
except IOError:
    pass  # or do something more elegant

Hope that helps. If not, just post a snippet and we'll track it down.

**jaybee** · Mar 11 '07, 02:52 AM

When using "except IOerror:" the script wouldn't run - message that IOerror was undefined. I thought IOerror was a built in exception?! I am assuming that the trigger error is the socket timeout. My script calls another library, which calls httplib - does that make it harder to handle the error?

The outline of what I have running now looks like this:

Code:

for each query_string:
     try:
        <~send query for query_string~>
     except error:
         # wait 15 seconds, log error, go to next query
         time.sleep(15)
         print 'timeout on ', query_string

**bartonc** · Mar 11 '07, 03:14 AM

Originally posted by jaybee

When using "except IOerror:" the script wouldn't run - message that IOerror was undefined. I thought IOerror was a built in exception?! I am assuming that the trigger error is the socket timeout. My script calls another library, which calls httplib - does that make it harder to handle the error?

The outline of what I have running now looks like this:

Code:

for each query_string:
     try:
        <~send query for query_string~>
     except error:
         # wait 15 seconds, log error, go to next query
         time.sleep(15)
         print 'timeout on ', query_string

IOerror should be IOError. All error constants have a big "E".

**ghostdog74** · Mar 11 '07, 03:14 AM

Originally posted by jaybee

When using "except IOerror:" the script wouldn't run - message that IOerror was undefined. I thought IOerror was a built in exception?! I am assuming that the trigger error is the socket timeout. My script calls another library, which calls httplib - does that make it harder to handle the error?

The outline of what I have running now looks like this:

Code:

for each query_string:
     try:
        <~send query for query_string~>
     except error:
         # wait 15 seconds, log error, go to next query
         time.sleep(15)
         print 'timeout on ', query_string

I usually use Exception,

Code:

for each query_string:
     try:
        <~send query for query_string~>
     except Exception,e:
         # wait 15 seconds, log error, go to next query
         time.sleep(15)
         print 'timeout on ', query_string
         print "errors : " , e

or just use raise

Code:

     except:
        print "some error"
        raise

**jaybee** · Mar 11 '07, 03:47 AM

Thanks, all. I am incorporating these suggestions.

**jaybee** · Mar 18 '07, 12:48 AM

Thanks to all, the program is much more stable and runs for days at a time now. It will occasionally lock up and crash, this time with a different error message. Any ideas on this:

Code:

Traceback (most recent call last):
  File "C:\shrubmed\programs\new_day.py", line 60, in <module>
    cross_reference()
  File "C:\shrubmed\programs\new_day.py", line 42, in cross_reference
    In_Pubmed = PubMed.search_for(composite_search_term)
  File "C:\Python25\lib\site-packages\Bio\PubMed.py", line 190, in search_for
    h = NCBI.esearch(**params)
  File "C:\Python25\lib\site-packages\Bio\WWW\NCBI.py", line 158, in esearch
    return _open(cgi, variables)
  File "C:\Python25\lib\site-packages\Bio\WWW\NCBI.py", line 191, in _open
    handle = urllib.urlopen(fullcgi)
  File "C:\Python25\lib\urllib.py", line 82, in urlopen
    return opener.open(url)
  File "C:\Python25\lib\urllib.py", line 190, in open
    return getattr(self, name)(url)
  File "C:\Python25\lib\urllib.py", line 334, in open_http
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "C:\Python25\lib\urllib.py", line 351, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, headers)
  File "C:\Python25\lib\urllib.py", line 608, in http_error_default
    return addinfourl(fp, headers, "http:" + url)
  File "C:\Python25\lib\urllib.py", line 951, in __init__
    addbase.__init__(self, fp)
  File "C:\Python25\lib\urllib.py", line 898, in __init__
    self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'
>>>

**bartonc** · Mar 18 '07, 03:19 AM

Originally posted by jaybee

Thanks to all, the program is much more stable and runs for days at a time now. It will occasionally lock up and crash, this time with a different error message. Any ideas on this:

Code:

Traceback (most recent call last):
  File "C:\shrubmed\programs\new_day.py", line 60, in <module>
    cross_reference()
  File "C:\shrubmed\programs\new_day.py", line 42, in cross_reference
    In_Pubmed = PubMed.search_for(composite_search_term)
  File "C:\Python25\lib\site-packages\Bio\PubMed.py", line 190, in search_for
    h = NCBI.esearch(**params)
  File "C:\Python25\lib\site-packages\Bio\WWW\NCBI.py", line 158, in esearch
    return _open(cgi, variables)
  File "C:\Python25\lib\site-packages\Bio\WWW\NCBI.py", line 191, in _open
    handle = urllib.urlopen(fullcgi)
  File "C:\Python25\lib\urllib.py", line 82, in urlopen
    return opener.open(url)
  File "C:\Python25\lib\urllib.py", line 190, in open
    return getattr(self, name)(url)
  File "C:\Python25\lib\urllib.py", line 334, in open_http
    return self.http_error(url, fp, errcode, errmsg, headers)
  File "C:\Python25\lib\urllib.py", line 351, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, headers)
  File "C:\Python25\lib\urllib.py", line 608, in http_error_default
    return addinfourl(fp, headers, "http:" + url)
  File "C:\Python25\lib\urllib.py", line 951, in __init__
    addbase.__init__(self, fp)
  File "C:\Python25\lib\urllib.py", line 898, in __init__
    self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'
>>>

It looks like somewhere in the urllib setup there's probably a default (fp=None) that you should be giving an arguement to.

TRY: and Error Handling

TRY: and Error Handling

Comment

Comment

Comment

Comment

Comment

Comment

Comment