How to except the unexpected?

[Rene Pijlman]

Re: How to except the unexpected?

Steven D'Aprano:[color=blue]
>The OP is doing it because catching all exceptions masks bugs. There are
>certain exceptions which should be allowed through, as they indicate a bug
>in the OP's code. Normally the tactic is to catch only the exceptions you
>are interested in, and let everything else through, but the OP needs to
>catch all exceptions because there are rare exceptions which he can't
>predict in advance.[/color]

To add to that explanation: this is in a multithreaded ZODB-application.
When an unexpected exception occurs and remains uncaught, a thread
terminates, causing the thread pool to wait forever since some thread is
not consuming its termination request from the queue, causing the app to
not terminate, causing the ZODB to remain locked, causing other apps to
fail, causing havoc on my server.

I don't mind this when it's caused by a bug in my code, since that creates
the sense of urgency required to fix the bug. But it's annoying when it
happens because of an I/O exception caused by some other guys bug on the
other side of the planet :-)

--
René Pijlman

[Scott David Daniels]

Re: How to except the unexpected?

Rene Pijlman wrote:[color=blue]
> Steven D'Aprano:[color=green]
>> The OP is doing it because catching all exceptions masks bugs. There are
>> certain exceptions which should be allowed through, as they indicate a bug
>> in the OP's code. Normally the tactic is to catch only the exceptions you
>> are interested in, and let everything else through, but the OP needs to
>> catch all exceptions because there are rare exceptions which he can't
>> predict in advance.[/color]
>
> ... This is in a multithreaded ZODB-application....
> When an unexpected exception occurs and remains uncaught, a thread
> terminates, ....[/color]

At the base of the thread code, you could put

import sys, threading, logging

class MyThread(thread ing.Thread):
def run(self):
try:
threading.Threa d.run(self) # or whatever
except:
etype, error, traceback = sys.exc_info()
logging.warning ('Exception %s: %s seen at %s' %
(etype.__name__ , error, _someformat_(tr aceback)))
_try_to_rescue_ or remove_this_thr ead_

If the condition is infrequent enough. If not (if you run a real
risk of multiple threads accessing the log simultaneously) , have
a queue of log messages that you feed to a single logging thread.

--
-Scott David Daniels
scott.daniels@a cm.org

[plahey@alumni.caltech.edu]

Re: How to except the unexpected?

>Yes, and that's the Right Thing(tm) to do. Source code don't lie. Source[color=blue]
> code don't get out of sync. So source code *is* the best documentation
>(or at least the most accurate).[/color]

I could not disagree more strongly with this. Not just no, but hell
no!
[color=blue]
>Yes, and that's the Right Thing(tm) to do.[/color]

No, it is a horrible thing to do. But since the documentation of some
modules is just plain horrible we sometimes have no choice.
[color=blue]
> Source code don't lie. Source code don't get out of sync.[/color]

True but implementation details change from release to release.
[color=blue]
> So source code *is* the best documentation (or at least the most accurate).[/color]

No, source code is the *worst possible* documentation because it makes
no distinction between implementation detail and method contract. If
the implementer considers the implementation to be the documentation
then his/her refactoring options are significantly reduced. Typically
implementers are not aware of this and they refactor anyway, breaking
client code left and right.

The C++ FAQ has a nice discussion of this issue. Minimally acceptable
documentation consists of the following (I think this is language
independent):

PURPOSE: What does this method/function do
REQUIRE: pre-conditions - What must have happened before calling this
method (or restrictions on the domain of the inputs)
PROMISE: post-conditions - What can you expect upon return or what
exceptions can be thrown

I consider the above to be the minimal amount of documentation that is
acceptable. If you have less than that, I consider the method to be
undocumented. Needless to say, I consider lots of code that I see to
be undocumented.

If you don't have the above, you get the problems that OP was hitting
(or worse, see the C++ FAQ). I am not a huge fan of Java's ubiquitous
use of checked exceptions or even of static typing but it does help
supply some of the above documentation (although in a suboptimal way)
that must be supplied by hand in Python. This is the dirty little
secret of dynamically typed languages. It makes proper documentation
even more important because method signatures supply less information.