Python style: exceptions vs. sys.exit()

[Ross Ridge]

Re: Python style: exceptions vs. sys.exit()

Grant Edwards <invalid@invali dwrote:

Same here. It's like an automotive engine controls designer
asking if a failed O2 sensor should turn on the check engine
light or blow up the car.

Ross Ridge <rridge@csclub. uwaterloo.cawro te:

No, it's more like asking if the failed sensor should turn on
a strange and mysterious light on the dashboard

Grant Edwards <invalid@invali dwrote:

You're right. I had forgotten that sys.exit() is actually
raising the system exit exception, and that the application
calling the library could handle that exception.

Ross Ridge a écrit :

Well, my point was that exceptions in Python are a bit like a car's
check engine light. Few drivers know what this mysterious light means,
and aren't prepared to do anything about it when it goes on.

Bruno Desthuilliers <bdesth.quelque chose@free.quel quepart.frwrote :

>You're kidding, aren't you ?

Of course not. Plenty of people were quick to say that the exception
should be passed through to the caller. No one said this behaviour
should be documented. There may be little practical difference bewteen
calling sys.exit() after printing an error and progating an exception
if no one using the library knows that it could generate that exception
in those circumstances.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rridge@csclub.u waterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

[Craig Allen]

Re: Python style: exceptions vs. sys.exit()

Why, yes, I am wearing my BOFH hat. How could you tell?
>
--
Tim Rowe

evil, but I think you may be a BSEFH, not a BOFH.

[Steven D'Aprano]

Re: Python style: exceptions vs. sys.exit()

On Wed, 24 Sep 2008 10:54:40 -0500, Grant Edwards wrote:

You're right. I had forgotten that sys.exit() is actually raising the
system exit exception, and that the application calling the library
could handle that exception.

Could but shouldn't.

The exception hierarchy was re-designed in Python 2.5 specifically so
that SystemExit and KeyboardInterru pt no longer inherit from Exception.
That means this will do the right thing:

try:
process(record)
if changed:
update(record)
except Exception:
# don't write the record if any error occurs
pass


Neither SystemExit nor KeyboardInterru pt are errors, and they should not
be treated as errors. I quote from the Fine Manual:

[SystemExit] inherits from BaseException instead of StandardError or
Exception so that it is not accidentally caught by code that catches
Exception. This allows the exception to properly propagate up and cause
the interpreter to exit.

http://docs.python.org/lib/module-exceptions.html

SystemExit is not an error. KeyboardInterru pt is not an error. Libraries
should not use SystemExit to signal an error any more than they should
use MemoryError to signal a missing attribute.

Whether or not an application exits is up to the application to decide,
not the library it wraps. Libraries should report errors by raising an
Exception (note the capital E), and the application should decide whether
or not it is recoverable, and if not, exit (perhaps by simply not
catching the exception).



--
Steven

[Steven D'Aprano]

Re: Python style: exceptions vs. sys.exit()

On Wed, 24 Sep 2008 17:11:28 -0400, Ross Ridge wrote:

Plenty of people were quick to say that the exception should be passed
through to the caller. No one said this behaviour should be documented.
There may be little practical difference bewteen calling sys.exit()
after printing an error and progating an exception if no one using the
library knows that it could generate that exception in those
circumstances.

That's true, I didn't explicitly say that the library should be
documented. Nor did I say that it shouldn't be riddled with bugs. There's
little practical difference between a buggy library and one that raises
unexpected (i.e. undocumented) exceptions either.



--
Steven

[Bruno Desthuilliers]

Re: Python style: exceptions vs. sys.exit()

Steven D'Aprano a écrit :

On Wed, 24 Sep 2008 17:11:28 -0400, Ross Ridge wrote:
>

>Plenty of people were quick to say that the exception should be passed
>through to the caller. No one said this behaviour should be documented.
> There may be little practical difference bewteen calling sys.exit()
>after printing an error and progating an exception if no one using the
>library knows that it could generate that exception in those
>circumstance s.

>
That's true, I didn't explicitly say that the library should be
documented. Nor did I say that it shouldn't be riddled with bugs. There's
little practical difference between a buggy library and one that raises
unexpected (i.e. undocumented) exceptions either.

Also note that there are quite a couples cases where the library authors
themselves cannot predict which exception types may be raised - as soon
as the library functions expect callback functions, file-like or
dict-like or whatever-like objects etc, it's the caller's responsability
to handle the exceptions that may be raised by what *he* passes to the
library...

[Ross Ridge]

Re: Python style: exceptions vs. sys.exit()

Ross Ridge wrote:

Plenty of people were quick to say that the exception should be passed
through to the caller. No one said this behaviour should be documented.
There may be little practical difference bewteen calling sys.exit()
after printing an error and progating an exception if no one using the
library knows that it could generate that exception in those
circumstances.

Steven D'Aprano <steve@REMOVE-THIS-cybersource.com .auwrote:

>That's true, I didn't explicitly say that the library should be
>documented. Nor did I say that it shouldn't be riddled with bugs. There's
>little practical difference between a buggy library and one that raises
>unexpected (i.e. undocumented) exceptions either.

The problem is that few Python libraries properly document where and
when they might generate exceptions. They'll document the fact that
they have an "error" exception, but only vaguely say which functions or
methods could generate it and why. You need either use trial and error
to find out, or look at the source.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rridge@csclub.u waterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

[Ross Ridge]

Re: Python style: exceptions vs. sys.exit()

Bruno Desthuilliers <bruno.42.desth uilliers@websit eburo.invalidwr ote:

>Also note that there are quite a couples cases where the library authors
>themselves cannot predict which exception types may be raised - as soon
>as the library functions expect callback functions, file-like or
>dict-like or whatever-like objects etc, it's the caller's responsability
>to handle the exceptions that may be raised by what *he* passes to the
>library...

Ug... that's another documentation pet-peeve of mine. Libraries that
say they take file-like (or whatever-like) object, but don't say exactly
how file-like it needs.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rridge@csclub.u waterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

[Bruno Desthuilliers]

Re: Python style: exceptions vs. sys.exit()

Ross Ridge a écrit :

Bruno Desthuilliers <bruno.42.desth uilliers@websit eburo.invalidwr ote:

>Also note that there are quite a couples cases where the library authors
>themselves cannot predict which exception types may be raised - as soon
>as the library functions expect callback functions, file-like or
>dict-like or whatever-like objects etc, it's the caller's responsability
>to handle the exceptions that may be raised by what *he* passes to the
>library...

>
Ug... that's another documentation pet-peeve of mine. Libraries that
say they take file-like (or whatever-like) object, but don't say exactly
how file-like it needs.

Seems not to be such a problem in practice...

[Ross Ridge]

Re: Python style: exceptions vs. sys.exit()

Bruno Desthuilliers <bruno.42.desth uilliers@websit eburo.invalidwr ote:

Also note that there are quite a couples cases where the library authors
themselves cannot predict which exception types may be raised - as soon
as the library functions expect callback functions, file-like or
dict-like or whatever-like objects etc, it's the caller's responsability
to handle the exceptions that may be raised by what *he* passes to the
library...

Ross Ridge a écrit :

Ug... that's another documentation pet-peeve of mine. Libraries that
say they take file-like (or whatever-like) object, but don't say exactly
how file-like it needs.

Bruno Desthuilliers <bruno.42.desth uilliers@websit eburo.invalidwr ote:

>Seems not to be such a problem in practice...

Not if you don't mind figuring out such things by trial and error or
looking at the source.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rridge@csclub.u waterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

[Lie]

Re: Python style: exceptions vs. sys.exit()

On Sep 25, 3:05 pm, Bruno Desthuilliers <bruno.
42.desthuilli.. .@websiteburo.i nvalidwrote:

Steven D'Aprano a écrit :
>

On Wed, 24 Sep 2008 17:11:28 -0400, Ross Ridge wrote:

>

Plenty of people were quick to say that the exception should be passed
through to the caller.  No one said this behaviour should be documented.
 There may be little practical difference bewteen calling sys.exit()
after printing an error and progating an exception if no one using the
library knows that it could generate that exception in those
circumstances.

>

That's true, I didn't explicitly say that the library should be
documented. Nor did I say that it shouldn't be riddled with bugs. There's
little practical difference between a buggy library and one that raises
unexpected (i.e. undocumented) exceptions either.

>
Also note that there are quite a couples cases where the library authors
themselves cannot predict which exception types may be raised - as soon
as the library functions expect callback functions, file-like or
dict-like or whatever-like objects etc, it's the caller's responsability
to handle the exceptions that may be raised by what *he* passes to the
library...

No, the library author can always predict which exception his library
may raise. If a callback function, file-like, dict-like, etc raises an
exception, it is not his libraries' exception anymore and is not his
responsibility. In that case we should refer to the documentation for
the callback/etc/etc instead of the documentation for the library.

[Lawrence D'Oliveiro]

Re: Python style: exceptions vs. sys.exit()

In message <gbgu3v$mie$1@r umours.uwaterlo o.ca>, Ross Ridge wrote:

You need either use trial and error to find out, or look at the source.

So what's wrong with using the source as documentation? :)

[Steven D'Aprano]

Re: Python style: exceptions vs. sys.exit()

On Mon, 29 Sep 2008 18:27:22 +0200, Bruno Desthuilliers wrote:

Lawrence D'Oliveiro a écrit :

>In message <gbgu3v$mie$1@r umours.uwaterlo o.ca>, Ross Ridge wrote:
>>

>>You need either use trial and error to find out, or look at the
>>source.

>>
>So what's wrong with using the source as documentation? :)

>
Don't know... Ok, having higher-level documentation (the big picture,
and quick description of what and how for classes and functions) really
helps. But when it comes to nitty-gritty details, source code is the
best documentation ever, since it's always accurate and up to date.
>
FWIW, I'm often surprised by people asking questions about some
implementation detail of some open-source library or framework that are
very easily answered just looking at the source code. Reading the source
is 1/ the best way to really know how something is implemented and 2/
usually very instructive.

Reading the source code is good, but it's not a panacea. There are at
least four things wrong with the advice to read the source code:


(1) It's not always available.

(2) Even when the source is available, it is sometimes a legal trap to
read it with respect to patents and copyright. E.g. some Microsoft so-
called "open" licences (but not all) allow you to read the source, but if
you do then everything you program in a related field from that point is
legally contaminated and could be considered a derivative work of
Microsoft's software.

(3) Source code not always understandable without significant effort.
Code can be obfuscated, either to hide the algorithm, as an optimization,
or simply because the coder is cleverer than you are. It might be in a
language you don't understand (e.g. Python built-ins are written in C,
not Python. I have to learn C to find out what exceptions sorted() can
raise?).

That's why accurate documentation should be preferred in the first place:
it is easier to read and understand. It might take you hours of hard
study to learn that function spam(x) raises ValueError if the argument is
invalid, or two seconds to read it in the docs.

Yes, documentation can fall behind the source code, but once the code is
stable and the documentation has caught up and is accurate, there's no
reason to re-invent the wheel by slugging through the source just to find
out something already documented.

(4) Even when the source code is available, legally unencumbered, in a
language you understand and not too complicated for you to grasp, there's
the combinatorial explosion: you may need to read and understand an
arbitrarily large and complex chain of software merely to know what a
single function can do. E.g. you want to know what exceptions function
spam() can raise:

def spam(x):
a = 1
b = 2
c = ham(x)
return fried_eggs(a, b, c)

Now you need to understand ham() and fried_eggs(), but they aren't
documented either. So you turn to their source code, and each of them
call two functions, each of which call another two functions, each of
which call *another* two functions...

How much source code are you expected to read just to understand the four-
line function spam()?


--
Steven

[Lawrence D'Oliveiro]

Re: Python style: exceptions vs. sys.exit()

In message <00f15d41$0$206 17$c3e8da3@news .astraweb.com>, Steven D'Aprano
wrote:

(1) It's not always available.

But we're talking about Python libraries here, right?

(2) Even when the source is available, it is sometimes a legal trap to
read it with respect to patents and copyright.

That's not how patents work.

[Bruno Desthuilliers]

Re: Python style: exceptions vs. sys.exit()

Steven D'Aprano a écrit :

On Mon, 29 Sep 2008 18:27:22 +0200, Bruno Desthuilliers wrote:
>

>Lawrence D'Oliveiro a écrit :

>>In message <gbgu3v$mie$1@r umours.uwaterlo o.ca>, Ross Ridge wrote:
>>>
>>>You need either use trial and error to find out, or look at the
>>>source.
>>So what's wrong with using the source as documentation? :)

>Don't know... Ok, having higher-level documentation (the big picture,
>and quick description of what and how for classes and functions) really
>helps. But when it comes to nitty-gritty details, source code is the
>best documentation ever, since it's always accurate and up to date.
>>
>FWIW, I'm often surprised by people asking questions about some
>implementati on detail of some open-source library or framework that are
>very easily answered just looking at the source code. Reading the source
>is 1/ the best way to really know how something is implemented and 2/
>usually very instructive.

>
Reading the source code is good, but it's not a panacea.

Not what I implied.

There are at
least four things wrong with the advice to read the source code:

My "advice to read the source code" was not meant as a *replacement* for
documentation, but as a *complement* to it. What I meant is that you
just can't document each and every detail of implementation.

>
(1) It's not always available.
>
(2) Even when the source is available, it is sometimes a legal trap to
read it with respect to patents and copyright. E.g. some Microsoft so-
called "open" licences (but not all) allow you to read the source, but if
you do then everything you program in a related field from that point is
legally contaminated and could be considered a derivative work of
Microsoft's software.

I obviously implied that source was freely available and you had the
right to read it. Else it just makes no sense.

(3) Source code not always understandable without significant effort.

That's why reading the source can have a great educational value, isn't
it ?-)

Code can be obfuscated, either to hide the algorithm,

same problem as closed-source software - not concerned by this advice.

as an optimization,
or simply because the coder is cleverer than you are. It might be in a
language you don't understand (e.g. Python built-ins are written in C,
not Python. I have to learn C to find out what exceptions sorted() can
raise?).

Every developer should have at least basic knowledge of C. MHO of course.

That's why accurate documentation should be preferred in the first place.

Indeed. Did I say otherwise ? Now not all code has accurate
documentation, and then you're happy to be able to access and possibly
understand the source code. I'm not talking about an ideal world here.

(snip)

Yes, documentation can fall behind the source code, but once the code is
stable and the documentation has caught up and is accurate, there's no
reason to re-invent the wheel by slugging through the source just to find
out something already documented.

Once again, that's not what I said.

(4) Even when the source code is available, legally unencumbered, in a
language you understand and not too complicated for you to grasp, there's
the combinatorial explosion: you may need to read and understand an
arbitrarily large and complex chain of software merely to know what a
single function can do.

Yes, this happens when hi-level documentation is lacking. At least you
have a chance to gain some insight !-)

E.g. you want to know what exceptions function
spam() can raise:
>
def spam(x):
a = 1
b = 2
c = ham(x)
return fried_eggs(a, b, c)
>
Now you need to understand ham() and fried_eggs(), but they aren't
documented either. So you turn to their source code, and each of them
call two functions, each of which call another two functions, each of
which call *another* two functions...

And still you're a lucky guy if there's no callback, conditional import
and / or polymorphic dispatch involved.


Steve, I may not have been clear, but I didn't meant that code shouldn't
be documented. I was :

1/ answering to the question "So what's wrong with using the source as
documentation?" , my own personal answer being that it's something I
often do, whether because I want to find out some detail not covered by
the available documentation, whatever this available documention is worth

2/ digressing about the fact that, to my suprise, few developpers seem
to *first* have a look at the source code when either documentation is
lacking or they'd like to know more about some implementation detail.
But FWIW, it seems that few developpers even bother reading the
documentation at all :(

[greg]

Re: Python style: exceptions vs. sys.exit()

Lawrence D'Oliveiro wrote:

In message <00f15d41$0$206 17$c3e8da3@news .astraweb.com>, Steven D'Aprano
wrote:
>

(2) Even when the source is available, it is sometimes a legal trap to
read it with respect to patents and copyright.

>
That's not how patents work.

I don't think that's how copyrights work either. As far as
I know, whether something is deemed a derivative work is
judged on the basis of how similar it is to another work,
not whether its author had knowledge of the other work.
As long as you express an idea in an original way, it
shouldn't matter where you got the idea from.

--
Greg