Try, except...retry?

**Isaac To** · Jul 18 '05, 05:32 AM

Re: Try, except...retry?

>>>>> "Robert" == Robert Brewer <fumanchu@amor. org> writes:
[color=blue][color=green]
>> One sign that somebody has moved from "Python newbie" to "good Python
>> programmer" is exactly the moment they realize why it's wrong to
>> code:
>>
>> try:
>> x = could_raise_an_ exception(23)
>> process(x)
>> except Exception, err:
>> deal_with_excep tion(err)
>> proceed_here()
>>
>> and why the one obvious way is, instead:
>>
>> try:
>> x = could_raise_an_ exception(23)
>> except Exception, err:
>> deal_with_excep tion(err)
>> else:
>> process(x)
>> proceed_here()[/color][/color]

Hmmm... I agree only with reservation. The primary advantage of
exceptions, rather than other error reporting mechanisms like a return
value, is that you can group a lot of statements and have a single block of
code handling all of those. This way you can separate the error handling
code and the normal code, to make the whole logic clearer. So if the
exception doesn't happen often enough to warrant the effort, I consider the
one at the begining, i.e.,

try:
x1 = could_raise_an_ exception(1)
x2 = could_raise_an_ exception(2)
x3 = could_raise_an_ exception(3)
x4 = could_raise_an_ exception(4)
process(x1)
process(x2)
process(x3)
process(x4)
except Exception, err:
deal_with_excep tion(err)
proceed_here()

to be a better alternative than the corresponding code when all the excepts
are written out in else parts, just because it is easier to understand.

Regards,
Isaac.

**Georgy Pruss** · Jul 18 '05, 05:32 AM

Re: Try, except...retry?

"Isaac To" <kkto@csis.hku. hk> wrote in message news:7iptfxlywi .fsf@enark.csis .hku.hk...
| <...> The primary advantage of
| exceptions, rather than other error reporting mechanisms like a return
| value, is that you can group a lot of statements and have a single block of
| code handling all of those. This way you can separate the error handling
| code and the normal code, to make the whole logic clearer. <...>
|
| Regards,
| Isaac.

Not only that simple. I would say not only a lot of statements, but a lot of
very complex code, which however has no slightest idea what to do with
error situations inside it and how to proceess the errors or where and how
report them. E.g. it's quite often that you have open(file,mode) at the very
low level of your function hierarchy, but it's only several levels up that you
can analyze the error and decide what to do about it and pop up a message
box with something like "Error in installation; Please reinstall the program"
and log this error, instead of printing "Can't open /usr/bin/blah/blah/config.data"
right across the screen, or having each and every function to return error
code together with some 'useful' data.

That is, I agree with you. Just wanted to add that it's not a simple trick to
process errors for a few statements in one place, but an important design
principle.

Regarding 'retry' - it's good that there's no 'retry' in Python. It would have
introduced another loop-like structure in the language. Anyway now you can
mimic it with try-except within a loop.

--
Georgy Pruss
u='u=;print u[:2]+chr(39)+u+chr( 39)+u[2:]';print u[:2]+chr(39)+u+chr( 39)+u[2:]

**Peter Otten** · Jul 18 '05, 05:32 AM

Re: Try, except...retry?

Robert Brewer wrote:
[color=blue]
> I just got there, Alex. :) However, it got me thinking. I keep running
> into cases like:
>
> try:
> aPiranha = allPiranhas['Doug']
> except KeyError:
> aPiranha = Pirhana()
> allPiranhas['Doug'] = aPiranha
> aPiranha.weapon = u'satire'
>
> which would, in my opinion, be better written (i.e. *clearer*) as:
>
> try:
> allPiranhas['Doug'].weapon = u'satire'
> except KeyError:
> allPiranhas['Doug'] = Pirhana()
> retry
>[/color]

retry does remind me of VBA's on error goto ... resume [next] a little too
much. You lose track of the actual program flow too fast. To me, the first
variant appears much clearer in that it has only code in the try clause
that is actually supposed to
fail. Also, in

try:
setWeapon("Doug ", u"satire")
except KeyError:
allPiranhas["Doug"] = Goldfish()
retry

where would you want to jump? Should setWeapon() be executed in a way
similar to generators, saving its state in case of an exception instead of
a yield? Sometimes you can *not* execute a statement twice in a reliable
manner.

Peter

**Peter Otten** · Jul 18 '05, 05:32 AM

Re: Try, except...retry?

Isaac To wrote:
[color=blue]
> Hmmm... I agree only with reservation. The primary advantage of
> exceptions, rather than other error reporting mechanisms like a return
> value, is that you can group a lot of statements and have a single block
> of
> code handling all of those. This way you can separate the error handling
> code and the normal code, to make the whole logic clearer. So if the
> exception doesn't happen often enough to warrant the effort, I consider
> the one at the begining, i.e.,
>
> try:
> x1 = could_raise_an_ exception(1)
> x2 = could_raise_an_ exception(2)
> x3 = could_raise_an_ exception(3)
> x4 = could_raise_an_ exception(4)
> process(x1)
> process(x2)
> process(x3)
> process(x4)
> except Exception, err:
> deal_with_excep tion(err)
> proceed_here()
>
> to be a better alternative than the corresponding code when all the
> excepts are written out in else parts, just because it is easier to
> understand.[/color]

I would prefer:

x1 = x2 = x3 = x4 = None
try:
x1 = acquireRessourc e(1)
x2 = acquireRessourc e(2)
x3 = acquireRessourc e(3)
x4 = acquireRessourc e(4)
process(x1, x2, x3, x4)
finally:
if x1: releaseRessourc e(x1)
if x2: releaseRessourc e(x2)
if x3: releaseRessourc e(x3)
if x4: releaseRessourc e(x4)

where acquireRessourc e() should try hard to deal with potential exceptions
and only pass the fatal ones to the caller. Exceptions occuring when
processing the ressource should be dealt with in process() if possible. As
for process() working orthoganally on xi, that does not make any sense to
me. In such cases, be generous and provide a try ... finally and/or except
for every ressource.

Now back to else:

try:
x = canRaise()
cannotRaise(x)
except Exception, e:
dealWithExcepti on(e)

You must be very confident in your code to assume that cannotRaise() will
never raise an exception that dealWithExcepti on() will then accidentally
handle. So after a short phase of accommodation,

try:
x = canRaise()
except Exception, e:
dealWithExcepti on(e)
else:
cannotRaise(x) # may still be lying

is both more readable and better code (neglecting for the moment the
redundancy that many python coders will find in these attributes :-)

Peter

**Alex Martelli** · Jul 18 '05, 05:33 AM

Re: Try, except...retry?

Isaac To wrote:
...[color=blue]
> try:
> x1 = could_raise_an_ exception(1)
> x2 = could_raise_an_ exception(2)
> x3 = could_raise_an_ exception(3)
> x4 = could_raise_an_ exception(4)
> process(x1)
> process(x2)
> process(x3)
> process(x4)
> except Exception, err:
> deal_with_excep tion(err)
> proceed_here()
>
> to be a better alternative than the corresponding code when all the
> excepts are written out in else parts, just because it is easier to
> understand.[/color]

So is "a + b" easier to understand than math.hypot(a, b) -- but if
what you need to compute is the square root of a**2 plus b**2, the
"ease of understanding" a very different computation means nothing.

How do you KNOW that function 'process' cannot raise an exception
(e.g., that it has no bugs) or, if it does, deal_with_excep tion
will be able to uniformly handle any exception coming from EITHER
process (on ANY of the x's, too) OR any of the several calls to
could_raise_an_ exception?

It is extremely unlikely (though not impossible) that this structure
is actually best for a given set of needs. When it isn't, what
you're doing is setting up a trap for yourself here.

Let's assume, for example, that deal_with_excep tion IS able to
uniformly handle ANY exception whatsoever raised by any call to
could_raise_an_ exception, but NO exceptions are expected from any
of the calls to process.

If this is the case, then, if you make it a habit to program like
this, it WILL eventually cost you some unpredictable but likely
large amount of debugging time. A bug will be introduced during
a refactoring of function 'process', or an 'x' will be generated
that is inappropriate for the function, etc, etc. The bug will
be swallowed and hidden by the too-wide "except" clause. Unit
tests will fail mysteriously (assuming you have them: people who
botch up program structure this way are unfortunately likely to
think that unit tests are too much trouble, not "easy to understand"
enough, or the like... -- or else, worse!!!, wrong but semi-plausible
results will come out of your program...!). The best wish I can
make is that the resulting utter waste of hours, days, whatever,
will fall on the head of whoever IS responsibile for promoting or
using this wrong structure, and not on some poor hapless maintainer
coming later onto the scene of the crime.

Alex

**Alex Martelli** · Jul 18 '05, 05:33 AM

Re: Try, except...retry?

Robert Brewer wrote:
...[color=blue]
> into cases like:
>
> try:
> aPiranha = allPiranhas['Doug']
> except KeyError:
> aPiranha = Pirhana()
> allPiranhas['Doug'] = aPiranha
> aPiranha.weapon = u'satire'[/color]

Yeah, it IS frequent enough that Python has two well-known idioms to
deal with it. If a call to Piranha() has very low cost,

allPiranhas.set default('Doug', Piranha()).weap on = u'satire'

is very compact. If calling Piranha() is potentially costly, this
is unfortunately no good (due to Python's "strict" execution order,
all arguments are evaluated before running the method), so:

if 'Doug' not in allPiranhas:
allPiranhas['Doug'] = Piranha()
allPiranhas['Doug'].weapon = u'satire'

or, to avoid indexing twice:

aPiranha = allPiranhas.get ('Doug')
if aPiranha is None:
aPiranha = allPiranhas['Doug'] = Piranha()
aPiranha.weapon = u'satire'

In retrospect, it WOULD perhaps be better if setdefault was designed
to take a callable (and optional args for it) and only call it if and
when needed -- that would add a little speed and clarity in the two most
typical use cases, currently:
dictOfDicts.set default(mainKey , {})[secondaryKey] = value
and
dictOfLists.set default(key, []).append(value)
which would just become:
dictOfDicts.set default(mainKey , dict)[secondaryKey] = value
and
dictOfLists.set default(key, list).append(va lue)
respectively; and widen the applicability of .setdefault to other cases,
while costing very little actual use cases (I've never seen setdefault
correctly called with a 2nd argument that wasn't (), {}, or the like).
Ah well, too late, just musing aloud.

Still, it seems to me that the existing idioms are nevertheless
superior to your desideratum:
[color=blue]
> which would, in my opinion, be better written (i.e. *clearer*) as:
>
> try:
> allPiranhas['Doug'].weapon = u'satire'
> except KeyError:
> allPiranhas['Doug'] = Pirhana()
> retry[/color]

which does index twice anyway.
[color=blue]
> Of course, there are other ways of doing it currently, most notably with
> 1) a while loop and a retry flag, or 2) just repeating the assignment:
>
> try:
> allPiranhas['Doug'].weapon = u'satire'
> except KeyError:
> allPiranhas['Doug'] = Pirhana()
> allPiranhas['Doug'].weapon = u'satire'
>
> Yuck to both.[/color]

Yes, flags (hiding control flow in data!) and repeated code do suck, but
you need neither to get exactly the same semantics as your desideratum:

while True:
try: allPiranhas['Doug'].weapon = u'satire'
except KeyError: allPiranhas['Doug'] = Pirhana()
else: break
[color=blue]
> Current docs, 4.2 Exceptions says, "Python uses the ``termination''
> model of error handling: an exception handler can find out what happened
> and continue execution at an outer level, but it cannot repair the cause
> of the error and retry the failing operation (except by re-entering the
> offending piece of code from the top)." I'm proposing that 'retry' does
> exactly that: reenter the offending piece of code from the top. Given
> the aforementioned pressure to reduce try: blocks to one line, this
> could become a more viable/common technique.[/color]

It does not appear to me that, even assuming that this looping is in
fact the best general approach, there is enough advantage to your
proposed "try/except ... retry" technique, with respect to the
"while/try/except/else: break" one that is already possible today.

I could be wrong, of course: there is nothing that appears to me
to be "outrageous ly nonPythonic" in your proposal -- it just seems
that new statements need to be more of a win than this in order to
stand a chance. But a PEP on this is surely warranted, if you want
to try one.
[color=blue]
> Apparently Ruby has this option? Gotta keep up with the Joneses. :) I'm[/color]

Yes, Ruby does allow retry in a begin/rescue/else/end construct (on
the rescue leg only). I don't think the use case in the "Programmin g
Ruby" book shows it in a good light _at all_, though -- and I quote:

"""
@esmtp = true

begin
# First try an extended login. If it fails because the
# server doesn't support it, fall back to a normal login

if @esmtp then
@command.ehlo(h elodom)
else
@command.helo(h elodom)
end

rescue ProtocolError
if @esmtp then
@esmtp = false
retry
else
raise
end
end
"""

Eep!-) The dreaded "flag", AND a rather intricated structure
too...!!! It seems to me that a much clearer way is (Python-ish
syntax, same semantics):

try:
self.command.eh lo(helodom)
except ProtocolError:
self.command.he lo(helodom)
self.smpt = False
else:
self.smpt = True

(this leaves the self.smpt flag unset if the ProtocolError
exception propagates, rather than ensuring it's false in that
case, but if [unlikely...] this gave any problem, then just
moving the "self.smpt = False" to the top would fix that:-).

Admittedly the use of else IS "a bit precious" here, since
assigning to self.smpt is VERY unlikely to raise a ProtocolError,
so, maybe,

try:
self.command.eh lo(helodom)
self.smpt = True
except ProtocolError:
self.command.he lo(helodom)
self.smpt = False

would also be OK. I just try to foster the HABIT of using
try/except/else to avoid "protecting ", with an except, ANY
more code than strictly necessary, on general principles. But
in this case the else-less form has a very pleasing symmetry
so I would nod it through during a code-inspection or the like:-).

But the point is, the existence of retry has tempted those
_excellent_ (and pragmatic:-) authors & programmers, Thomas and
Hunt, into perverting a clean, simple structure into a little
but definite mess. This sure ain't good recommendation for
adding 'retry' to Python...:-). Given that the "key missing
in a dict" case is also dealt with quite decently without
looping, I would suggest you look for other use cases.
Perhaps simplest...:

try:
spin = raw_input("Plea se enter your PIN: ")
pin = int(spin)
except (EOFError, KeyboardInterru pt):
print "Bye bye!"
return 0
except ValueError:
print "PIN must be an integer [just digits!!!], please re-enter"
retry
else:
return validatePIN(pin )

[color=blue]
> not enough of a Pythonista yet to understand all the implications of
> such a scheme (which is why this is not a PEP), so I offer it to the
> community to discuss.[/color]

No special "implicatio ns", as the semantics are just about the
same as the above-indicated (flags-less, duplication-less)
"while True:" loop that is so easy to code explicitly today.

It's just that, partly because of this (and attendant benefits
that writing out "while" DOES clearly indicate to the reader
that the following code may repeat, etc etc), it does not seem
to me that 'retry' is worth adding. But unless somebody does
write a PEP, you'll just have my opinion about this...

Alex

**Edvard Majakari** · Jul 18 '05, 05:39 AM

Re: Try, except...retry?

Alex Martelli <aleax@aleax.it > writes:
[color=blue]
> If this is the case, then, if you make it a habit to program like
> this, it WILL eventually cost you some unpredictable but likely
> large amount of debugging time. A bug will be introduced during
> a refactoring of function 'process', or an 'x' will be generated
> that is inappropriate for the function, etc, etc. The bug will
> be swallowed and hidden by the too-wide "except" clause. Unit[/color]

How about

class xmlParseExcepti on(Exception): pass
class xmlSemanticsExc eption(Exceptio n): pass
class fileReadExcepti on(Exception): pass

try:

process()
process2()
...lots of code...

except xmlSemanticsExc eption, e:
handle_xml_sema ntics_errors()
except xmlParseExcepti on, e:
handle_xml_pars ing_errors()
except fileReadExcepti on, e:
handl_file_read _errors()

At least I find it easier to group possibly-exceptions-raising code
separately, inherit specific exceptions from class Exception using the
most simple method (ie. derived class contains only pass as body). Now it
is not that hard to pinpoint where the exception occurred, because there
is custom exception for each type of error situation, and the parameter e
of course contains appropriate error message which even further helps to
pinpoint the problem.
[color=blue]
> think that unit tests are too much trouble, not "easy to understand"
> enough, or the like... -- or else, worse!!!, wrong but semi-plausible
> results will come out of your program...!). The best wish I can
> make is that the resulting utter waste of hours, days, whatever,
> will fall on the head of whoever IS responsibile for promoting or
> using this wrong structure, and not on some poor hapless maintainer
> coming later onto the scene of the crime.[/color]

....but of course, unit tests should be used whenever possible :)

--
# Edvard Majakari Software Engineer
# PGP PUBLIC KEY available Soli Deo Gloria!

$_ = '45647661726420 4d616a616b61726 92c206120436872 69737469616e20' ; print
join('',map{chr hex}(split/(\w{2})/)),uc substr(crypt(60 281449,'es'),2, 4),"\n";

Try, except...retry?

Try, except...retry?

Comment

Comment

Comment

Comment

Comment

Comment

Comment