Python Code Auditing Tool

[Robey Holderith]

Does anybody know of a tool that can tell me all possible exceptions that
might occur in each line of code? What I'm hoping to find is something
like the following:

given all necessary python source and a given line ( my.py:40 ) it would
generate a list of possible exception classes sorted by function
(preferably in a tree).

Example:
------------------

my.py:40 | parsestring(gen string())

Possible Exceptions:

-def parsestring():
InvalidCharacte rException
EmptyStringExce ption
-class string, def split():
(All Exceptions that might occur directly in string.split() that are
not caught by parsestring())
(All functions called by string.split() and their exceptions and sub-
functions)
-def genstring():
SomeException
...

--------------------

This would be extremely useful for deciding how to write try: except
blocks and in figuring out what all possible errors that might occur would
be.

-Robey Holderith

[Paul Rubin]

Re: Python Code Auditing Tool

Robey Holderith <robey@flamingl unchbox.net> writes:[color=blue]
> Does anybody know of a tool that can tell me all possible exceptions that
> might occur in each line of code? What I'm hoping to find is something
> like the following:[/color]

That is impossible. The parameter to the raise statement is a class
object, which can be anything. I.e. you could say:

class ex1: pass
class ex2: pass

if something(): my_ex = ex1
else: my_ex = ex2

raise my_ex # static tool can't know what exception gets raised here.

[Robey Holderith]

Re: Python Code Auditing Tool

On Tue, 01 Feb 2005 21:52:28 -0800, Paul Rubin wrote:
[color=blue]
> Robey Holderith <robey@flamingl unchbox.net> writes:[color=green]
>> Does anybody know of a tool that can tell me all possible exceptions that
>> might occur in each line of code? What I'm hoping to find is something
>> like the following:[/color]
>
> That is impossible. The parameter to the raise statement is a class
> object, which can be anything. I.e. you could say:
>
> class ex1: pass
> class ex2: pass
>
> if something(): my_ex = ex1
> else: my_ex = ex2
>
> raise my_ex # static tool can't know what exception gets raised here.[/color]

I suppose that I am willing to lessen my expectations from _all_ to most.
;-) Regarding your example I could also do:

if something():
def nothing(): return 0
else:
def nothing(): return 1

But this doesn't stop IDEs from attempting to do auto-completion. I'm not
trying to find hidden exceptions... just trying to easily get an idea of
what could go wrong on each line of code.

-Robey Holderith

[Diez B. Roggisch]

Re: Python Code Auditing Tool

> I suppose that I am willing to lessen my expectations from _all_ to most.[color=blue]
> ;-) Regarding your example I could also do:
>
> if something():
> def nothing(): return 0
> else:
> def nothing(): return 1
>
> But this doesn't stop IDEs from attempting to do auto-completion. I'm not
> trying to find hidden exceptions... just trying to easily get an idea of
> what could go wrong on each line of code.[/color]

There is AFAIK only one language that this can de accomplished - java, and
that's because of these checked exceptions of theirs. But checked
exceptions are considered harmful:

http://www.gcek.net/ref/books/sw/ooad/tip/#_Toc41169682

I totally agree with that - in java, I tend to throw SystemException s to rid
myself of endless try/catch clauses that obscure the real problem.

So - there is no way of knowing this. The only thing I can think of is to
keep some docs around that specify what exceptions to be expected, and that
tool of yours could try and see if it can identify a function/method by
name and notify you of the possible exceptions thrown. Might actually work
out quite well for the standardlib, if one does the work for annotating all
functions/methods properly.

--
Regards,

Diez B. Roggisch

[Neil Benn]

Re: Python Code Auditing Tool

Diez B. Roggisch wrote:
[color=blue][color=green]
>>I suppose that I am willing to lessen my expectations from _all_ to most.
>>;-) Regarding your example I could also do:
>>
>><
>>
>>[/color][/color]
<snip>
[color=blue][color=green]
>>There is AFAIK only one language that this can de accomplished - java, and
>>that's because of these checked exceptions of theirs. But checked
>>exceptions are considered harmful:
>>
>>http://www.gcek.net/ref/books/sw/ooad/tip/#_Toc41169682
>>
>>I totally agree with that - in java, I tend to throw SystemException s to rid
>>myself of endless try/catch clauses that obscure the real problem.
>>
>>
>>[/color][/color]
<snip>
Hello,

I'm afraid that the only reliable way to gather what exceptions are
raised is to read docs and/or come up with test cases. This has been a
bugbear of mine in Python as it's not common to find a nice :Exceptions:
IOError <desc>, IllegalArgument Error <desc> type of description in the docs.

However if you want an incomplete test, you could parse the code and
check for raises and retrieve the class name of the exception - however
this would be patchy at best. Therefore it would sort of negate the
point of doing the analysis in the first place.

Even in Java you cannot find every exception that will be
thrown, only 'checked' exceptions but this is a percentage of all the
exceptions (BTW why do you throw SystemException - it's a CORBA
exception! OK, it's a runtime exception but why not just simply extend
RuntimeExceptio n?). Also, if someone ever puts - catch (Exception e){}
in their code they deserve to be kneecapped, IMHO the fault is with
sloppy coding not with the supplied tools.

Unfortunately its docs and testing again, that's why we get paid (if
you're doing a job) or not paid (if you're doing it for fun!). Although
one language which comes closer is Eiffel which has require and ensure
clauses on every method (following Meyer's Programming by contract
philosophy).

Cheers,

Neil

--

Neil Benn
Senior Automation Engineer
Cenix BioScience
BioInnovations Zentrum
Tatzberg 46
D-01307
Dresden
Germany

Tel : +49 (0)351 4173 154
e-mail : benn@cenix-bioscience.com
Cenix Website : http://www.cenix-bioscience.com

[Diez B. Roggisch]

Re: Python Code Auditing Tool

Hi,
[color=blue]
> I'm afraid that the only reliable way to gather what exceptions are
> raised is to read docs and/or come up with test cases. This has been a
> bugbear of mine in Python as it's not common to find a nice :Exceptions:
> IOError <desc>, IllegalArgument Error <desc> type of description in the
> docs.
>
> However if you want an incomplete test, you could parse the code and
> check for raises and retrieve the class name of the exception - however
> this would be patchy at best. Therefore it would sort of negate the
> point of doing the analysis in the first place.[/color]

I don't want that - the OP wants. I agree with you.
[color=blue]
> Even in Java you cannot find every exception that will be
> thrown, only 'checked' exceptions but this is a percentage of all the
> exceptions (BTW why do you throw SystemException - it's a CORBA
> exception! OK, it's a runtime exception but why not just simply extend
> RuntimeExceptio n?). Also, if someone ever puts - catch (Exception e){}
> in their code they deserve to be kneecapped, IMHO the fault is with
> sloppy coding not with the supplied tools.[/color]

Most probably I throw RuntimeExceptio n - that was out of my head, I luckily
I haven't been coding java too much lately :)
[color=blue]
> Unfortunately its docs and testing again, that's why we get paid (if
> you're doing a job) or not paid (if you're doing it for fun!). Although
> one language which comes closer is Eiffel which has require and ensure
> clauses on every method (following Meyer's Programming by contract
> philosophy).[/color]

Full ack again.

--
Regards,

Diez B. Roggisch

[System Administrator]

Re: Python Code Auditing Tool

[color=blue][color=green]
>> Does anybody know of a tool that can tell me all possible exceptions
>> that might occur in each line of code? What I'm hoping to find is
>> something like the following:[/color][/color]

Paul> That is impossible. The parameter to the raise statement is a
Paul> class object, which can be anything.

Sure, but in all but the rarest of cases the first arg to raise is a
specific exception, probably one of the standard exceptions. In the Python
code in the distribution (ignoring the test directory where all sorts of
mischief is done to stress things), here are the most common words following
"raise" where "raise" is the first word on the line:

% find . -name '*.py' \[color=blue]
> | egrep -v '\./test' \
> | xargs egrep '^ *raise ' \
> | awk '{print $3}' \
> | sed -e 's/[(,].*//' \
> | sort \
> | uniq -c \
> | sort -rn \
> | head -15[/color]
246 ValueError
227 aetools.Error
216 Error
124 TypeError
101 error
75 RuntimeError
53 IOError
36 NotImplementedE rror
36 ImportError
36 EOFError
31 SyntaxError
23 KeyError
23 AttributeError
22 DistutilsPlatfo rmError
21 UnicodeError

Without checking, my guess is that #5 ("error") is one of a handful of
exception classes defined at module scope (ftplib, anydbm, sre_constants,
poplib, among others all define such an exception class), and not a variable
that accepts multiple values as in your example.

In short, while not perfect, simply grepping for '^ *(class|def|rai se) ' and
printing the first and second words of each output line would probably give
you a pretty good idea of what gets raised where.

Skip

[Roy Smith]

Re: Python Code Auditing Tool

In article <mailman.1759.1 107360282.22381 .python-list@python.org >,
System Administrator <skip@pobox.com > wrote:[color=blue]
>[color=green][color=darkred]
> >> Does anybody know of a tool that can tell me all possible exceptions
> >> that might occur in each line of code? What I'm hoping to find is
> >> something like the following:[/color][/color]
>
> Paul> That is impossible. The parameter to the raise statement is a
> Paul> class object, which can be anything.
>
>Sure, but in all but the rarest of cases the first arg to raise is a
>specific exception, probably one of the standard exceptions. In the Python
>code in the distribution (ignoring the test directory where all sorts of
>mischief is done to stress things), here are the most common words following
>"raise" where "raise" is the first word on the line:
>
> % find . -name '*.py' \[color=green]
> > | egrep -v '\./test' \
> > | xargs egrep '^ *raise ' \
> > | awk '{print $3}' \
> > | sed -e 's/[(,].*//' \
> > | sort \
> > | uniq -c \
> > | sort -rn \
> > | head -15[/color]
> 246 ValueError
> 227 aetools.Error
> 216 Error
> 124 TypeError
> 101 error
> 75 RuntimeError
> 53 IOError
> 36 NotImplementedE rror
> 36 ImportError
> 36 EOFError
> 31 SyntaxError
> 23 KeyError
> 23 AttributeError
> 22 DistutilsPlatfo rmError
> 21 UnicodeError[/color]

It's kind of interesting (scarry?) that in roughly 20% of the cases
nothing more specific than Error is raised.

[Peter Hansen]

Re: Python Code Auditing Tool

Roy Smith wrote:[color=blue]
> Skip Montanaro wrote:[color=green]
>> 246 ValueError
>> 227 aetools.Error
>> 216 Error
>> 124 TypeError
>> 101 error
>> 75 RuntimeError
>> 53 IOError
>> 36 NotImplementedE rror
>> 36 ImportError
>> 36 EOFError
>> 31 SyntaxError
>> 23 KeyError
>> 23 AttributeError
>> 22 DistutilsPlatfo rmError
>> 21 UnicodeError[/color]
>
> It's kind of interesting (scarry?) that in roughly 20% of the cases
> nothing more specific than Error is raised.[/color]

(In case someone reading doesn't know) there isn't actually
an "Error" in the standard set of exceptions. It seems
likely that pretty much all of those uses are actually
module-specific Errors, and as such they are probably much
more specific than the unadorned name might imply.

Also, when one is trying to pick an appropriate exception
to raise, it is often the case that none of the standard
exceptions seems appropriate. In those cases (although I
personally would prefer a different name than "Error")
there's often no good alternative to making your own
unique module-specific Exception subclass.

-Peter

[Skip Montanaro]

Re: Python Code Auditing Tool

[color=blue][color=green]
>> 246 ValueError
>> 227 aetools.Error
>> 216 Error
>> 124 TypeError
>> 101 error
>> 75 RuntimeError
>> 53 IOError
>> 36 NotImplementedE rror
>> 36 ImportError
>> 36 EOFError
>> 31 SyntaxError
>> 23 KeyError
>> 23 AttributeError
>> 22 DistutilsPlatfo rmError
>> 21 UnicodeError[/color][/color]

Roy> It's kind of interesting (scarry?) that in roughly 20% of the cases
Roy> nothing more specific than Error is raised.

Not really. You might have code in a module named mod like this:

class Error(Exception ): pass

then later:

def foo(...):
... blah blah blah ...
if condition:
raise Error, "hey dummy!"

The caller might look like:

import mod

...

try:
mod.foo()
except mod.Error, msg:
print msg

That said, the tendency for many newer modules seems to be to discriminate
exceptions based on type (look at urllib2 for example) while older modules
tended to have just a single exception they raised (look at ftplib). I
suspect that has something to do with whether the module was originally
written before or after Python introduced class exceptions.

Skip