Exception raising, and performance implications.

[leo]

Hello all -

I was wondering about the performance implications of explicitly
raising exceptions to get information about the current frame.
Something like what the inspect module does, with:

---
def currentframe():
"""Return the frame object for the caller's stack frame."""
try:
raise 'catch me'
except:
return sys.exc_traceba ck.tb_frame.f_b ack
---

I come from a java background, where Exceptions are a big Avoid Me, but
are the performance implications the same in Python? We're expecting a
big load on our app (100,000 users/hour) , so we'd like to be as tuned
as possible.

Thanks,
leo

[Paul Rubin]

Re: Exception raising, and performance implications.

"leo" <leomendoza@gma il.com> writes:[color=blue]
> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python?[/color]

Well, you could measure it experimentally pretty easily, but anyway,
Python exceptions are much less expensive than Java exceptions.

[jepler@unpythonic.net]

Re: Exception raising, and performance implications.

As for performance, you'll need to benchmark it.

However, I think the functionality you're asking for is available as
inspect.current frame(), and if the implementation is in "C" it may have a tiny
performance advantage over the Python version.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQah+Jd0 1MZaTXX0RAv0KAJ 4/H3m7sSaszecPpA4 hN6DX+hkN9QCfYy 9F
vc/97Vn8K+A30ZNAVK Ci4R0=
=OyAA
-----END PGP SIGNATURE-----

[Tony Nelson]

Re: Exception raising, and performance implications.

In article <1128375280.034 852.119910@o13g 2000cwo.googleg roups.com>,
"leo" <leomendoza@gma il.com> wrote:
[color=blue]
> Hello all -
>
> I was wondering about the performance implications of explicitly
> raising exceptions to get information about the current frame.
> Something like what the inspect module does, with:[/color]

Python uses exceptions internally, using StopIteration to terminate the
iterator in a for: loop.
[color=blue]
> ---
> def currentframe():
> """Return the frame object for the caller's stack frame."""
> try:
> raise 'catch me'
> except:
> return sys.exc_traceba ck.tb_frame.f_b ack
> ---[/color]

This also does a traceback; you might want to measure the cost of that.
[color=blue]
> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python? We're expecting a
> big load on our app (100,000 users/hour) , so we'd like to be as tuned
> as possible.[/color]

Switching to Python, eh? Remember to measure, measure, measure!
_______________ _______________ _______________ _______________ ____________
TonyN.:' *firstname*nlsn ews@georgea*las tname*.com
' <http://www.georgeanels on.com/>

[jepler@unpythonic.net]

Re: Exception raising, and performance implications.

On Mon, Oct 03, 2005 at 02:34:40PM -0700, leo wrote:[color=blue]
> I come from a java background, where Exceptions are a big Avoid Me, but
> are the performance implications the same in Python? We're expecting a
> big load on our app (100,000 users/hour) , so we'd like to be as tuned
> as possible.[/color]

I don't know what you do for each user, but here's a data point: My 1.8GHz
Sempron (firmly in the "budget" category) uses a fairly unoptimized piece of
software called aether[1] to serve my blog and a few other things. It uses
Apache and good old fashioned cgi-bin one-process-per-request to serve up most
pages. It parses a substantial amount of Python code each time with execfile
(not using byte-compiled files). It still does this in 87ms on average (albeit
for a simple page), or about 41k requests per hour.

By simply buying a faster CPU, or by avoiding execfile, or by using a
higher-performance technology than CGI, or with judicious use of caching, I'm
sure I could reach 100,000 requests per hour, and by using several of the
techniques together I might be able to reach 400k requests per hour. Probably
it would be after these fairly obvious, high-level optimization ideas were
exhausted that I would look at the code at a microscopic level for optimization
ideas.

But because my website is stuck on the slow end of a DSL line, there's not much
point to any of this.

Jeff
[1] http://www.logarithmic.net/pfh/aether (not my site)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQcd0Jd0 1MZaTXX0RAqMEAJ 9UDEYsW+v9toSGb cepC1oH/PEeNACdHgQk
tcUZnt/nBNFW+nNWhOKG29 E=
=56lv
-----END PGP SIGNATURE-----

[leo]

Re: Exception raising, and performance implications.

> However, I think the functionality you're asking for is available as[color=blue]
> inspect.current frame(), and if the implementation is in "C" it may have a tiny
> performance advantage over the Python version.[/color]

You're absolutely right, in fact the code snippet from my OP was taken
directly from inspect.current frame. We're intending on using this in
production, and I'm trying to gauge what the implications may be.
[color=blue]
> Python uses exceptions internally, using StopIteration to terminate the
> iterator in a for: loop.[/color]

Wow, I was unaware of this. If Python internally uses exceptions, maybe
they aren't as detrimental as I thought.

That said, I will be judiciously profiling my code and measuring as
much as possible, I just wanted to throw this question out to the NG in
case anyone had done this before (and so I can put off learning the
profiler a little bit longer :) )

Thanks all for the replies.

[Steve Holden]

Re: Exception raising, and performance implications.

leo wrote:[color=blue][color=green]
>>However, I think the functionality you're asking for is available as
>>inspect.curre ntframe(), and if the implementation is in "C" it may have a tiny
>>performance advantage over the Python version.[/color]
>
>
> You're absolutely right, in fact the code snippet from my OP was taken
> directly from inspect.current frame. We're intending on using this in
> production, and I'm trying to gauge what the implications may be.
>
>[color=green]
>>Python uses exceptions internally, using StopIteration to terminate the
>>iterator in a for: loop.[/color]
>
>
> Wow, I was unaware of this. If Python internally uses exceptions, maybe
> they aren't as detrimental as I thought.
>
> That said, I will be judiciously profiling my code and measuring as
> much as possible, I just wanted to throw this question out to the NG in
> case anyone had done this before (and so I can put off learning the
> profiler a little bit longer :) )
>
> Thanks all for the replies.
>[/color]
Do note, however, that detecting an exception inside the C framework of
the interpreter carries less overhead than detecting an exception in
Python itself.

That said, exceptions are probably rather more "lightweigh t" than you
might imagine, so benchmarking (the profiler may not be best - have you
come across "timeit.py" ?) is the best way to go.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

[Phillip J. Eby]

Re: Exception raising, and performance implications.

leo wrote:[color=blue]
>
> You're absolutely right, in fact the code snippet from my OP was taken
> directly from inspect.current frame. We're intending on using this in
> production, and I'm trying to gauge what the implications may be.[/color]

Use sys._getframe() instead; it doesn't raise an exception.

[color=blue]
> Wow, I was unaware of this. If Python internally uses exceptions, maybe
> they aren't as detrimental as I thought.[/color]

Exceptions raised from C code and caught by C code use considerably
less resources, so a "for" loop catching a StopIteration raised by a
built-in iterator will have better performance than Python code
catching an exception raised in Python code. So, it's not necessarily
the case that the "for" loop scenario matches your scenario(s). Always
measure.

[Tom Anderson]

Re: Exception raising, and performance implications.

On Mon, 3 Oct 2005, it was written:
[color=blue]
> "leo" <leomendoza@gma il.com> writes:
>[color=green]
>> I come from a java background, where Exceptions are a big Avoid Me, but
>> are the performance implications the same in Python?[/color]
>
> Well, you could measure it experimentally pretty easily, but anyway,
> Python exceptions are much less expensive than Java exceptions.[/color]

Really? How come? What is it that stops java using the same technique as
python? There's been quite a lot of work put into making java fast, so
it'd be interesting if we had something they didn't.

tom

--
What we learn about is not nature itself, but nature exposed to our methods of questioning. -- Werner Heisenberg