Performance impact of using decorators

[vinjvinj]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactr ionAndBuildPage
@partOfTabUi(ta bId)
@convert(arg1=i nt, arg2=str)
def do_main_page(se lf, arg1, arg2):
some code


I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

[Alex Martelli]

Re: Performance impact of using decorators

vinjvinj <vinjvinj@gmail .com> wrote:
[color=blue]
> I'm building an application with cherrypy and have started using
> decorators quite extensively. A lot of my exposed functions look like:
>
> @expose
> @startTransactr ionAndBuildPage
> @partOfTabUi(ta bId)
> @convert(arg1=i nt, arg2=str)
> def do_main_page(se lf, arg1, arg2):
> some code
>
> I've become really fond of decorators and use them quite a lot. I've
> also ready that function calls are expensive in python. In the above
> example, does the interpreter call 5 different functions?[/color]

At def-execution time, presumably 6 (the two decorators w/o args, plus 2
each for those w/args); at call time, it depends what the decorators are
doing (if each adds exactly one wrapping closure, for example, there
will indeed be 5 nested calls). Unfortunately I do not know much of
today's cherrypy internals, so I don't know what each decorator is doing
internally.


Alex

[Steve Holden]

Re: Performance impact of using decorators

vinjvinj wrote:[color=blue]
> I'm building an application with cherrypy and have started using
> decorators quite extensively. A lot of my exposed functions look like:
>
> @expose
> @startTransactr ionAndBuildPage
> @partOfTabUi(ta bId)
> @convert(arg1=i nt, arg2=str)
> def do_main_page(se lf, arg1, arg2):
> some code
>
>
> I've become really fond of decorators and use them quite a lot. I've
> also ready that function calls are expensive in python. In the above
> example, does the interpreter call 5 different functions?
>[/color]

Without reading your code I can't be sure, but this is sure looking like
the classic "to a man with only a hammer all problems look like a nail"
solution. I'm not going to tell you that decorators aren't the answer to
all programming problems, because you already know that in your heart :-)

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogs pot.com

[vinjvinj]

Re: Performance impact of using decorators

>> solution. I'm not going to tell you that decorators aren't the answer to[color=blue][color=green]
>>all programming problems, because you already know that in your heart :-[/color][/color]

I was fearing that. The expose decorator is the only one that comes
with cherrypy. The other ones are mine and are of the format:

def decorator(func) :
def wrapper(self, *args, **kwargs)
some code
return wrapper

I'll stick with what I'm doing currently and eventually (if the need
arises from a performance perspective) merge the three into one
decorator. I'll also need to eventually add a caching docrator.Do other
people have this problem, especially for developing web applications.
How many decorators, if any, do you use?

@expose -> cherrypy decorator
@startTransactr ionAndBuildPage -> starts a db transaction, populates
the user in the session. Does some error handling. Adds header, footer
and error messages to the page.
@partOfTabUi -> besides the top level navigation, I have tab level
(with actions) for navigation on individual pages
@convert -> this converts boolean like 'True' or '0' to python True,
'231' -> int
@cache -> (not implemented, but will ad it).

I would love to hear other people's experience with using decorators
for web application building.

[Diez B. Roggisch]

Re: Performance impact of using decorators

> @expose -> cherrypy decorator[color=blue]
> @startTransactr ionAndBuildPage -> starts a db transaction, populates
> the user in the session.[/color]

I guess that is ok - transaction handling is a "classic" for decorator-like
concepts. After all, you don't want

begin()
try:
pass
commit()
finally:
if not comitted():
rollback()

all over the place.
[color=blue]
> Does some error handling. Adds header, footer
> and error messages to the page.[/color]

That sounds like something for the templating engine, and _certainly_ not
for a decorator that otherwise deals with transactions.
[color=blue]
> @partOfTabUi -> besides the top level navigation, I have tab level
> (with actions) for navigation on individual pages[/color]

Template I guess.
[color=blue]
> @convert -> this converts boolean like 'True' or '0' to python True,
> '231' -> int[/color]

Looks ok to me.
[color=blue]
> @cache -> (not implemented, but will ad it).
>
> I would love to hear other people's experience with using decorators
> for web application building.[/color]

I used them in turbogears (which builds on cherrypy) - and I found them
useful for some aspects.

Diez

[Fredrik Lundh]

Re: Performance impact of using decorators

"vinjvinj" wrote:
[color=blue]
> I'm building an application with cherrypy and have started using
> decorators quite extensively. A lot of my exposed functions look like:
>
> @expose
> @startTransactr ionAndBuildPage
> @partOfTabUi(ta bId)
> @convert(arg1=i nt, arg2=str)
> def do_main_page(se lf, arg1, arg2):
> some code
>
> I've become really fond of decorators and use them quite a lot. I've
> also ready that function calls are expensive in python. In the above
> example, does the interpreter call 5 different functions?[/color]

the decorators themselves are only called when the function is defined.

what happens at runtime depends on what the decorators do (in pretty
much the same way as the output and execution time for this script

x = lambda: return "hello"
x = foo(x)
x = fie(x)
x = fum(x)
print x()

depends on what the foo, fie, and fum functions do...)

</F>

[vinjvinj]

Re: Performance impact of using decorators

>>That sounds like something for the templating engine, and _certainly_ not[color=blue][color=green]
>>for a decorator that otherwise deals with transactions.[/color][/color]

The actual code for the page layout is in a preppy template. But the
calls to the template engine are made in the
startTransactri onAndBuildPage decorator
[color=blue][color=green]
>>Template I guess.[/color][/color]

Seemed like an overkill for the template engine.

[Fredrik Lundh]

Re: Performance impact of using decorators

"vinjvinj" wrote:
[color=blue]
> I was fearing that. The expose decorator is the only one that comes
> with cherrypy. The other ones are mine and are of the format:
>
> def decorator(func) :
> def wrapper(self, *args, **kwargs)
> some code
> return wrapper[/color]

what exactly made you think that Python would be able to run your
code *without* calling your function ?

</F>

[Terry Reedy]

Re: Performance impact of using decorators


"vinjvinj" <vinjvinj@gmail .com> wrote in message
news:1142003515 .886709.290360@ j33g2000cwa.goo glegroups.com.. .[color=blue]
> I'm building an application with cherrypy and have started using
> decorators quite extensively. A lot of my exposed functions look like:
>
> @expose
> @startTransactr ionAndBuildPage
> @partOfTabUi(ta bId)
> @convert(arg1=i nt, arg2=str)
> def do_main_page(se lf, arg1, arg2):
> some code
>
>
> I've become really fond of decorators and use them quite a lot. I've
> also ready that function calls are expensive in python. In the above
> example, does the interpreter call 5 different functions?[/color]

As Alex said, perhaps. A decorator that would not result in a runtime call
would be one that, for example, registers the function somewhere and
returns it unchanged and unwrapped. @expose and @partOfTabUI both seem
like they might do something like that.

IF the overhead becomes a problem, and if you use the same stack (or parts
of a stack) of decorators for multiple functions, then you could write a
few decorators that combine multiple actions with one call.

Terry Jan Reedy

[vinjvinj]

Re: Performance impact of using decorators

>>what exactly made you think that Python would be able to run your[color=blue][color=green]
>>code *without* calling your function ?[/color][/color]

I was hoping that when the compiler finds decorators with wrapers that
have the same signature it can some how "magically" combine them into
one function (which gets called at run time) and not have 5 nested
function calls. That is similar to what I will have to do eventually.

Given python's dynamic nature, I'm sure there are reasons why this is
not done.

[Diez B. Roggisch]

Re: Performance impact of using decorators

vinjvinj schrieb:[color=blue][color=green][color=darkred]
>>> That sounds like something for the templating engine, and _certainly_ not
>>> for a decorator that otherwise deals with transactions.[/color][/color]
>
> The actual code for the page layout is in a preppy template. But the
> calls to the template engine are made in the
> startTransactri onAndBuildPage decorator[/color]

Well, even if it would fit in a decorator - it certainly belongs to its
_own_ decorator.


Diez

[Alex Martelli]

Re: Performance impact of using decorators

Diez B. Roggisch <deets@nospam.w eb.de> wrote:
...[color=blue]
> begin()
> try:
> pass
> commit()
> finally:
> if not comitted():
> rollback()[/color]

Feels like a natural for 2.5's 'with' statement -- as has been the case
for 2.3 and 2.4, 2.5 won't have many language-level changes, but what
little there IS, is... wonderful!

with transaction():
...your code goes here...

is SO much better than the try/finally routine...!


Alex

[Alex Martelli]

Re: Performance impact of using decorators

vinjvinj <vinjvinj@gmail .com> wrote:
[color=blue][color=green][color=darkred]
> >>what exactly made you think that Python would be able to run your
> >>code *without* calling your function ?[/color][/color]
>
> I was hoping that when the compiler finds decorators with wrapers that
> have the same signature it can some how "magically" combine them into
> one function (which gets called at run time) and not have 5 nested
> function calls. That is similar to what I will have to do eventually.
>
> Given python's dynamic nature, I'm sure there are reasons why this is
> not done.[/color]

Yep, you'll have to build new functions yourself if you need them:-(


Alex

[Peter Otten]

Re: Performance impact of using decorators

vinjvinj wrote:
[color=blue]
> I'm building an application with cherrypy and have started using
> decorators quite extensively. A lot of my exposed functions look like:
>
> @expose
> @startTransactr ionAndBuildPage
> @partOfTabUi(ta bId)
> @convert(arg1=i nt, arg2=str)
> def do_main_page(se lf, arg1, arg2):
> some code
>
>
> I've become really fond of decorators and use them quite a lot. I've
> also ready that function calls are expensive in python. In the above
> example, does the interpreter call 5 different functions?[/color]

A typical function calls a few other functions already, so three extra
function calls (I suppose expose just sets an attribute) shouldn't matter
much. You shouldn't even start rewriting your code unless you have
identified do_main_page() as a performance bottleneck. In a web app,
candidates would be functions that are called hundred or thousand times per
rendered page rather than once. Does do_main_page() render a complete page?
Forget optimizing three function calls away. You will see no effect.

Peter