rewrite for achieving speedup

**Steve Holden** · Apr 17 '07, 03:25 PM

Re: rewrite for achieving speedup

Johnny Blonde wrote:

Hello Group!
>
I really tried hard for two hours to rewrite the following expression
(python 2.4):
--------------------------
teilnehmer = []
for r in Reisen.select(A ND(Reisen.q.RES VON <= datum, Reisen.q.RESBIS

>= datum)):

for g in r.BUCHUNGEN:
for t in g.aktiveTeilneh mer:
teilnehmer.appe nd(t)
--------------------------
>
to something like
--------------------------
teilnehmer = [x for x in ........]
--------------------------
>
Reisen is a SQLObject class, Reisen.select(. ..), aktiveTeilnehme r and
BUCHUNGEN all are of the type SelectResults.
>
unfortunately i just can´t figure it out to make it work.
i hope someone maybe can help me?
>
I hope to gain performance by rewriting it...
>
Thanks a lot for your help!
>

>>lt = [[[1,2,3], [2,3,4]], [[3,4,5], [4,5,6]]]
>>lf = [c for a in lt for b in a for c in b]
>>lf

[1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6]

>>>

Untested:

teilnehmer = [t for r in Reisen.select(A ND(Reisen.q.RES VON <= datum,
reisen.q.RESBIS >= datum)) for g in r.BUCHUNGEN for t in g.aktiveTeilneh mer]

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

**Steven Bethard** · Apr 17 '07, 03:25 PM

Re: rewrite for achieving speedup

Steve Holden wrote:

Johnny Blonde wrote:

>Hello Group!
>>
>I really tried hard for two hours to rewrite the following expression
>(python 2.4):
>--------------------------
>teilnehmer = []
>for r in Reisen.select(A ND(Reisen.q.RES VON <= datum, Reisen.q.RESBIS

>>= datum)):

> for g in r.BUCHUNGEN:
> for t in g.aktiveTeilneh mer:
> teilnehmer.appe nd(t)
>--------------------------
>>
>to something like
>--------------------------
>teilnehmer = [x for x in ........]
>--------------------------
>>
>Reisen is a SQLObject class, Reisen.select(. ..), aktiveTeilnehme r and
>BUCHUNGEN all are of the type SelectResults.
>>
>unfortunatel y i just can´t figure it out to make it work.
>i hope someone maybe can help me?
>>
>I hope to gain performance by rewriting it...
>>
>Thanks a lot for your help!
>>

>>lt = [[[1,2,3], [2,3,4]], [[3,4,5], [4,5,6]]]
>>lf = [c for a in lt for b in a for c in b]
>>lf

[1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6]

>>>

>
Untested:
>
teilnehmer = [t for r in Reisen.select(A ND(Reisen.q.RES VON <= datum,
reisen.q.RESBIS >= datum)) for g in r.BUCHUNGEN for t in
g.aktiveTeilneh mer]

Note also that you can probably get most of the speedup above by binding
the append method to a function-local name::

teilnehmer = []
append = teilnehmer.appe nd
for r in Reisen.select(. ..):
for g in r.BUCHUNGEN:
for t in g.aktiveTeilneh mer:
append(t)

That's pretty much all a list comprehension is doing anyway.

STeVe

**Johnny Blonde** · Apr 17 '07, 04:25 PM

Re: rewrite for achieving speedup

thanks steve h.,

works like this just perfectly!

steve b.:
for the next time if i cannot figure it out i will just do it like
this!

thanks a lot guys,
Frank

**Jun.Jin.act+group.python@gmail.com** · Apr 18 '07, 01:25 AM

Re: rewrite for achieving speedup

On Apr 17, 11:25 pm, Steven Bethard <steven.beth... @gmail.comwrote :

Steve Holden wrote:

Johnny Blonde wrote:

Hello Group!

>

I really tried hard for two hours to rewrite the following expression
(python 2.4):
--------------------------
teilnehmer = []
for r in Reisen.select(A ND(Reisen.q.RES VON <= datum, Reisen.q.RESBIS
>= datum)):
for g in r.BUCHUNGEN:
for t in g.aktiveTeilneh mer:
teilnehmer.appe nd(t)
--------------------------

>

to something like
--------------------------
teilnehmer = [x for x in ........]
--------------------------

>

Reisen is a SQLObject class, Reisen.select(. ..), aktiveTeilnehme r and
BUCHUNGEN all are of the type SelectResults.

>

unfortunately i just can´t figure it out to make it work.
i hope someone maybe can help me?

>

I hope to gain performance by rewriting it...

>

Thanks a lot for your help!

>

>>lt = [[[1,2,3], [2,3,4]], [[3,4,5], [4,5,6]]]
>>lf = [c for a in lt for b in a for c in b]
>>lf

[1, 2, 3, 2, 3, 4, 3, 4, 5, 4, 5, 6]

>

Untested:

>

teilnehmer = [t for r in Reisen.select(A ND(Reisen.q.RES VON <= datum,
reisen.q.RESBIS >= datum)) for g in r.BUCHUNGEN for t in
g.aktiveTeilneh mer]

>
Note also that you can probably get most of the speedup above by binding
the append method to a function-local name::
>
teilnehmer = []
append = teilnehmer.appe nd
for r in Reisen.select(. ..):
for g in r.BUCHUNGEN:
for t in g.aktiveTeilneh mer:
append(t)
>
That's pretty much all a list comprehension is doing anyway.
>
STeVe

hi steve,
why would binding to a function-local name speeds up performance?
sorry for asking such a simple question. many thanks.

**Alex Martelli** · Apr 18 '07, 01:55 AM

Re: rewrite for achieving speedup

Jun.Jin.act+gro up.python@gmail .com <Jun.Jin.act@gm ail.comwrote:

why would binding to a function-local name speeds up performance?

Like any other constant-hoisting, pulling the lookup out of the loop
speeds things up because otherwise Python must repeat the lookup each
time through the loop (Python doesn't _know_ that, for example, zip.zap
is always the same bound method object -- without potentially deep
analysis, it can't rule out that looking up zap on zip has side effects
and the like, so when you tell it to look it up, it looks it up, no ifs,
no buts).

You can measure the effect with -mtimeit ...:

brain:~ alex$ python -mtimeit -s'def f(L):

for x in xrange(1000): L.append' 'f([])'

1000 loops, best of 3: 215 usec per loop

brain:~ alex$ python -mtimeit -s'def g(L):

ap = L.append
for x in xrange(1000): ap' 'g([])'

10000 loops, best of 3: 79.1 usec per loop

note that in each function f and g I'm doing only the lookup, not the
call to the method thus looked up.

Of course, this depends on the fact that for a function to use a local
variable takes very little time indeed (since the compiler identifies
which variables are local, and transforms any use of their names into a
rapid indexing of a "local variables array", at the time the def
statement executes -- so by the time the function gets called, i.e.,
that its body executes, that execution takes advantage of this
optimization automatically performed by the Python compiler).

Alex

rewrite for achieving speedup

rewrite for achieving speedup

Comment

Comment

Comment

Comment

Comment