Function application optimization.

[Jacek Generowicz]

Given

fncs = [func1, func2, ..., funcN]
args = [arg1, arg2, ..., argN]

How should one spell

results = map(lambda f,a: f(a), fncs, args)

in order to get the result most quickly ?



Unfortunately "apply" takes a tuple of arguments, and there is no
"funcall"[*] in Python.

[*] def funcall(fn, *args):
return fn(*args)

[Bengt Richter]

Re: Function application optimization.

On 12 Dec 2003 10:47:50 +0100, Jacek Generowicz <jacek.generowi cz@cern.ch> wrote:
[color=blue]
>Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
>How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
>in order to get the result most quickly ?
>
>
>
>Unfortunatel y "apply" takes a tuple of arguments, and there is no
>"funcall"[*] in Python.
>
>
>[*] def funcall(fn, *args):
> return fn(*args)
>[/color]
[color=blue][color=green][color=darkred]
>>> fncs = [lambda x,f='func_%s(%% s)'%i:f%x for i in xrange(4)]
>>> args = 'zero one two three'.split()
>>> map(lambda f,a: f(a), fncs, args)[/color][/color][/color]
['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']

I'd probably try a list comprehension
[color=blue][color=green][color=darkred]
>>> [f(a) for f,a in zip(fncs,args)][/color][/color][/color]
['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']

Regards,
Bengt Richter

[Duncan Booth]

Re: Function application optimization.

Jacek Generowicz <jacek.generowi cz@cern.ch> wrote in
news:tyf7k128j0 9.fsf@pcepsft00 1.cern.ch:
[color=blue]
> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>[/color]
Well, the way you wrote it isn't actually bad. The obvious alternative
(using a list comprehension) is slightly slower, but there isn't an awful
lot to choos between any of them, I suspect most of the time goes in the
actual f(a) function call. Any of these methods runs at about 300,000
function calls/second on my slow laptop.
[color=blue][color=green][color=darkred]
>>> setup = """import itertools[/color][/color][/color]
def fn1(a): pass
fns = [fn1] * 100
args = [0] * 100
"""[color=blue][color=green][color=darkred]
>>> stmt1 = "result = [ f(a) for (f,a) in itertools.izip( fns, args) ]"
>>> stmt2 = "result = [ f(a) for (f,a) in zip(fns, args) ]"
>>> t = timeit.Timer(st mt1, setup)
>>> t.repeat(3,1000 0)[/color][/color][/color]
[3.5571303056673 855, 3.5537404893639 177, 3.5594278043718 077][color=blue][color=green][color=darkred]
>>> t = timeit.Timer(st mt2, setup)
>>> t.repeat(3,1000 0)[/color][/color][/color]
[3.8932819674008 67, 3.8783479464568 7, 3.8829105375124 868][color=blue][color=green][color=darkred]
>>> setup = """import itertools[/color][/color][/color]
def fn1(a): pass
fns = [fn1] * 1000
args = [0] * 1000
"""[color=blue][color=green][color=darkred]
>>> t = timeit.Timer(st mt1, setup)
>>> t.repeat(3,1000 )[/color][/color][/color]
[3.3503928571927 304, 3.3343195853104 248, 3.3495254285111 287][color=blue][color=green][color=darkred]
>>> t = timeit.Timer(st mt2, setup)
>>> t.repeat(3,1000 )[/color][/color][/color]
[3.8062683944467 608, 3.7946001516952 492, 3.7881063096007 779][color=blue][color=green][color=darkred]
>>> stmt3 = "results = map(lambda f,a: f(a), fns, args)"
>>> t = timeit.Timer(st mt3, setup)
>>> t.repeat(3,1000 )[/color][/color][/color]
[3.3275902384241 363, 3.3010907810909 202, 3.3174872784110 789]

The f(a) call is taking about half the time in any of these methods, so you
aren't going to get very much improvement whatever you do to the loop.

--
Duncan Booth duncan@rcp.co.u k
int month(char *p){return(1248 64/((p[0]+p[1]-p[2]&0x1f)+1)%12 )["\5\x8\3"
"\6\7\xb\1\x9\x a\2\0\4"];} // Who said my code was obscure?

[Bruno Desthuilliers]

Re: Function application optimization.

Jacek Generowicz wrote:[color=blue]
> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>[/color]

I dont know if it will be faster (timeit might tell you this), but what
about something as simple stupid as:

results = []
nbfuns = len(fncs)
for i in range(len):
results.append( fncs[i](args[i]))

Note that you must have at least len(fncs) args.


HTH,
Bruno

[Jacek Generowicz]

Re: Function application optimization.

bokr@oz.net (Bengt Richter) writes:
[color=blue]
> I'd probably try a list comprehension
>[color=green][color=darkred]
> >>> [f(a) for f,a in zip(fncs,args)][/color][/color]
> ['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)'][/color]

Yup, been there, done that, it's slower. (I guess I should have
mentioned that in the original post.)

[Peter Otten]

Re: Function application optimization.

Jacek Generowicz wrote:
[color=blue]
> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>[/color]

If you can afford to destroy the args list, a tiny speed gain might be in
for you (not verified):

for index, func in enumerate(fncs) :
args[index] = func[index]

Peter

[John Roth]

Re: Function application optimization.


"Jacek Generowicz" <jacek.generowi cz@cern.ch> wrote in message
news:tyf7k128j0 9.fsf@pcepsft00 1.cern.ch...[color=blue]
> Given
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
>
> How should one spell
>
> results = map(lambda f,a: f(a), fncs, args)
>
> in order to get the result most quickly ?
>
>
>
> Unfortunately "apply" takes a tuple of arguments, and there is no
> "funcall"[*] in Python.
>
>
>[*] def funcall(fn, *args):
> return fn(*args)[/color]


Building on a couple of other responses:

Untested code:

fncs = [func1, func2, ..., funcN]
args = [arg1, arg2, ..., argN]
results = []
for function, arguement in zip(fncs, args):
results.append( function(arguem ent))

Notice the use of zip() to put the two lists together.
I haven't timed it, but removing an extra layer of
function call has got to be faster. Likewise, zip
and tuple unpacking is most likely going to be
faster than indexing every time through the loop.
The append, on the other hand, might slow things
down a bit.

John Roth
[color=blue]
>[/color]

[Duncan Booth]

Re: Function application optimization.

"John Roth" <newsgroups@jhr othjr.com> wrote in
news:vtjhmt7idn 3k6b@news.super news.com:
[color=blue]
> Building on a couple of other responses:
>
> Untested code:
>
> fncs = [func1, func2, ..., funcN]
> args = [arg1, arg2, ..., argN]
> results = []
> for function, arguement in zip(fncs, args):
> results.append( function(arguem ent))
>
> Notice the use of zip() to put the two lists together.
> I haven't timed it, but removing an extra layer of
> function call has got to be faster. Likewise, zip
> and tuple unpacking is most likely going to be
> faster than indexing every time through the loop.
> The append, on the other hand, might slow things
> down a bit.[/color]

Yes, getting rid of the append does indeed speed things up. On the same
system as I posted timings for the list comprehension, the fastest way I've
found so far is to get rid of the appends by preallocating the list, and to
go for the plain old simple technique of writing a for loop out explicitly.
Using 'enumerate' to avoid the lookup on one of the input lists is nearly
as fast, but not quite.
[color=blue][color=green][color=darkred]
>>> setup = """import itertools[/color][/color][/color]
def fn1(a): pass
fns = [fn1] * 1000
args = [0] * 1000
"""
[color=blue][color=green][color=darkred]
>>> stmt1 = """result = args[:][/color][/color][/color]
for i in range(len(fns)) :
result[i] = fns[i](args[i])
"""[color=blue][color=green][color=darkred]
>>> min(timeit.Time r(stmt1, setup).repeat(3 ,1000))[/color][/color][/color]
2.9747384094916 924[color=blue][color=green][color=darkred]
>>> stmt3 = "results = map(lambda f,a: f(a), fns, args)"
>>> min(timeit.Time r(stmt3, setup).repeat(3 ,1000))[/color][/color][/color]
3.3257092731055 309[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

--
Duncan Booth duncan@rcp.co.u k
int month(char *p){return(1248 64/((p[0]+p[1]-p[2]&0x1f)+1)%12 )["\5\x8\3"
"\6\7\xb\1\x9\x a\2\0\4"];} // Who said my code was obscure?

[Peter Otten]

Re: Function application optimization.

Peter Otten wrote:
[color=blue]
> for index, func in enumerate(fncs) :
> args[index] = func[index][/color]

Oops,
args[index] = func(args[index])

And it's much slower than result.append(f unc(args[index]), too :-(

Peter