How do I dynamically create functions without lambda?

[Russell]

I want my code to be Python 3000 compliant, and hear
that lambda is being eliminated. The problem is that I
want to partially bind an existing function with a value
"foo" that isn't known until run-time:

someobject.newf unc = lambda x: f(foo, x)

The reason a nested function doesn't work for this is
that it is, well, dynamic. I don't know how many times
or with what foo's this will be done.

Now, I am sure there are a half-dozen ways to do this.
I just want the one, new and shiny, Pythonic way. ;-)

[Robert Kern]

Re: How do I dynamically create functions without lambda?

Russell wrote:[color=blue]
> I want my code to be Python 3000 compliant, and hear
> that lambda is being eliminated. The problem is that I
> want to partially bind an existing function with a value
> "foo" that isn't known until run-time:
>
> someobject.newf unc = lambda x: f(foo, x)
>
> The reason a nested function doesn't work for this is
> that it is, well, dynamic. I don't know how many times
> or with what foo's this will be done.
>
> Now, I am sure there are a half-dozen ways to do this.
> I just want the one, new and shiny, Pythonic way. ;-)[/color]

It doesn't exist, yet. Python 3000 isn't even in planning stages, yet. There are
just some loose ideas floating around about what will (and won't!) be in it.

You can't write Python 3000 compliant code right now because there is nothing to
comply with.

--
Robert Kern
robert.kern@gma il.com

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Paul Rubin]

Re: How do I dynamically create functions without lambda?

"Russell" <russell_turpin @hotmail.com> writes:[color=blue]
> I want my code to be Python 3000 compliant, and hear
> that lambda is being eliminated.[/color]

Nobody knows yet what Python 3000 will change, so relax.

[Kay Schluehr]

Re: How do I dynamically create functions without lambda?


Russell wrote:[color=blue]
> I want my code to be Python 3000 compliant, and hear
> that lambda is being eliminated. The problem is that I
> want to partially bind an existing function with a value
> "foo" that isn't known until run-time:
>
> someobject.newf unc = lambda x: f(foo, x)
>
> The reason a nested function doesn't work for this is
> that it is, well, dynamic. I don't know how many times
> or with what foo's this will be done.
>
> Now, I am sure there are a half-dozen ways to do this.
> I just want the one, new and shiny, Pythonic way. ;-)[/color]

If you want to code partial application without lambda I recommend
using the code presented in the accepted PEP 309 that will be
implemented in the functional module in Python 2.5.

http://www.python.org/peps/pep-0309.html

Kay

[James Stroud]

Re: How do I dynamically create functions without lambda?

Russell wrote:[color=blue]
> I want my code to be Python 3000 compliant, and hear
> that lambda is being eliminated. The problem is that I
> want to partially bind an existing function with a value
> "foo" that isn't known until run-time:
>
> someobject.newf unc = lambda x: f(foo, x)
>
> The reason a nested function doesn't work for this is
> that it is, well, dynamic. I don't know how many times
> or with what foo's this will be done.
>
> Now, I am sure there are a half-dozen ways to do this.
> I just want the one, new and shiny, Pythonic way. ;-)
>[/color]

Is this what you mean?

# making functions partially bound with foo, dynamically

def make_newfunc(fo o):
def _newfunc(x, f=foo):
print x, foo
return _newfunc

foo = 42

newfunc = make_newfunc(fo o)

newfunc()

[James Stroud]

Re: How do I dynamically create functions without lambda?

Kay Schluehr wrote:[color=blue]
> Russell wrote:
>[color=green]
>>I want my code to be Python 3000 compliant, and hear
>>that lambda is being eliminated. The problem is that I
>>want to partially bind an existing function with a value
>>"foo" that isn't known until run-time:
>>
>> someobject.newf unc = lambda x: f(foo, x)
>>
>>The reason a nested function doesn't work for this is
>>that it is, well, dynamic. I don't know how many times
>>or with what foo's this will be done.
>>
>>Now, I am sure there are a half-dozen ways to do this.
>>I just want the one, new and shiny, Pythonic way. ;-)[/color]
>
>
> If you want to code partial application without lambda I recommend
> using the code presented in the accepted PEP 309 that will be
> implemented in the functional module in Python 2.5.
>
> http://www.python.org/peps/pep-0309.html
>
> Kay
>[/color]
For anyone who got my last post: sorry, typos....


def f(foo, x):
print foo, x

def make_newfunc(fo o):
def _newfunc(x, foo=foo):
f(foo, x)
return _newfunc

foo = 42 # or "dynamicall y generated"

newfunc = make_newfunc(fo o)

newfunc(14) # output will be "42 14"

newfunc2 = make_newfunc(69 )

newfunc(21) # output will be "69 21"


If this doesn't fit your needs, then please elucidate.

[Gary Herron]

Re: How do I dynamically create functions without lambda?

Russell wrote:
[color=blue]
>I want my code to be Python 3000 compliant, and hear
>that lambda is being eliminated. The problem is that I
>want to partially bind an existing function with a value
>"foo" that isn't known until run-time:
>
> someobject.newf unc = lambda x: f(foo, x)
>
>The reason a nested function doesn't work for this is
>that it is, well, dynamic. I don't know how many times
>or with what foo's this will be done.
>
>[/color]
That's nonsense! A lambda function *IS* a nested function, and has *NO*
extra capabilities over a nested function. If you can do something with
a lambda, you can also do it with a nested function defined at the same
point. However, a nested function *does* give you several extra
capabilities: (1) it has a name rather than being anonymous like the
lambda, and (2) the body can use statements rather just one expression.

So use a nested function. You'll get the same capabilities, plus a
name, plus more expressive power in the body. You won't lose anything.
and your code will survive any eventual removal of the lambda functionality.
[color=blue]
>Now, I am sure there are a half-dozen ways to do this.
>I just want the one, new and shiny, Pythonic way. ;-)
>
>
>[/color]

[Steven Bethard]

Re: How do I dynamically create functions without lambda?

Russell wrote:[color=blue]
> I want my code to be Python 3000 compliant, and hear
> that lambda is being eliminated. The problem is that I
> want to partially bind an existing function with a value
> "foo" that isn't known until run-time:
>
> someobject.newf unc = lambda x: f(foo, x)
>
> The reason a nested function doesn't work for this is
> that it is, well, dynamic. I don't know how many times
> or with what foo's this will be done.[/color]

I don't understand this argument here. The code above is almost exactly
equivalent to:

def newfunc(x):
return f(foo, x)
someobject.newf unc = newfunc

Observe:
[color=blue][color=green][color=darkred]
>>> class C(object):[/color][/color][/color]
.... pass
....[color=blue][color=green][color=darkred]
>>> someobject = C()
>>> someobject.newf unc = lambda x: f(foo, x)
>>> import dis
>>> dis.dis(someobj ect.newfunc)[/color][/color][/color]
1 0 LOAD_GLOBAL 0 (f)
3 LOAD_GLOBAL 1 (foo)
6 LOAD_FAST 0 (x)
9 CALL_FUNCTION 2
12 RETURN_VALUE[color=blue][color=green][color=darkred]
>>> def newfunc(x):[/color][/color][/color]
.... return f(foo, x)
....[color=blue][color=green][color=darkred]
>>> someobject.newf unc = newfunc
>>> dis.dis(someobj ect.newfunc)[/color][/color][/color]
2 0 LOAD_GLOBAL 0 (f)
3 LOAD_GLOBAL 1 (foo)
6 LOAD_FAST 0 (x)
9 CALL_FUNCTION 2
12 RETURN_VALUE

Note that both the lambda and the function have exactly the same
byte-code. The only real difference is that if you use a def-statement
instead of a lambda, your function will get a real name, "newfunc",
instead of <lambda>.

STeVe

[Steven D'Aprano]

Re: How do I dynamically create functions without lambda?

On Fri, 27 Jan 2006 11:41:56 -0800, Kay Schluehr wrote:
[color=blue]
>
> Russell wrote:[color=green]
>> I want my code to be Python 3000 compliant, and hear
>> that lambda is being eliminated. The problem is that I
>> want to partially bind an existing function with a value
>> "foo" that isn't known until run-time:
>>
>> someobject.newf unc = lambda x: f(foo, x)
>>
>> The reason a nested function doesn't work for this is
>> that it is, well, dynamic. I don't know how many times
>> or with what foo's this will be done.
>>
>> Now, I am sure there are a half-dozen ways to do this.
>> I just want the one, new and shiny, Pythonic way. ;-)[/color]
>
> If you want to code partial application without lambda I recommend
> using the code presented in the accepted PEP 309 that will be
> implemented in the functional module in Python 2.5.
>
> http://www.python.org/peps/pep-0309.html[/color]


Fascinating. A couple of thoughts:

- I really wish that people would make up their minds about what currying
is, and how it differs from partials and closures. If the people who
do know can't agree, how do they expect the rest of us to understand?

- What, if anything, is the difference between a closure and an iterator?
Is an iterator just a closure wrapped up with an API?

- The first sample code seems needlessly confusing to me. It gives:

class partial(object) :
def __init__(*args, **kw):
self = args[0]
self.fn, self.args, self.kw = (args[1], args[2:], kw)
def __call__(self, *args, **kw):
if kw and self.kw:
d = self.kw.copy()
d.update(kw)
else:
d = kw or self.kw
return self.fn(*(self. args + args), **d)

It seems to me to needlessly break the convention that the first argument
is self, to no benefit and considerable reduction in clarity. After some
experimentation , I worked out what it was doing, and realised that it
would work just as well but much less obscurely if the __init__ function
was written as:

def __init__(self, fn, *args, **kw):
self.fn, self.args, self.kw = fn, args, kw


- It seems a shame to me that having created a partial _function_ using
that technique, type(partial(.. .)) returns <class partial>. It would be
nicer if the type made it more obvious that the instance was callable.
Something like <callable class partial> perhaps? Is there any way for a
class to customise the type representation?



--
Steven.

[Scott David Daniels]

Re: How do I dynamically create functions without lambda?

Steven D'Aprano wrote:[color=blue]
> On Fri, 27 Jan 2006 11:41:56 -0800, Kay Schluehr wrote:
>[color=green]
>> Russell wrote:[color=darkred]
>>> I want my code to be Python 3000 compliant, and hear
>>> that lambda is being eliminated. The problem is that I
>>> want to partially bind an existing function with a value
>>> "foo" that isn't known until run-time:
>>>
>>> someobject.newf unc = lambda x: f(foo, x)
>>>
>>> The reason a nested function doesn't work for this is
>>> that it is, well, dynamic. I don't know how many times
>>> or with what foo's this will be done.
>>>
>>> Now, I am sure there are a half-dozen ways to do this.
>>> I just want the one, new and shiny, Pythonic way. ;-)[/color]
>> If you want to code partial application without lambda I recommend
>> using the code presented in the accepted PEP 309 that will be
>> implemented in the functional module in Python 2.5.
>>
>> http://www.python.org/peps/pep-0309.html[/color]
>
>
> Fascinating. A couple of thoughts:
>
> - I really wish that people would make up their minds about what currying
> is, and how it differs from partials and closures. If the people who
> do know can't agree, how do they expect the rest of us to understand?[/color]

Me too, but I disagree with the people who won the argument on the name.
Curry derives its name from the mathematician who formulated the
Curry-Howard isomorphism that says a function of multiple arguments
can be expressed as a function taking a single argument which returns a
function which takes a single argument which... takes a single argument
and returns the result of the original multi-argument function. This
isomorphism means that you need only define functions which take single
arguments. In some functional languages, functions are called adjacent
to their arguments, and a function which looks like it takes multiple
arguments is actually taking a single argument of a tuple.

The Curry recipe I put in the Python Cookbook did the analogous thing
for the Python language, since there is no way for a "Curry" argument
to know when to call the underlying function, rather than simply
accumulate arguments.
[color=blue]
> - What, if anything, is the difference between a closure and an iterator?
> Is an iterator just a closure wrapped up with an API?[/color]

A closure is a function and the environment it was created in.
So, for example, the result of calling:

def multiplier(n):
def result(m):
return n * m

multiplier(2) returns a closure (the function called result and its
environment where n is set to 2).

An iterator is a different beasty.
[color=blue]
> - The first sample code seems needlessly confusing to me. It gives:
>
> class partial(object) :
> def __init__(*args, **kw):
> self = args[0]
> self.fn, self.args, self.kw = (args[1], args[2:], kw)
> def __call__(self, *args, **kw):
> if kw and self.kw:
> d = self.kw.copy()
> d.update(kw)
> else:
> d = kw or self.kw
> return self.fn(*(self. args + args), **d)
>
> It seems to me to needlessly break the convention that the first argument
> is self.[/color]
Actually, it does, but only because the second doesn't break it as well.
[color=blue]
> to no benefit and considerable reduction in clarity. After some
> experimentation , I worked out what it was doing, and realised that it
> would work just as well but much less obscurely if the __init__ function
> was written as:
>
> def __init__(self, fn, *args, **kw):
> self.fn, self.args, self.kw = fn, args, kw[/color]

It would pretty much work like this, _but_ you could not used named args
of self or fn when using one of these things.

A better definition:
class partial(object) :
def __init__(*args, **kw):
self = args[0]
self.fn, self.args, self.kw = (args[1], args[2:], kw)
def __call__(*args, **kw):
self = args[0]
if kw and self.kw:
d = self.kw.copy()
d.update(kw)
else:
d = kw or self.kw
return self.fn(*(self. args + args[1:]), **d)

Now you can, for example:

def function(self, other, fn):
return fn([self, other])

defaults = partial(functio n, self=object(), fn=repr)
print defaults(other= 3), defaults(self=' self', other=3)

The fancy-schmancy stuff is to keep from hiding some names. I suspect
both args and kw have this problem as well. It might have been
reasonable to call them __self, __function, __args, and __kwargs
and do it your way.
[color=blue]
> - It seems a shame to me that having created a partial _function_ using
> that technique, type(partial(.. .)) returns <class partial>. It would be
> nicer if the type made it more obvious that the instance was callable.[/color]
But callable(defaul ts) returns True. You will eventually learn that
partial is callable, just as you know "type"s are callable. Seeing
that it is class partial, you can (if you are nosey), see what the
arguments already provided are (so you could get a nicer print).
For example, adding a method to partial:

def __repr__(self):
return 'partial(%r, *%r, **%r)' % (self.fn, self.args, self.kw)
[color=blue]
> Something like <callable class partial> perhaps? Is there any way for a
> class to customise the type representation?[/color]

--
-Scott David Daniels
scott.daniels@a cm.org

[Kay Schluehr]

Re: How do I dynamically create functions without lambda?

Steven D'Aprano wrote:[color=blue]
> On Fri, 27 Jan 2006 11:41:56 -0800, Kay Schluehr wrote:
>[color=green]
> >
> > Russell wrote:[color=darkred]
> >> I want my code to be Python 3000 compliant, and hear
> >> that lambda is being eliminated. The problem is that I
> >> want to partially bind an existing function with a value
> >> "foo" that isn't known until run-time:
> >>
> >> someobject.newf unc = lambda x: f(foo, x)
> >>
> >> The reason a nested function doesn't work for this is
> >> that it is, well, dynamic. I don't know how many times
> >> or with what foo's this will be done.
> >>
> >> Now, I am sure there are a half-dozen ways to do this.
> >> I just want the one, new and shiny, Pythonic way. ;-)[/color]
> >
> > If you want to code partial application without lambda I recommend
> > using the code presented in the accepted PEP 309 that will be
> > implemented in the functional module in Python 2.5.
> >
> > http://www.python.org/peps/pep-0309.html[/color]
>
>
> Fascinating. A couple of thoughts:
>
> - I really wish that people would make up their minds about what currying
> is, and how it differs from partials and closures. If the people who
> do know can't agree, how do they expect the rest of us to understand?[/color]

If they can't agree they probably don't know ;)

Concepts like "closure" and "currying" are well defined in lambda
calculus and hence in functional programming.
[color=blue]
> - What, if anything, is the difference between a closure and an iterator?
> Is an iterator just a closure wrapped up with an API?[/color]

I would expect a closure to be a function with free/unbound variables
that are bound by an enclosing context. In FP slang a function without
any free variables is called a "combinator ". An iterator is a function
related to an abstract data type equipped with a partial order. The
iterator abstracts from the particular ordering scheme and makes the
ADT look like a list. In Python we have a nice correspondence between
generators as implicit defined sequences of yielded values, iterators
that enable uniform access to the values produced by generators and
lists that make the sequence explicit ( if finite ).

Fun with combinators
=============== =======
There are quite interesting relationships deep down in the theory of
computation. For instance understanding recursion in lambda calculus is
not that easy because you cannot simply reintroduce an anonymous
function by the name into its own body. The idea is to create a
fixpoint of a higher order curried function instead. See for example
this curried extension of a factorial:

F = lambda h: lambda n : n<2 and 1 or n*h(n-1)

Now imagine you have a fixpoint fix(F) of F which is again a function
and pass it into F then you will get:

fix(F) = F(fix(F)) = lambda n : n<2 and 1 or n*fix(F)(n-1)

That is fix(F) is actually the searched factorial ! There is a generic
way to create fixpoints of higher order curried functions such as F.
The most popular one is the so called Y-combinator ( not to confuse
with Paul Grahams company ;).

In Python you can write:

Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
arg: f(f)(arg)))

This serves the purpose. Try Y(F) and see.

Kay

[Steven D'Aprano]

Re: How do I dynamically create functions without lambda?

On Sat, 28 Jan 2006 00:13:28 -0800, Kay Schluehr wrote:

[snip lambda calculus stuff]
[color=blue]
> In Python you can write:
>
> Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
> arg: f(f)(arg)))
>
> This serves the purpose. Try Y(F) and see.[/color]


Is any of this stuff maintainable in the real world of IT, where
most programmers don't have computer science degrees? You come along six
months after the project was finished to maintain this code and discover
that the whiz-kid lambda calculus guy never commented anything because
that would detract from the elegance of his one liners; what happens next?



--
Steven.

[Kay Schluehr]

Re: How do I dynamically create functions without lambda?

Steven D'Aprano wrote:[color=blue]
> On Sat, 28 Jan 2006 00:13:28 -0800, Kay Schluehr wrote:
>
> [snip lambda calculus stuff]
>[color=green]
> > In Python you can write:
> >
> > Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
> > arg: f(f)(arg)))
> >
> > This serves the purpose. Try Y(F) and see.[/color]
>
>
> Is any of this stuff maintainable in the real world of IT, where
> most programmers don't have computer science degrees?[/color]

Probably not. But the good thing about Y is that it is in closed form
and the expression is not infinitely long. By the way I have no less a
hard time to read C code with advanced data-structures and many type
casts which is much more likely to happen in the "real world".
[color=blue]
>From what I've seen the only *practical* purpose fixpoint combinators[/color]
serve is a kind of recursion "overloadin g". Without modifying F one can
memoize values by adapting the fixpoint combinator. We already know
something similar from the Go4 "command pattern" but there is no
self-referential entanglement and it is less general.
[color=blue]
> You come along six
> months after the project was finished to maintain this code and discover
> that the whiz-kid lambda calculus guy never commented anything because
> that would detract from the elegance of his one liners; what happens next?[/color]

Ask a Scheme guy at LtU. Accuse the original author in Den Haag and if
caught put him to shame by making photos at a dentist. Google for
"lambda calculus Y". Ask someone here at comp.lang.pytho n. Ask your
boss for more time ;)

Kay

[Antoon Pardon]

Re: How do I dynamically create functions without lambda?

Op 2006-01-27, Russell schreef <russell_turpin @hotmail.com>:[color=blue]
> I want my code to be Python 3000 compliant, and hear
> that lambda is being eliminated. The problem is that I
> want to partially bind an existing function with a value
> "foo" that isn't known until run-time:
>
> someobject.newf unc = lambda x: f(foo, x)
>
> The reason a nested function doesn't work for this is
> that it is, well, dynamic. I don't know how many times
> or with what foo's this will be done.[/color]

I don't see how dynamic is supposed to contradict with
nested functions. I assume from your tekst that foo
has some parameter-like charateristics. So the following
should work.

def produce(foo):

def func(x):
return f(foo, x)

return func


someobject.newf unc = produce(foo)


This kind of code can be generalized. Look for partial
application or curry.

[color=blue]
> Now, I am sure there are a half-dozen ways to do this.
> I just want the one, new and shiny, Pythonic way. ;-)[/color]

The new and shiny, pythonic way depends on what you
really want, your question was a bit vague to answer
that.

--
Antoon pardon

[Alan Morgan]

Re: How do I dynamically create functions without lambda?

In article <pan.2006.01.28 .12.02.34.29328 6@REMOVETHIScyb er.com.au>,
Steven D'Aprano <steve@REMOVETH IScyber.com.au> wrote:[color=blue]
>On Sat, 28 Jan 2006 00:13:28 -0800, Kay Schluehr wrote:
>
>[snip lambda calculus stuff]
>[color=green]
>> In Python you can write:
>>
>> Y = lambda g: (lambda f: g(lambda arg: f(f)(arg))) (lambda f: g(lambda
>> arg: f(f)(arg)))
>>
>> This serves the purpose. Try Y(F) and see.[/color]
>
>
>Is any of this stuff maintainable in the real world of IT, where
>most programmers don't have computer science degrees? You come along six
>months after the project was finished to maintain this code and discover
>that the whiz-kid lambda calculus guy never commented anything because
>that would detract from the elegance of his one liners; what happens next?[/color]

Excessive cleverness can lead to unmaintainable code. So can excessive stupidity.
Since there are a lot more stupid people than clever people out there I think
the more likely scenario is having to maintain unmaintainable code written by a
complete idiot whose programming knowledge comes solely from books whose titles
end with "In 7 Days".

Oh, and I'd hope that code reviews, etc. would have kept lambda-boy from getting
too far down this path of self invocation.

Alan
--
Defendit numerus