Lambda going out of fashion

[Alex Martelli]

Re: Lambda going out of fashion

Alan Gauld <alan.gauld@bti nternet.com> wrote:
...[color=blue]
> It can't be that hard to maintain the lambda code, why not just
> leave it there for the minority of us who like the concept?[/color]

I guess the key motivator is the usual set of design principles:

3. Keep the language small and simple.
4. Provide only one way to do an operation.

Well, this phrasing is actually from the "Spirit of C" section of the
introduction to the ISO C standard, but the Zen of Python phrasing,
"""There should be one-- and preferably only one --obvious way to do
it.""" isn't all that different.

Having lambda in the language in addition to def provides two ways to do
an operation (make a function), and for that it makes the language a
little bit less simple and less small than if only def was there. I
guess that's why Guido now regrets ever having accepted lambda into the
language, and hopes to take it away when backwards compatibility can be
broken (i.e., in 3.0).


Having just reviewed, edited and merged over 1000 recipes to select
about 1/3 of those for the 2nd edition of the Cookbook, I think I'm in a
good position to state that, at least as far as CB contributors are
representative of the Python community, uses of lambda that are dubious
to very dubious to absurd outnumber those which are decent to good by
around 4:1. If I see another case of:

somename = lambda x: ...

instead of the obvious

def somename(x): ...

I think I'll scream, though not quite as loud as for my next seeing:

map(lambda x: f(x), ...

instead of

map(f, ...


I don't know what it IS about lambda that prompts so much dubious to
absurd use, but that's what I observed. I don't know if that plays any
role in Guido's current thinking, though -- I have no idea how much
"dubious Python" he's had to struggle with. One could argue that the
typical abuses of redundant lambda as shown above are closely related,
e.g., to typical abuses of booleans, such as:

if (x>y) == True:

Funny enough, though, I've seen the equivalent of this latter abuse very
often in C and C++, but not in Python (again judging e.g. from the
submissions to the cookbook site). Maybe it's bool's relatively recent
introduction in the language; after all, one DOES often see:

if len(somecontain er) > 0:

instead of the obvious

if somecontainer:

so it's not as if pythonistas in general are blessed with some magical
"redundancy avoidance spell"...


Alex

[Nick Coghlan]

Re: Lambda going out of fashion

Alan Gauld wrote:[color=blue]
> It can't be that hard to maintain the lambda code, why not just
> leave it there for the minority of us who like the concept?[/color]

Because one of the points of Py3K is to clean up the language concepts and
syntax, and the current lambda just doesn't fit cleanly. If it was proposed in a
PEP, the syntax would be rejected as not being sufficiently Pythonic and for
being a castrated near-synonym for the def statement.

Now, Python 3K will retain the statement/expression distinction, so anonymous
functions will indeed be impossible without lambda - the link from a statement
to an expression will need to be made through the local namespace.

So, rather than pushing to retain lambda for Py3K, it might be more productive
to find a better statement -> expression translation for function definitions.
Guido seems to prefer named functions, so it would still be tough to gain his
acceptance. However, a more Pythonic syntax is the only way I can see anonymous
functions making into 3.0

The current best example of a statement->expression translation is generator
expressions:

def squares(seq)
for x in seq:
yield x * x

total = sum(squares(seq ))

versus:

total = sum(x * x for x in seq)

If we consider a function definition (omitting decorators and docstrings) we get:

def foo(a, b, c):
return f(a) + o(b) - o(c)

accepts_func(fo o)

What would a Pythonic 'function as expression' look like?

Perhaps something like:

accepts_func( (def (a, b, c) to f(a) + o(b) - o(c)) )

Here we simply omit the function name and use 'to' instead of ':' (the colon is
omitted to avoid making our expression look like it might be a statement). We
also don't need a return statement, since our target is an expression. The
surrounding parentheses would be required.

The "def (arg-tuple) to (result)" form is intended to show that we're talking
about a function in the mathematical sense of a single expression, rather than
the general Python sense of a suite of statements. That is, it has the same
behaviour as the current lambda - if you want a real Python function, write a
real Python function :)

Personally, I don't see the restriction of anonymous functions to single
expressions any more of a wart than the restriction of the value portion of a
generator expression to a single expression. If an algorithm is too complex for
a single expression, it's probably worth giving a name.

Some more examples:

(def (x) to x * x) # Map values to their squares
(def () to x) # No arguments, always map to x
(def (*a, **k) to x.bar(*a, **k)) # Look up the method now, call it later

And in combination with a generator expression:

( (def () to x(*a, **k)) for x, a, k in funcs_and_args_ list)

Replacing the 'to' with -> might actually read better:

(def (a, b, c) -> f(a) + o(b) - o(c))
(def (x) -> x * x)
(def () -> x)
(def (*a, **k) -> x.bar(*a, **k))
( (def () -> x(*a, **k)) for x, a, k in func_list)

Anyway, thats just some ideas if you're concerned about the plan to have lambda
disappear in 3K.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[Fredrik Lundh]

Re: Lambda going out of fashion

Alex Martelli wrote:
[color=blue]
> I think I'll scream, though not quite as loud as for my next seeing:
>
> map(lambda x: f(x), ...
>
> instead of
>
> map(f, ...[/color]

note that if you replace "map" with "some function that takes a callable", the difference
between these two constructs may be crucially important.

</F>

[Peter Otten]

Re: Lambda going out of fashion

Alex Martelli wrote:
[color=blue]
> if len(somecontain er) > 0:
>
> instead of the obvious
>
> if somecontainer:
>
> so it's not as if pythonistas in general are blessed with some magical
> "redundancy avoidance spell"...[/color]

That is not always equivalent:
[color=blue][color=green][color=darkred]
>>> z = Numeric.zeros(5 )
>>> z[/color][/color][/color]
array([0, 0, 0, 0, 0])[color=blue][color=green][color=darkred]
>>> bool(z)[/color][/color][/color]
False[color=blue][color=green][color=darkred]
>>> len(z) > 0[/color][/color][/color]
True

Peter

[Alex Martelli]

Re: Lambda going out of fashion

Fredrik Lundh <fredrik@python ware.com> wrote:
[color=blue]
> Alex Martelli wrote:
>[color=green]
> > I think I'll scream, though not quite as loud as for my next seeing:
> >
> > map(lambda x: f(x), ...
> >
> > instead of
> >
> > map(f, ...[/color]
>
> note that if you replace "map" with "some function that takes a callable",
> the difference between these two constructs may be crucially important.[/color]

Sure -- if global name f gets rebound during the execution of the ``some
function'' (or if that function stashes the callable away, and that name
gets rebound later, etc), the semantics of passing f are different -- to
get the same semantics with a lambda, you'd have to use the old trick:

somefunc(lambda x, f=f: f(x), ...


I don't recall ever having seen the late-binding semantics (of the
lambda I originally showed) ``used properly'' -- I _have_ sometimes seen
that semantics cause subtle bugs, though. Do you have any real-life
examples where expecting and allowing for rebinding of global name `f'
is proper and desirable behavior? Being able to show a "good" use of
this late-binding could be helpful, if I knew of any.


Alex

[jfj]

Re: Lambda going out of fashion

Stephen Thorne wrote:[color=blue]
> Hi guys,
>
> I'm a little worried about the expected disappearance of lambda in
> python3000. I've had my brain badly broken by functional programming
> in the past, and I would hate to see things suddenly become harder
> than they need to be.
>[/color]

Don't worry, it's not gonna go away because too much software is
depending on it. If you don't follow the advice that lambda is
deprecated and you keep using it, more software will depend on it and it
will never disappear :)

Personally I'm not a fan of functional programming but lambda *is*
useful when I want to say for example:

f (callback=lambd a x, y: foo (y,x))

I don't believe it will ever disappear.


G.

[Alex Martelli]

Re: Lambda going out of fashion

Peter Otten <__peter__@web. de> wrote:
[color=blue]
> Alex Martelli wrote:
>[color=green]
> > if len(somecontain er) > 0:
> >
> > instead of the obvious
> >
> > if somecontainer:
> >
> > so it's not as if pythonistas in general are blessed with some magical
> > "redundancy avoidance spell"...[/color]
>
> That is not always equivalent:
>[color=green][color=darkred]
> >>> z = Numeric.zeros(5 )
> >>> z[/color][/color]
> array([0, 0, 0, 0, 0])[color=green][color=darkred]
> >>> bool(z)[/color][/color]
> False[color=green][color=darkred]
> >>> len(z) > 0[/color][/color]
> True[/color]

Good point! Numeric sure has semantics here that can be confusing;-).

numarray's arrays just can't be used as the argument to bool(...) -- you
get a runtime error "An array doesn't make sense as a truth value. Use
sometrue(a) or alltrue(a)" ("in the face of ambiguity, refuse the
temptation to guess"). Still, this ALSO means that len(z)>0 is very
different from bool(z), for numarray as for Numeric.

Most containers don't choose to implement __nonzero__, so bool(z) just
calls __len__ instead; that's the scenario I had in mind, of course.
But sure, if somecontainer might define peculiar semantics in
__nonzero__, then in order to test if it's empty you can't use the
normal pythonic idiom of checking its truth value, but rather must check
its length. (I still prefer "if len(z):", avoiding the redundant ">0",
but that's just a small difference, of course).


Alex

[Nick Coghlan]

Re: Lambda going out of fashion

jfj wrote:[color=blue]
> Stephen Thorne wrote:
>[color=green]
>> Hi guys,
>>
>> I'm a little worried about the expected disappearance of lambda in
>> python3000. I've had my brain badly broken by functional programming
>> in the past, and I would hate to see things suddenly become harder
>> than they need to be.
>>[/color]
>
> Don't worry, it's not gonna go away because too much software is
> depending on it. If you don't follow the advice that lambda is
> deprecated and you keep using it, more software will depend on it and it
> will never disappear :)[/color]

No, that doesn't apply for Python 3.0

Indeed, lambda as it currently stands will disappear for the Python 2.x series.

Python 3.0 will be a case of "OK, let's take the things we learned were good and
keep them, and throw away the things we realised were bad"

Undoubtedly, the two languages will co-exist for quite some time.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[Alex Martelli]

Re: Lambda going out of fashion

Nick Coghlan <ncoghlan@iinet .net.au> wrote:
...[color=blue]
> Perhaps something like:
>
> accepts_func( (def (a, b, c) to f(a) + o(b) - o(c)) )[/color]

Nice, except I think 'as' would be better than 'to'. 'as' should be a
full keyword in 3.0 anyway (rather than a surprisingly-NOT-keyword like
today), and "define something as somethingelse" seems marginally more
readable to me than "define something to somethingelse" anyway.


Alex

[Craig Ringer]

Re: Lambda going out of fashion

Fredrik Lundh wrote:
[color=blue]
>Craig Ringer wrote:
>
>
>[color=green]
>> It's hard to consistently support Unicode in extension modules without
>>doing a lot of jumping through hoops. Unicode in docstrings is
>>particularl y painful. This may not be a big deal for normal extension
>>modules, but when embedding Python it's a source of considerable
>>frustration . It's also not easy to make docstrings translatable.
>>Supporting an optional encoding argument for docstrings in the
>>PyMethodDef struct would help a lot, especially for docstrings returned
>>by translation mechanisms.
>>
>>[/color]
>
>docstrings should be moved out of the C modules, and into resource
>files. (possibly via macros and an extractor). when I ship programs
>to users, I should be able to decide whether or not to include docstrings
>without having to recompile the darn thing.
>
>[/color]
Good point. Unfortunately, many translation mechanisms - like Qt's, for
example - don't make it easy to translate strings in resource files
using the same tools as the core app. Translators have a hard enough job
already with one set of tools in my experience, lumping more on them
probably isn't nice. On the other hand, this isn't really Python's
problem - but neither is where they come from. Even if I load docstrings
from resource files, I still have a fair bit of work ahead to make
Python accept them if they're not plain ASCII. In my current project, I
actually gave up and changed sysdefaultencod ing to utf-8 . (It's an
embedded interpreter, so it's not too bad - and all modules should
handle that correctly by now anyway).
[color=blue][color=green]
>> Const. I know there's a whole giant can of worms here, but even so -
>>some parts of the Python/C API take arguments that will almost always be
>>string literals, such as format values for Py_BuildValue and
>>PyArg_ParseTu ple or the string arguments to Py*_(Get|Set)St ring calls.
>>Many of these are not declared const, though they're not passed on to
>>anywhere else. This means that especially in c++, one ends up doing a
>>lot of swearing and including a lot of ugly and unnecessary string
>>copies or const_cast<char *>()s to silence the compiler's whining. It
>>would be nice if functions in the Python/C API would declare arguments
>>(not return values) const if they do not pass them on and do not change
>>them.
>>
>>[/color]
>
>I think the only reason that this hasn't already been done is to reduce the
>amount of swearing during the conversion process (both for the core developer
>and C extension developers...).
>
>[/color]
Agreed. However, it's my understanding that one can convert:

PyObject* *Py_BuildValue* ( char *format, ...)

to
PyObject* *Py_BuildValue* ( const char *format, ...)

(for example) without affecting module developers or users of that
function elsewhere in the core code. const _arguments_ are often very
safe, its const return values that tend to suck.

--
Craig Ringer

[Robin Becker]

Re: Lambda going out of fashion

Alex Martelli wrote:
......[color=blue]
> By the way, if that's very important to you, you might enjoy Mozart
> (http://www.mozart-oz.org/) -- I'm looking at it and it does appear to
> go even further in this specific regard (rich support for multi -
> paradigm programming). It's also blessed with a great book,
> <http://www.info.ucl.ac .be/people/PVR/book.html> -- I've just started
> browsing it, but it appears to be worthy of being called "SICP for the
> 21st century"...!-). Just like SICP made it worthwhile to learn a
> little Scheme even if you'd never use it in production, so does CTMCP
> (acronym for this new book by Van Roy and Haridi) work for Oz, it
> appears to me.
>[/color]
......very interesting, but it wants to make me install emacs. :(
[color=blue]
> Alex[/color]


--
Robin Becker

[Nick Coghlan]

Re: Lambda going out of fashion

Alex Martelli wrote:[color=blue]
> Nick Coghlan <ncoghlan@iinet .net.au> wrote:
> ...
>[color=green]
>>Perhaps something like:
>>
>>accepts_fun c( (def (a, b, c) to f(a) + o(b) - o(c)) )[/color]
>
>
> Nice, except I think 'as' would be better than 'to'. 'as' should be a
> full keyword in 3.0 anyway (rather than a surprisingly-NOT-keyword like
> today), and "define something as somethingelse" seems marginally more
> readable to me than "define something to somethingelse" anyway.[/color]

I actually flipped back and forth between preferring 'as' and 'to' while writing
the message.

It was the mathematical phrasing of "f is a function from R to R" (replacing the
capital R's with the symbol for the real numbers) that first made me think of
'to', since I was trying to emphasise the parallels with mathematical functions.
(To my mind, theoretical mathematics is one of the areas with significant
legitimate uses for lambda functions).

The '->' suggestion had a similar source.

The way I'd read the 'to' version out loud when explaining to someone what the
code did:

"DEFine an anonymous function from arguments a, b and c TO the value f of a plus
o of b minus o of c"

or the short version:

"DEFine a function from a, b and c TO f a plus o b minus o c"

As an even simpler example, (def (x) to x * x) would be "define a function from
x to x squared". (def () to <whatever>) would be "define a function from no
arguments to <whatever>"

'as' already implies renaming semantics due to from-style imports. In Py3k, it's
likely to pick up the naming duties in except clauses as well (i.e. "except
ValueError, TypeError as ex:").

'as' is also a candidate for optional static typing and adaptation (and
hopefully someone will talk Guido out of his punctuation happy version of that!).

I eventually decided that using 'as' for anonymous functions as well would just
be plain confusing.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[Nick Coghlan]

Re: Lambda going out of fashion

Nick Coghlan wrote:[color=blue]
> Indeed, lambda as it currently stands will disappear for the Python 2.x
> series.[/color]

Will NOT disappear. I repeat, will NOT disappear.

Cheers,
Nick.
Damn those missing negators. . .

--
Nick Coghlan | ncoghlan@email. com | Brisbane, Australia
---------------------------------------------------------------

http://boredomandlaziness.skystorm.net

[Stephen Thorne]

Re: Lambda going out of fashion

On Thu, 23 Dec 2004 13:47:49 +0100, Alex Martelli <aleaxit@yahoo. com> wrote:[color=blue]
> Nick Coghlan <ncoghlan@iinet .net.au> wrote:
> ...[color=green]
> > Perhaps something like:
> >
> > accepts_func( (def (a, b, c) to f(a) + o(b) - o(c)) )[/color]
>
> Nice, except I think 'as' would be better than 'to'. 'as' should be a
> full keyword in 3.0 anyway (rather than a surprisingly-NOT-keyword like
> today), and "define something as somethingelse" seems marginally more
> readable to me than "define something to somethingelse" anyway.[/color]

I'm sorry, but I dislike this quite a bit, 'def arglist as expression'
just doesn't fit right for me.

After reading the discussion, and trying to glean answers to my
original questions, I realise that there's a pragmatic way of doing
the things that I mentally 'require' lambda for, without using lambda
at all. I might try and catalog lambda patterns and their alternatives
at some point.

Alex, thankyou for the feedback concerning the misuse of lambda. I
consider prevention of misuse to be much more important than the
denial of use. I recognise fully the frustration you must experience
when you see map(lambda x:f(x).

I think it is important to voice concern, and I recieved not a few
'me-too's in reply to this thread. But at the end of the day, I'm not
against removing lambda in py3k.

slightly-less-concerned-ly yr's
Stephen Thorne.

[rzed]

Re: Lambda going out of fashion

Stephen Thorne <stephen.thorne @gmail.com> wrote in
news:mailman.82 99.1103775211.5 135.python-list@python.org :
[color=blue]
> Hi guys,
>
> I'm a little worried about the expected disappearance of lambda
> in python3000. I've had my brain badly broken by functional
> programming in the past, and I would hate to see things suddenly
> become harder than they need to be.
>
> An example of what I mean is a quick script I wrote for doing
> certain actions based on a regexp, which I will simlify in this
> instance to make the pertanant points more relevent.
>
> {
> 'one': lambda x:x.blat(),
> 'two': lambda x:x.blah(),
> }.get(someValue , lambda x:0)(someOtherV alue)
>
> The alternatives to this, reletively simple pattern, which is a
> rough parallel to the 'switch' statement in C, involve creating
> named functions, and remove the code from the context it is to
> be called from (my major gripe).
>
> So, the questions I am asking are:
> Is this okay with everyone?
> Does anyone else feel that lambda is useful in this kind of
> context? Are there alternatives I have not considered?
>[/color]

Not addressing lambdas per se, but something similar to your pseudo-
switch statement can be done without using them at all. One way might
be to use a function dictionary wrapped in an accessor function, such
as this:

def dofns(key):
fd2 = {
0:'print "Key 0"',
1:'print "one"',
4:"\n".join(['for ix in range(15):',
' print "%s%d" % (" "*ix,ix)']),
}
try:
exec(fd2[key])
except KeyError:
print 'Key',key,'not found'

The keys can as easily be strings, of course. One virtue of this
method is that it allows multi-line statements and looping (as in fd2
[4]).

Now the actual 'switch' statement becomes as simple as:
dofns(key)

If there are parameters involved, this solution becomes a little
tackier. One way might be something like:

def dofns(key,**kws ):
fd2 = {
0:'print "Key 0"',
1:'print "one"',
2:"\n".join(['n=kws["name"]','x=kws["x"]',
'y=kws["y"]','print "%s:"%n,x,y ']),
3:"\n".join(['val=kws["x"]*7 + kws["y"]',
'print kws["x"],kws["y"],val']),
4:"\n".join(['for ix in range(kws["x"]):',
' print "%s%d" % (" "*ix,ix)']),
5:'exec(fd2[3])',
6:"\n".join(['print kws["z"],',
'dofns(3,x=13,y =22)']),
}
try:
exec(fd2[key])
except KeyError:
print 'Key',key,'not found'


# switch (key kwdparms)
for key in [0, 1,2,3,4,5,6,7]:
dofns(key,name= 'Gladys',y=50,x =8,z=34)

You could remove the function dictionary from the wrapper and use it
similarly to your example, but I find that less readable.
exec({
multi-line dictionary definition
}[key] parms) # what were we doing again? Oh! exec!

Anyway, for those who use lambdas, this approach is likely to be
unappealing. For those who find lambdas puzzling, it may be an
alternative, at least for a way to handle the switch equivalent.

--
rzed