scoping with lambda in loops

**Christos TZOTZIOY Georgiou** · Jul 18 '05, 02:38 AM

Re: scoping with lambda in loops

On 16 Sep 2003 14:38:16 -0700, rumours say that imcmeans@telus. net (Ian
McMeans) might have written:
[color=blue]
>First, I made multiple lambda functions inside a loop, each of which
>depended on the current loop variable.
>[color=green][color=darkred]
>>>> a = []
>>>> for index in range(5):[/color][/color]
> a.append(lambda : index)
>
>Now, see if you can guess what the output was for each of the
>functions in the list a:[color=green][color=darkred]
>>>> a[0](), a[1](), a[2](), a[3](), a[4]()[/color][/color]
>I had expected it to be (0, 1, 2, 3, 4), but actually, it's:
>
>(4, 4, 4, 4, 4)
>
>This really surprised me. I guess what is happening is that each
>lambda knows what the context of execution is where it was defined,
>and doesn't actually evaluate until the function is called, and when
>it does evaluate, it uses the current value of the variable. Is this
>related to static scoping? A similar thing would happen if you defined
>a nested function that used a variable declared in the outer function,
>then changed that variable, and called the nested function.
>
>Can someone recommend a way to code around this gotcha? I'm having
>trouble. I want the functions created inside the loop to execute with
>the value of the loop index at the moment when the function is made.[/color]

I think this is a FAQ (perhaps it was FAQ 6.10?), and you can find many
threads on the subject if you do a search on groups.google.c om.

The typical way to deal with this, IIRC, is to change your lambda
declaration into:
[color=blue]
> a.append(lambda index=index: index)[/color]

so that index gets evaluated at definition time.
--
TZOTZIOY, I speak England very best,
Microsoft Security Alert: the Matrix began as open source.

**Joost Kremers** · Jul 18 '05, 02:38 AM

Re: scoping with lambda in loops

Ian McMeans wrote:[color=blue][color=green][color=darkred]
>>>> a = []
>>>> for index in range(5):[/color][/color]
> a.append(lambda : index)[/color]
[...][color=blue]
> Can someone recommend a way to code around this gotcha? I'm having
> trouble. I want the functions created inside the loop to execute with
> the value of the loop index at the moment when the function is made.[/color]

intuitively, i would think this should do the trick:
[color=blue][color=green][color=darkred]
>>> a = []
>>> for index in range(5):[/color][/color][/color]
.... a.append(lambda x=index: x)

and testing shows that it does indeed.

but the reason why is rather vague to me, (i'm still rather new at this...)
so perhaps i should think a little more before trying to explain. (and i'm
sure someone else will come and do it better than i ever could.)

--
Joost Kremers
since when is vi an editor? a discussion on vi belongs in
comp.tools.unus able or something... ;-)

**Andrew Koenig** · Jul 18 '05, 02:38 AM

Re: scoping with lambda in loops

Ian> First, I made multiple lambda functions inside a loop, each of which
Ian> depended on the current loop variable.
[color=blue][color=green][color=darkred]
>>>> a = []
>>>> for index in range(5):
>>>> a.append(lambda : index)[/color][/color][/color]

Ian> Now, see if you can guess what the output was for each of the
Ian> functions in the list a:[color=blue][color=green][color=darkred]
>>>> a[0](), a[1](), a[2](), a[3](), a[4]()[/color][/color][/color]
Ian> I had expected it to be (0, 1, 2, 3, 4), but actually, it's:

Ian> (4, 4, 4, 4, 4)

Ian> This really surprised me.

Suppose you did it this way:

a = []
for index in range(5):
def foo():
return index
a.append(foo)

What result would you expect now, and why?

--
Andrew Koenig, ark@acm.org

**David Eppstein** · Jul 18 '05, 02:38 AM

Re: scoping with lambda in loops

In article <7f9e1817.03091 61338.20fbdcbc@ posting.google. com>,
imcmeans@telus. net (Ian McMeans) wrote:
[color=blue]
> First, I made multiple lambda functions inside a loop, each of which
> depended on the current loop variable.
>[color=green][color=darkred]
> >>> a = []
> >>> for index in range(5):[/color][/color]
> a.append(lambda : index)
>
>
> Now, see if you can guess what the output was for each of the
> functions in the list a:[color=green][color=darkred]
> >>> a[0](), a[1](), a[2](), a[3](), a[4]()[/color][/color]
> I had expected it to be (0, 1, 2, 3, 4), but actually, it's:
>
> (4, 4, 4, 4, 4)
>
> This really surprised me. I guess what is happening is that each
> lambda knows what the context of execution is where it was defined,
> and doesn't actually evaluate until the function is called, and when
> it does evaluate, it uses the current value of the variable. Is this
> related to static scoping?[/color]

It's related to closures. If you're using lambda, you're probably a
lisp programmer, and should know all about closures. Creating a
function object with def or lambda, within an outer function scope,
creates a closure for that outer function call. The inner function's
accesses to variables from the outer function will return the
most-recently-updated binding from the closure. If you call the outer
function again, you will get a different unrelated closure.

If you the inner function to have its own local variable that stores
some expression value as it existed at the creation time of the inner
function, rather than re-evaluating the expression whenever the inner
function is called, the standard way is to use a defaulted keyword
parameter:

a = []
for index in range(5):
a.append(lambda index=index: index)

or maybe more concisely

a = [lambda index=index: index for index in range(5)]

--
David Eppstein http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science

**martin z** · Jul 18 '05, 02:38 AM

Re: scoping with lambda in loops

> a = [][color=blue]
> for index in range(5):
> a.append(lambda index=index: index)
>
> or maybe more concisely
>
> a = [lambda index=index: index for index in range(5)][/color]

You know how Python is supposed to be executable pseudocode? Well that
stuff is farking ugly. If I handed pseudocode like that into any TA in one
of my classes, I'd be toast. Is there any way to do that in a legible
manner?

**Dave Benjamin** · Jul 18 '05, 02:38 AM

Re: scoping with lambda in loops

"martin z" <pxtl@hotmail.c om> wrote in message
news:KvP9b.4908 9$DZ.34699@news 04.bloor.is.net .cable.rogers.c om...[color=blue][color=green]
> > a = []
> > for index in range(5):
> > a.append(lambda index=index: index)
> >
> > or maybe more concisely
> >
> > a = [lambda index=index: index for index in range(5)][/color]
>
> You know how Python is supposed to be executable pseudocode? Well that
> stuff is farking ugly. If I handed pseudocode like that into any TA in[/color]
one[color=blue]
> of my classes, I'd be toast. Is there any way to do that in a legible
> manner?[/color]

The following reads pretty well to me:
[color=blue][color=green][color=darkred]
>>> produce_value = lambda value: lambda: value
>>> a = [produce_value(i ndex) for index in range(5)]
>>> a[3]()[/color][/color][/color]
3

Dave

**David Eppstein** · Jul 18 '05, 02:39 AM

Re: scoping with lambda in loops

In article <KvP9b.49089$DZ .34699@news04.b loor.is.net.cab le.rogers.com>,
"martin z" <pxtl@hotmail.c om> wrote:
[color=blue][color=green]
> > a = []
> > for index in range(5):
> > a.append(lambda index=index: index)
> >
> > or maybe more concisely
> >
> > a = [lambda index=index: index for index in range(5)][/color]
>
> You know how Python is supposed to be executable pseudocode? Well that
> stuff is farking ugly. If I handed pseudocode like that into any TA in one
> of my classes, I'd be toast. Is there any way to do that in a legible
> manner?[/color]

How about this:

def makefunction(x) :
def thefunction():
return x
return thefunction

a = map(makefunctio n, range(5))

The identifiers are still a little uninformative, but it's hard to do
better without more information from the original poster...

--
David Eppstein http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science

**Jacek Generowicz** · Jul 18 '05, 02:40 AM

Re: scoping with lambda in loops

imcmeans@telus. net (Ian McMeans) writes:
[color=blue][color=green][color=darkred]
> >>> a = []
> >>> for index in range(5):[/color][/color]
> a.append(lambda : index)
>
>
> Now, see if you can guess what the output was for each of the
> functions in the list a:[color=green][color=darkred]
> >>> a[0](), a[1](), a[2](), a[3](), a[4]()[/color][/color]
> I had expected it to be (0, 1, 2, 3, 4), but actually, it's:
>
> (4, 4, 4, 4, 4)
>
> This really surprised me. I guess what is happening is that each
> lambda knows what the context of execution is where it was defined,
> and doesn't actually evaluate until the function is called, and when
> it does evaluate, it uses the current value of the variable. Is this
> related to static scoping?[/color]

It's related to _lexical_ scoping. It is called a lexical closure.

As of the time when nested scopes were introduced into Python,
whenever a name is referenced, it is first sought in the local lexical
scope (ie, the bit of text in the local function body); if it is not
found there, it is sought in the closest enclosing lexical scope (the
text of any enclosing functions), and so on; when you run out of
enclosing functions you try global, then builtin scope.

There is no "index" variable in your lambda's local scope, so it has
to resort to using the one in the global scope ... which, by the time
you get around to calling your functions, has been set to 4.
[color=blue]
> A similar thing would happen if you defined a nested function that
> used a variable declared in the outer function, then changed that
> variable, and called the nested function.[/color]

Yup.
[color=blue]
> Can someone recommend a way to code around this gotcha?[/color]

Make your own local binding. Function call parameters make local
bindings. So, within a lambda, you can achieve this by using a keyword
argument.

lambda index=index:ind ex (or lambda i=index:i)

Now there is a local index and a global index, so the lambda uses the
local one. Because the default value is evaluated at the time the
lambda expression is evaluated, each local index has a value
corresponding to whatever the global index had at the time the lambda
was evaluated.

scoping with lambda in loops

scoping with lambda in loops

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment