closures and dynamic binding

[Paul Boddie]

Re: closures and dynamic binding

On 29 Sep, 19:26, Terry Reedy <tjre...@udel.e duwrote:

>
Please: Python does not have 'lambda functions'. Def statements and
lambda expressions both define instances of the function class. So this
amounts to saying "functions are subject to the same caveats as functions."

I myself am aware of the nature of "lambda expressions", for want of a
better term, but it's important to emphasise their nature to anyone
reading who isn't fully aware of what they represent. My closing
paragraph touches on the issues of readability and programmer
expectation when I write that 'function definition statements do not
behave like, say, "for", "if" or "while" statements'. Although this
may seem obvious, a newcomer might overlook lambda expressions in this
regard.

Personally, I'm not a great enthusiast of closures, anyway. Perhaps I
spent too long writing Java to be able to look at them as being
anything other than a fairly imprecise way of encapsulating state in a
callable.

Paul

[jhermann]

Re: closures and dynamic binding

I didn't see this mentioned in the thread yet: the double-lambda is
unnecessary (and a hack). What you should do when you need early
binding is... early binding. ;)

Namely:

f = [lambda n=n: n for n in range(10)]
print f[0]()
print f[1]()

Note the "n=n", this prints 0 and 1 instead of 9/9.

[Paul Boddie]

Re: closures and dynamic binding

On 1 Okt, 12:43, jhermann <Juergen.Herm.. .@1und1.dewrote :

>
f = [lambda n=n: n for n in range(10)]
print f[0]()
print f[1]()
>
Note the "n=n", this prints 0 and 1 instead of 9/9.

Yes, Terry mentioned this in his response to my first message. Not
with lambdas, however, but he did state that he didn't believe that
such a distinction needed to be made clear to everyone.

Paul

[Aaron \Castironpi\ Brady]

Re: closures and dynamic binding

On Oct 1, 5:43 am, jhermann <Juergen.Herm.. .@1und1.dewrote :

I didn't see this mentioned in the thread yet: the double-lambda is
unnecessary (and a hack). What you should do when you need early
binding is... early binding. ;)
>
Namely:
>
f = [lambda n=n: n for n in range(10)]
print f[0]()
print f[1]()
>
Note the "n=n", this prints 0 and 1 instead of 9/9.

Yes it was mentioned earlier. I think its similar. They both create
ten new namespaces. You could do something like this (I hit a bump
with eval and globals() when I tried it):

def early( string ):
return eval( string, current_namespa ce )

f = [early( 'lambda: n' ) for n for n in range(10)]
print f[0]()
print f[1]()

Furthermore, I don't think the binding semantics of the language are
completely static binding. What does that definition say about
mutating a value? I think it's ambiguous and there's no obvious use
case that favors either one.

[Michele Simionato]

Re: closures and dynamic binding

On Oct 3, 10:44 am, greg <g...@cosc.cant erbury.ac.nzwro te:

So if anything were to be done to the language to
fix this, it really should be focused on fixing the
semantics of the for-loop. Unfortunately, the
fact that the loop variable leaks out of the scope
of the loop is regarded as a feature, so anything
which changes that seems to be a non-starter.

And Guido stated many times in the past that he is happy with the for
loop as it is,
so I don't think this will never change, even if the question keep
getting asked here
and there.
Notice that even generator expressions, where the loop variable does
not leak outside the loop,
have the same behavior.
The behavior of the list comprehension is a good test of how much
functional a language is;
Common Lisp and Python behaves in the same way (there is a single loop
variable which is
mutated at each iteration) wherea Scheme and Haskell introduce a new
binding at each iteration.

[Hrvoje Niksic]

Re: closures and dynamic binding

greg <greg@cosc.cant erbury.ac.nzwri tes:

The root of the problem actually has nothing to do with lambdas or
static vs. non-static scoping. It's the fact that Python's for-loop
doesn't create a new environment for the loop variable each time
around, but re-uses a slot in the containing environment.
>
Contrast this with Scheme, where the equivalent of a for-loop *does*
create a new environment for each value of the loop variable.

Note that Python's semantics of "for" regarding closures are not
unique to Python. Common Lisp behaves similar to Python in this
regard:

* (loop for i from 0 to 2 collect (lambda () i))
(#<CLOSURE... {A86F3CD}#<CLOS URE... {A86F3E5}#<CLOS URE... {A86F3FD}>)
* (mapcar #'funcall *)
(3 3 3)

Other looping constructs, such as "do", behave in equivalent fashion.

So if anything were to be done to the language to fix this, it
really should be focused on fixing the semantics of the
for-loop. Unfortunately, the fact that the loop variable leaks out
of the scope of the loop is regarded as a feature, so anything which
changes that seems to be a non-starter.

I don't think it has anything to do with variable leaking out of the
loop. Common Lisp doesn't leak the loop variable, and it behaves the
same. It is more a result of the different views of what iteration
is. Common Lisp and Python define iteration in terms of repeating the
same instructions over and over, not different from what C does, in
which case it makes sense to reuse the same environment for all loop
passes. (Python's language ref defines that a standard "assignment "
is done for each new iteration.) In contrast, Scheme regards
iteration as a special case of recursion, and R5RS "do" prescribes
assigning loop variables to "fresh locations" to match what recursion
normally does. In most cases both definitions exhibit the same
behavior, but unfortunately not when closures are created inside the
loop.

[Aaron \Castironpi\ Brady]

Re: closures and dynamic binding

On Oct 3, 3:44 am, greg <g...@cosc.cant erbury.ac.nzwro te:

jhermann wrote:

I didn't see this mentioned in the thread yet: the double-lambda is
unnecessary (and a hack).

>
Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.
It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.
>
The double lambda is conceptually more sound in some
ways, and can be made to work correctly in all cases.
>
The root of the problem actually has nothing to do with
lambdas or static vs. non-static scoping. It's the fact
that Python's for-loop doesn't create a new environment
for the loop variable each time around, but re-uses a
slot in the containing environment.
>
Contrast this with Scheme, where the equivalent of a
for-loop *does* create a new environment for each
value of the loop variable. Effectively it's using a
double lambda, except that one of the lambdas is
folded into the syntax of the loop, so you don't
notice it.
>
So if anything were to be done to the language to
fix this, it really should be focused on fixing the
semantics of the for-loop. Unfortunately, the
fact that the loop variable leaks out of the scope
of the loop is regarded as a feature, so anything
which changes that seems to be a non-starter.
>
--
Greg

I agree that the default argument syntax is an abuse, but it
accomplishes exactly what I want: to create a copy of a namespace. I
don't think there's a way to create a closure in Python without
another function, so you might need new syntax if you wanted to.

Otherwise, using function syntax, I want a new namespace on each
iteration that nests inside the old one, except for one variable which
overrides the outer scope. I agree that a new variable isn't the
obviously correct meaning of a for loop, and functions are the same as
a new scope, just you have to call them, so why not use them as is?

(untested)
for i in range( 3 ):
def f( n ):
def g( ):
return n
return g
closures[ i ]= f( i )

Or:

(non-standard)
for i in range( 3 ):
closure f( i ):
def g( ):
return i
return g
closures[ i ]= f

Here the only difference is whether you call 'f' or not. 'closure'
would theoretically "call itself", and make a copy of its scope upon
execution of the definition, overriding the arguments. So, functions
are the same.

[Terry Reedy]

Re: closures and dynamic binding

greg wrote:

jhermann wrote:
>

>I didn't see this mentioned in the thread yet: the double-lambda is
>unnecessary (and a hack).

>
Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.

I disagree. It is one way to evaluate an expression when a function is
compiled.

It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.

So? Many things do not work 'in general'. If one wants multiple
closures with a variable number of arguments, one should use a def
statement and some other binding method, such as given below

Here are four ways to get the list of closures desired:
All print 0 ... 9 with for f in lst: print(f()) #3.0

lst = []
for i in range(10):
lst.append(eval ("lambda: %d" %i))

# use exec instead of eval with def statement instead of lambda expression

lst = []
def f(i): return lambda: i
for i in range(10):
lst.append(f(i) )

#I would most likely use this, with a def instead of lambda inside f for
any real, non-trivial example.

def populate(n):
n -= 1
if n >= 0: return populate(n)+[lambda:n]
else: return []
lst = populate(10)

# body recursion

def populate(i,n,ls t):
if i < n: return populate(i+1,n, lst+[lambda:i])
else: return lst
lst = populate(0,10,[])

# tail recursion

Terry Jan Reedy

[Terry Reedy]

Re: closures and dynamic binding

Hrvoje Niksic wrote:

greg <greg@cosc.cant erbury.ac.nzwri tes:

I don't think it has anything to do with variable leaking out of the
loop. Common Lisp doesn't leak the loop variable, and it behaves the
same. It is more a result of the different views of what iteration
is. Common Lisp and Python define iteration in terms of repeating the
same instructions over and over, not different from what C does, in
which case it makes sense to reuse the same environment for all loop
passes. (Python's language ref defines that a standard "assignment "
is done for each new iteration.) In contrast, Scheme regards
iteration as a special case of recursion,

Whereas I regard iteration as 'within-namespace recursion' ;-)
which amounts to what Python, etc, do.

[Aaron \Castironpi\ Brady]

Re: closures and dynamic binding

On Oct 3, 3:47 pm, Terry Reedy <tjre...@udel.e duwrote:

greg wrote:

jhermann wrote:

>

I didn't see this mentioned in the thread yet: the double-lambda is
unnecessary (and a hack).

>

Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.

>
I disagree.  It is one way to evaluate an expression when a function is
compiled.
>

It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.

>
So?  Many things do not work 'in general'. If one wants multiple
closures with a variable number of arguments, one should use a def
statement and some other binding method, such as given below
>
Here are four ways to get the list of closures desired:
All print 0 ... 9 with for f in lst:  print(f()) #3.0
>
lst = []
for i in range(10):
     lst.append(eval ("lambda: %d" %i))
>
# use exec instead of eval with def statement instead of lambda expression
>
lst = []
def f(i): return lambda: i
for i in range(10):
     lst.append(f(i) )
>
#I would most likely use this, with a def instead of lambda inside f for
any real, non-trivial example.
>
def populate(n):
   n -= 1
   if n >= 0: return populate(n)+[lambda:n]
   else: return []
lst = populate(10)
>
# body recursion
>
def populate(i,n,ls t):
     if i < n: return populate(i+1,n, lst+[lambda:i])
     else: return lst
lst = populate(0,10,[])
>
# tail recursion
>
Terry Jan Reedy

Is there a way to get at the 'reduce' / 'setstate' mechanism that
pickle uses without going through the serialization? I.e. doing a
serial round trip?

In either case, 'copy' module or 'loads( dumps( obj ) )' gets you a
copy of the object, but 'def' is the only way to get its own namespace.

[greg]

Re: closures and dynamic binding

Hrvoje Niksic wrote:

Common Lisp behaves similar to Python in this
regard:
>
* (loop for i from 0 to 2 collect (lambda () i))

I wouldn't call the CL loop macro the equivalent of
a for-loop. It's more like a while-loop.

The Lisp equivalent of a Python for-loop would be
to use one of the mapping functions. Then, since
the loop body is a lambda, you get a new scope for
each iteration automatically.

I don't think it has anything to do with variable leaking out of the
loop.

That's right, it doesn't. If I were designing a
for-loop from scratch, I wouldn't let it leak, but
that part seems to be necessary for backwards
compatibility.

In contrast, Scheme regards
iteration as a special case of recursion, and R5RS "do" prescribes
assigning loop variables to "fresh locations" to match what recursion
normally does.

I'd say it prescribes that because it's useful
behaviour, not because it has anything to do with
recursion.

--
Greg

[greg]

Re: closures and dynamic binding

jhermann wrote:

I didn't see this mentioned in the thread yet: the double-lambda is
unnecessary (and a hack).

Well, the alternative -- abusing default argument values --
is seen by many to be a hack as well, possibly a worse one.
It doesn't work in general, e.g. it fails if the function
needs to be called with a variable number of arguments.

The double lambda is conceptually more sound in some
ways, and can be made to work correctly in all cases.

The root of the problem actually has nothing to do with
lambdas or static vs. non-static scoping. It's the fact
that Python's for-loop doesn't create a new environment
for the loop variable each time around, but re-uses a
slot in the containing environment.

Contrast this with Scheme, where the equivalent of a
for-loop *does* create a new environment for each
value of the loop variable. Effectively it's using a
double lambda, except that one of the lambdas is
folded into the syntax of the loop, so you don't
notice it.

So if anything were to be done to the language to
fix this, it really should be focused on fixing the
semantics of the for-loop. Unfortunately, the
fact that the loop variable leaks out of the scope
of the loop is regarded as a feature, so anything
which changes that seems to be a non-starter.

--
Greg