Closures python,scheme,ruby

[paolo veronelli]

I've a vague idea of the differences,I don't know scheme anyway.

I'd like to see an example to show what is missing in python about
closures and
possibly understand if ruby is better in this sense.

Iuse ruby and python in parallel for my job just to learn them and their
differences and python is shorter and cleaner ,but i feel it's missing
something,
in closures.Any hints?

grazie Paolino
--
.....lotta dura per la verdura

[Michele Simionato]

Re: Closures python,scheme,r uby

paolo veronelli <paolo_veronell i@yahoo.it> wrote in message news:<mailman.2 50.1089623585.5 135.python-list@python.org >...[color=blue]
> I've a vague idea of the differences,I don't know scheme anyway.
>
> I'd like to see an example to show what is missing in python about
> closures and
> possibly understand if ruby is better in this sense.
>
> Iuse ruby and python in parallel for my job just to learn them and their
> differences and python is shorter and cleaner ,but i feel it's missing
> something,
> in closures.Any hints?
>
> grazie Paolino[/color]

Python is cleaner but not shorter ;) I don't know about closures in Ruby,
the basic difference between Python and Scheme is that Python
closures are read-only, whereas Scheme closures are read-write
(Scheme has a set! primitive). Here is an example:

(define (outer)
(let ((x 1))
(lambda () (set! x 2) x)))

((outer))
2

You cannot do that in Python, unless you use ugly tricks with mutable objects.


Michele Simionato

[Jacek Generowicz]

Re: Closures python,scheme,r uby

paolo veronelli <paolo_veronell i@yahoo.it> writes:
[color=blue]
> I've a vague idea of the differences,I don't know scheme anyway.
>
> I'd like to see an example to show what is missing in python about
> closures[/color]

A long, long time ago, in a Python far away, there were three
namesapaces: local (to a function), global and builtin. When looking
up a name, Python would first search the local namespace (if there
were one, i.e. we're inside a function), if the name were not found,
it would search the global namespace, and, if still not found, fall
back on the builtin namespace.

(Let's forget about builtins, for this discussion. (You can always
explicitly access the builtins namespace via __builtins__.))


a = 3
def fn():
print a

fn() # => 3

No surprises there. "a" is found in the global namespace.

a = 3
def fn():
a = 4
print a # => 4

fn()
print a # => 3

Inside "fn" a local variable "a" is created (shadowing the global
one), it is printed, goes out of scope and disappears. Inside "fn" it
shadowed the global "a" ... which is why the global "a" is unaffected
by the "a = 4" inside "fn", and "3" was printed on the last line.

Now, let's make a tiny alteration to the last example:

a = 3
def fn():
print a # This is the added line
a = 4
print a

fn()
print a

Run this, and you'll get an error, on the line we added:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/tmp/python-BxFkNE", line 7, in ?
fn()
File "/usr/tmp/python-BxFkNE", line 3, in fn
print a
UnboundLocalErr or: local variable 'a' referenced before assignment

Why?

Because the compiler decides that "a" refers to a local variable in
"fn" by noticing that there is an "a = ..." in the function body, and
therefore never looks for "a" outside the local scope. But the first
"print a" comes before that local variable is bound, so it's referring
to a variable which has not been bound yet.

This illustrates that the appearance of "a =" in a function body makes
python believe that "a" is local. But what if you want the function to
modify some global state? What if you want to modify the global "a" ?

For this reason, Python has the "global" keyword:

a = 3
def fn():
global a
print a # => 3
a = 4
print a # => 4

fn()
print a # => 4

All is perfect.

But then, one dark day, nested scopes were added to Python. What does
this mean? Well, in the good old times (say, python 1.5.2) you would
get the following behaviour:

a = 3
def outer():
a = 4
def inner():
print a
return inner

outer()() # => 3

Why? "a" is not found in the local scope of "inner", so Python looks
for it in the global scope, where it does find it ... and where it is
bound to "3".

But in a modern Python, the output of "outer()()" is "4", because
nowadays Python looks for the name in any enclosing function-local
scopes, before getting out to the global scope. So, in a modern
Python, the "a = 4" binding, which is local to "outer" is found.

Now, introducing a local binding of "a" in "inner", should offer no
surprises:

a = 3
def outer():
a = 4
def inner():
a = 5
print a
return inner

outer()() # =>


And adding the same "print a" we added to "fn" earlier ...


a = 3
def outer():
a = 4
def inner():
print a
a = 5
print a
return inner

outer()() # =>

.... gives us exactly the same error

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/tmp/python-BxFY2c", line 12, in ?
outer()()
File "/usr/tmp/python-BxFY2c", line 7, in inner
print a
UnboundLocalErr or: local variable 'a' referenced before assignment

We could try to fix it, as we did before, by adding a "global a"
declaration:

a = 3
def outer():
a = 4
def inner():
global a
print a # => 3
a = 5
print a # => 5
return inner

outer()() # =>


.... but this results in the intermediate local function scope (the one
of "outer") being ignored (as evidenced by the fact that the first
"print a" gives us "3" rather than "4"). So, it seems that nested
scopes were introduced into Python in a read-only fashion.

The problem is that there is no way of saying "mutate the binding for
the name, wherever you find it". You can say:

a) Mutate or create a binding in the builtin scope:

__builtins__.a = ...

b) Mutate or create a binding in the global scope:

global a
a = ...

c) Mutate or create a binding in the local scope:

a = ...

So, one solution might be to introduce a way of saying "Mutate the
binding, wherever you find it":

lexical a
a =

But, viewed from another perspective, the problem is that Python
offers no way to distinguish between mutating existing bindings and
creating new ones; both are spelt "=". If there were a way to specify
whether you mean "create a new binding" or "rebind the name", then
there would be no need for the "lexical" (or even the "global")
declaration.

This is what is done in some other languages. For example in Scheme:

- Find the nearest binding of "a" and rebind it to "3":

(set! a 3)

- Create a new (lexically nested) variable "a" and bind it to "3":

(let ((a 3) ...)


So, what is missing about Python closures, is the mutability of the
enclosed variables. There is a simple way of hacking around it: make
the enclosed variable a mutable, and mutate it, rather than trying to
rebind the name (thereby avoiding the "a =" syntax which tells Python
to create a local binding of "a") ...

def outer():
a = [4]
def inner():
print a[0]
a[0] = a[0] + 1
return inner

fn = outer()
fn() # => 4
fn() # => 5
fn() # => 6

It works, but it severely reduces the elegance and convenience of
closures. If you appreciate closures, you are likely to consider this
a big pain. If you don't appreciate closures you probabl can't see
what the fuss is about, and are likely to tell people that stateful
functions in Python should be implemented as methods of classes.

I happen to appreciate closures.

[Michael Geary]

Re: Closures python,scheme,r uby

Jacek Generowicz wrote:[color=blue]
> [extensive description of the situation with Python closures][/color]

That's a great explanation, Jacek. Very informative.
[color=blue]
> It works, but it severely reduces the elegance and convenience
> of closures. If you appreciate closures, you are likely to consider
> this a big pain. If you don't appreciate closures you probably
> can't see what the fuss is about, and are likely to tell people that
> stateful functions in Python should be implemented as methods
> of classes.
>
> I happen to appreciate closures.[/color]

I really like closures too. They can make for some very simple and elegant
code.

For anyone who wonders what the fuss is about, I posted some Acrobat
JavaScript code a while back with two versions of a piece of code, one with
closures and one with objects:

http://groups.google.com/groups?selm=109eg534ca5cj8b%40corp.supernews.com

-Mike

[Bengt Richter]

Re: Closures python,scheme,r uby

On 15 Jul 2004 13:09:44 +0200, Jacek Generowicz <jacek.generowi cz@cern.ch> wrote:
[...][color=blue]
>This illustrates that the appearance of "a =" in a function body makes
>python believe that "a" is local. But what if you want the function to
>modify some global state? What if you want to modify the global "a" ?
>[/color]
I was going to pick a nit and point to a corner case where an "a = ..."
in the function body still allows^Hed a (also) to refer to a non-local, i.e.,

[color=blue][color=green][color=darkred]
>>> def fn():[/color][/color][/color]
... a = a # local a used to get bound to non-local a, IIRC
... print a
...[color=blue][color=green][color=darkred]
>>> import dis
>>> dis.dis(fn)[/color][/color][/color]
2 0 LOAD_FAST 0 (a)
3 STORE_FAST 0 (a)

3 6 LOAD_FAST 0 (a)
9 PRINT_ITEM
10 PRINT_NEWLINE
11 LOAD_CONST 0 (None)
14 RETURN_VALUE

but code generation seems to have changed, or I misremember.
Maybe the change is to make it absolutely consistent with determining
semantics by full function body lookahead (which incidentally also
rubs me the wrong way for discovering yields to change a function
to a generator). I prefer meaning to depend just on what's
been read so far, top to bottom, left to right, unless there is some
overriding reason not to do it that way (e.g. normal expression and
assignment evaluation order).

I'm still a big python fan though ;-)
[...][color=blue]
>I happen to appreciate closures.[/color]
I do too. What if
x <= expr
meant evaluate expr and then
find x (as if to do a read access) and rebind it, whatever name space it's found in?
(Might be dangerous if you include searching builtins though)
Hm, ... not sure about rebinding a binding discovered by attribute name access -- i.e.,
x.y <= expr
might rebind a class variable somewhere or an instance variable, depending.
Don't know how useful that might be.

Regards,
Bengt Richter

[Jacek Generowicz]

Re: Closures python,scheme,r uby

bokr@oz.net (Bengt Richter) writes:
[color=blue]
> What if
> x <= expr
> meant evaluate expr and then find x (as if to do a read access) and
> rebind it, whatever name space it's found in?[/color]

I suspect that it's too Perlish for Python, but that sort of approach
is probably the simplest, clearest, most extensible ...

[Aside:

It's in situations like this that sexpr-based languages are very
adavantageous: you don't have to invent fancy syntax, you don't have
to come up with new keywords that break old code ... you just say what
you mean in the usual fashion:

(find-and-rebind x expr)

[Of course, you can still argue endlessly over how it should be spelt
(rebind, set!, setq, lexical-set ...) but at least the mechanism is
clear and uncontroversial .]

]
[color=blue]
> (Might be dangerous if you include searching builtins though)[/color]

Yes, you'd probably want to keep builtins out of it.
[color=blue]
> Hm, ... not sure about rebinding a binding discovered by attribute
> name access -- i.e.,
> x.y <= expr
> might rebind a class variable somewhere or an instance variable, depending.
> Don't know how useful that might be.[/color]

Could be fun ! :-)