Optional parameter object re-used when instantiating multiple objects

[Aaron Brady]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 16, 7:28 am, Steve Holden <st...@holdenwe b.comwrote:

George Sakkis wrote:

On Nov 16, 2:05 am, Arnaud Delobelle <arno...@google mail.comwrote:

>

Steven D'Aprano <st...@REMOVE-THIS-cybersource.com .auwrites:
>On Sat, 15 Nov 2008 01:40:04 -0800, Rick Giuly wrote:
>>Hello All,
>>Why is python designed so that b and c (according to code below)
>>actually share the same list object? It seems more natural to me that
>>each object would be created with a new list object in the points
>>variable.
>That's not natural *at all*. You're initialising the argument "points"
>with the same list every time. If you wanted it to have a different list
>each time, you should have said so. Don't blame the language for doing
>exactly what you told it to do.
Come on.  The fact that this questions comes up so often (twice in 24h)
is proof that this is a surprising behaviour.  I do think it is the
correct one but it is very natural to assume that when you write

>

    def foo(bar=[]):
         bar.append(6)
         ...

>

you are describing what happens when you _call_ foo, i.e.:

>

    1. if bar is not provided, make it equal to []
    2. Append 6 to bar
    3. ...

>

+1. Understanding and accepting the current behavior (mainly because
of the extra performance penalty of evaluating the default expressions
on every call would incur) is one thing, claiming that it is somehow
natural is plain silly, as dozens of threads keep showing time and
time again. For better or for worse the current semantics will
probably stay forever but I wish Python grows at least a syntax to
make the expected semantics easier to express, something like:

>

def foo(bar=`[]`):
    bar.append(6)

>

where `expr` would mean "evaluate the expression in the function
body". Apart from the obvious usage for mutable objects, an added
benefit would be to have default arguments that depend on previous
arguments:

>
Would you also retain the context surrounding the function declaration
so it's obvious how it will be evaluated, or would you limit the default
values to expressions with no bound variables?
>

def foo(x, y=`x*x`, z=`x+y`):
    return x+y+z

>

as opposed to the more verbose and less obvious current hack:

>

def foo(x, y=None, z=None):
    if y is None: y = x*x
    if z is None: z = x+y
    return x+y+z

>
"Less obvious" is entirely in the mind of the reader. However I can see
far more justification for the behavior Python currently exhibits than
the semantic time-bomb you are proposing.

It is too bad you are not better at sharing what you see. I would
like to see 'far more' of it, if it's there.

[Aaron Brady]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 16, 12:52 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com .auwrote:

On Sat, 15 Nov 2008 21:29:22 -0800, Aaron Brady wrote:

....

If you want to be pedantic, then my "answer" (which you seem to approve
of) doesn't correspond to either of the original poster's questions. If
you're going to be pedantic, then be pedantic all the way, and criticize
me for answering a question that wasn't asked :-P

Not pedantic. He was questioning the reasons and motivation.

....

Actually, the correct answer to "Why?" would be "Because that's the
decision Guido van Rossum made back in the early 1990s when he first
started designing Python". That of course leads to the obvious question
"Why did he make that decision?", and the answer to that is:
>
* it leads to far more efficient performance when calling functions;
>
E.g. if the default value is expensive to calculate, it is better to
calculate it once, when the function is created, than every single time
the function is called.

It is best to calculate it just how many times you need to, and no
more.

Additionally, the effbot once mentioned in a similar thread that there
are real performance benefits in the Python VM from binding the default
value once only. I don't know the exact details of that, but I trust
Fredrik knows what he's talking about.
>
* it has less scope for surprise when calling functions.

Obviously, either behavior would be surprising to different sets of
people, with probably little overlap. Is one set larger, or has
certain characteristics , like contains more deep thinkers, etc.?

E.g. I would argue that most people would be surprised, and dismayed, if
this code fails:
>
x = 1
def foo(a, b=x):
   return a+b
>
del x
print foo(2)

Point taken.

From your post, it's hard to tell whether this 'duh'-type observation
would point out the salient feature of the construct, or whether you're
after something deeper.

>

If you're asking, 'Why isn't the argument list considered to be inside
the body?', then the answer is, it's pretty much arbitrary.

>
No, it is not an arbitrary choice.

I didn't mean arbitrary as in out-of-the-blue. I meant arbitrary as
in dart-board decision from hand-picked possibilities, that is, that
the original decision maker thought, 'close call' between two, and
just picked one.

I have a different perspective than he did at the time, and he does
too now. It's not clear that if he came to think that the cases were
closer than he originally judged, he would say so, though knowing
humans at large, I'd guess that if the case was stronger, he would. I
would. Of course not just any human can invent Python.

I've given practical reasons why the
Python choice is better. If you want default argument to be created from
scratch when the function is called, you can get it with little
inconvenience, but the opposite isn't true. It is very difficult to get
static default arguments given a hypothetical Python where default
arguments are created from scratch. There's no simple, easy idiom that
will work. The best I can come up with is a convention:

I'm not so sure.

## Default evaluated at definition time. (Current.)

# Static arg.
def f( a= [] ):
...

# Non-static arg.
def f( a= None ):
if a is None: a= []

## Default evaluated at call-time. (Alternative.)

# Static arg.
@static( a= [] )
def f( a ):
...

# Non-static arg.
def f( a= [] ):
...

They're about the same difficulty. This comparison makes it look like
efficiency is the strongest argument-- just an 'if' vs. an entire
extra function call.

[Steve Holden]

Re: Optional parameter object re-used when instantiating multipleobjects

Aaron Brady wrote:

On Nov 16, 7:28 am, Steve Holden <st...@holdenwe b.comwrote:

>George Sakkis wrote:

>>On Nov 16, 2:05 am, Arnaud Delobelle <arno...@google mail.comwrote:
>>>Steven D'Aprano <st...@REMOVE-THIS-cybersource.com .auwrites:
>>>>On Sat, 15 Nov 2008 01:40:04 -0800, Rick Giuly wrote:
>>>>>Hello All,
>>>>>Why is python designed so that b and c (according to code below)
>>>>>actually share the same list object? It seems more natural to me that
>>>>>each object would be created with a new list object in the points
>>>>>variable .
>>>>That's not natural *at all*. You're initialising the argument "points"
>>>>with the same list every time. If you wanted it to have a different list
>>>>each time, you should have said so. Don't blame the language for doing
>>>>exactly what you told it to do.
>>>Come on. The fact that this questions comes up so often (twice in 24h)
>>>is proof that this is a surprising behaviour. I do think it is the
>>>correct one but it is very natural to assume that when you write
>>> def foo(bar=[]):
>>> bar.append(6)
>>> ...
>>>you are describing what happens when you _call_ foo, i.e.:
>>> 1. if bar is not provided, make it equal to []
>>> 2. Append 6 to bar
>>> 3. ...
>>+1. Understanding and accepting the current behavior (mainly because
>>of the extra performance penalty of evaluating the default expressions
>>on every call would incur) is one thing, claiming that it is somehow
>>natural is plain silly, as dozens of threads keep showing time and
>>time again. For better or for worse the current semantics will
>>probably stay forever but I wish Python grows at least a syntax to
>>make the expected semantics easier to express, something like:
>>def foo(bar=`[]`):
>> bar.append(6)
>>where `expr` would mean "evaluate the expression in the function
>>body". Apart from the obvious usage for mutable objects, an added
>>benefit would be to have default arguments that depend on previous
>>arguments:

>Would you also retain the context surrounding the function declaration
>so it's obvious how it will be evaluated, or would you limit the default
>values to expressions with no bound variables?
>>

>>def foo(x, y=`x*x`, z=`x+y`):
>> return x+y+z
>>as opposed to the more verbose and less obvious current hack:
>>def foo(x, y=None, z=None):
>> if y is None: y = x*x
>> if z is None: z = x+y
>> return x+y+z

>"Less obvious" is entirely in the mind of the reader. However I can see
>far more justification for the behavior Python currently exhibits than
>the semantic time-bomb you are proposing.

>
It is too bad you are not better at sharing what you see. I would
like to see 'far more' of it, if it's there.

Well, briefly: it seems far simpler to me to use

def f(x=None):
if x is None:
x = <some value interpreted in the context of the current call>
...

than it does to use

def f(x=<some value to be interpreted in the context of the current call>):
...

particularly since at present the language contains no syntax for the
latter. But maybe I am focusing too much on the difficulty of compiling
whatever bizarre syntax is eventually adopted.

Understand, my point of view is biased by forty years of programming
experience, to the extent that I disagreed with Guido's proposal to make
integer division return a floating-point value. So I wouldn't
necessarily claim alignment with the majority.

Consider, though, the case were one argument value has to refer to
another. I would say the function body is the proper place to be doing
that computation. You appear to feel the def statement is the
appropriate place. If multiple statements are needed to perform the
argument initialization, how would you then propose the problem should
be solved?

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

[alex23]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 17, 12:27 pm, Steve Holden <st...@holdenwe b.comwrote:

If multiple statements are needed to perform the
argument initialization, how would you then propose the problem should
be solved?

Why, with another function of course!

def f(x, y=`f_arg_comput ation(x)`): ...

Or my personal favourite:

def f(x, **`f_arg_comput ation(x)`): ...

Seriously, though, I agree with Steve; the function body -is- the
place for computation to occur.

[Chris Rebert]

Re: Optional parameter object re-used when instantiating multipleobjects

For the Nth time this year that this has come up, I'll point out yet
again that this issue has already been discussed to death before:

[Python-ideas] proto-PEP: Fixing Non-constant Default Arguments

http://mail.python.org/pipermail/python-ideas/2007-January/000121.html

[Python-3000] pre-PEP: Default Argument Expressions

http://mail.python.org/pipermail/python-3000/2007-February/005704.html

Result: Evaluating the arguments at runtime rather than
definition-time was deemed too magical; the backward compatibility
issues make changes unlikely; it's hard to find an acceptable syntax.

But hey, if some people want to have another go at it, best of luck.

Cheers,
Chris
--
Follow the path of the Iguana...

http://rebertia.com

On Sun, Nov 16, 2008 at 8:11 PM, alex23 <wuwei23@gmail. comwrote:

On Nov 17, 12:27 pm, Steve Holden <st...@holdenwe b.comwrote:

>If multiple statements are needed to perform the
>argument initialization, how would you then propose the problem should
>be solved?

>
Why, with another function of course!
>
def f(x, y=`f_arg_comput ation(x)`): ...
>
Or my personal favourite:
>
def f(x, **`f_arg_comput ation(x)`): ...
>
Seriously, though, I agree with Steve; the function body -is- the
place for computation to occur.
--

http://mail.python.org/mailman/listinfo/python-list

>

[Aaron Brady]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 16, 8:16 pm, Aaron Brady <castiro...@gma il.comwrote:

On Nov 16, 12:52 am, Steven D'Aprano <st...@REMOVE-THIS-

I've given practical reasons why the
Python choice is better. If you want default argument to be created from
scratch when the function is called, you can get it with little
inconvenience, but the opposite isn't true. It is very difficult to get
static default arguments given a hypothetical Python where default
arguments are created from scratch. There's no simple, easy idiom that
will work. The best I can come up with is a convention:

>
I'm not so sure.
>
## Default evaluated at definition time.  (Current.)
>
# Static arg.
def f( a= [] ):
  ...
>
# Non-static arg.
def f( a= None ):
  if a is None: a= []

Oops. Forgot one, after the subsequent posts.

# Non-static arg.
@nonstatic( a= list )
def f( a ):
...

This can achieve the 'if a is None' effect. 'nonstatic' takes a
callable or a string, '@nonstatic( a= "[]" )'.

I don't see a way to achieve George Sakkis's example:

if y is None: y = x*x
if z is None: z = x+y

Without a change to the language (the other options don't need one).

#emulates 'def foo(x, y=`x*x`, z=`x+y`):'
@nonstatic( y= 'x*x' ) #illegal
@nonstatic( z= 'x+y' ) #illegal
def foo(x, y, z):
return x+y+z

[Rick Giuly]

Re: Optional parameter object re-used when instantiating multipleobjects

Thanks to all for your replies

All things considered, I vote for evaluating the arguments at runtime
(each time the function is called). Good reasons for this have already
been mentioned so I won't repeat them. A programming language is a
user interface of sorts. (Pretty much all languages are logical so
"correctnes s" is not what I'm meaning to discuss - I'm thinking about
usability.) Python provides, for the most part, an *excellent* user
interface to the programmer. Why not make it even "better" by
evaluating the arguments each time the function is called? It will be
harder to change the language 10 years from now, so why not change it
now?

(By "better" I mean that over many years of time programmers will be
more productive because the language will be learned a bit faster with
a fewer surprises - and still retain its power.)

-Rick

On Nov 16, 10:28 pm, "Chris Rebert" <c...@rebertia. comwrote:

For the Nth time this year that this has come up, I'll point out yet
again that this issue has already been discussed to death before:
>
[Python-ideas] proto-PEP: Fixing Non-constant Default Argumentshttp://mail.python.org/pipermail/python-ideas/2007-January/000121.html
>
[Python-3000] pre-PEP: Default Argument Expressionshttp ://mail.python.org/pipermail/python-3000/2007-February/005704.html
>
Result: Evaluating the arguments at runtime rather than
definition-time was deemed too magical; the backward compatibility
issues make changes unlikely; it's hard to find an acceptable syntax.
>
But hey, if some people want to have another go at it, best of luck.
>
Cheers,
Chris
--
Follow the path of the Iguana...http://rebertia.com
>
On Sun, Nov 16, 2008 at 8:11 PM, alex23 <wuwe...@gmail. comwrote:

On Nov 17, 12:27 pm, Steve Holden <st...@holdenwe b.comwrote:

If multiple statements are needed to perform the
argument initialization, how would you then propose the problem should
be solved?

>

Why, with another function of course!

>

def f(x, y=`f_arg_comput ation(x)`): ...

>

Or my personal favourite:

>

def f(x, **`f_arg_comput ation(x)`): ...

>

Seriously, though, I agree with Steve; the function body -is- the
place for computation to occur.
--
http://mail.python.org/mailman/listinfo/python-list

[George Sakkis]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 19, 8:41 am, Rick Giuly <rgiuly.gr...@y ahoo.comwrote:

Python provides, for the most part, an *excellent* user
interface to the programmer. Why not make it even "better"
by evaluating the arguments each time the function is called?
It will be harder to change the language 10 years from now,
so why not change it now?

You probably messed up with your time machine; "now" is 2008, not
1991 ;-)

George

[Aaron Brady]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 19, 12:05 pm, Dennis Lee Bieber <wlfr...@ix.net com.comwrote:

>
        I wouldn't expect a language like Ada to somehow re-evaluate a
default argument on each call; why would I expect Python to do such?

Lots of people do.

If you had a menu in a browser interface that had the items, say,
'Stop' and 'Reload', what would you expect to happen if you clicked on
them?

[alex23]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 20, 10:14 am, Aaron Brady <castiro...@gma il.comwrote:

If you had a menu in a browser interface that had the items, say,
'Stop' and 'Reload', what would you expect to happen if you clicked on
them?

If you had a keyword called 'def', which defined functions, would you
expect it to define said functions when it executed, or on each
function call?

[Aaron Brady]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 19, 7:58 pm, alex23 <wuwe...@gmail. comwrote:

On Nov 20, 10:14 am, Aaron Brady <castiro...@gma il.comwrote:
>

If you had a menu in a browser interface that had the items, say,
'Stop' and 'Reload', what would you expect to happen if you clicked on
them?

>
If you had a keyword called 'def', which defined functions, would you
expect it to define said functions when it executed, or on each
function call?

At first, I would expect it to define them at compile-time. Then,
when I learned there was no such thing, I would expect it to define
them at execute-time. What does that have to do with evaluating a
default argument?

[alex23]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 20, 10:42 pm, Aaron Brady <castiro...@gma il.comwrote:

At first, I would expect it to define them at compile-time.  Then,
when I learned there was no such thing, I would expect it to define
them at execute-time.  What does that have to do with evaluating a
default argument?

It has -everything- to do with it when we're talking about the
defining of functions, and certainly a lot more relevance than straw
man arguments on the behaviour of browser buttons.

You said that you "would expect it to define them at execute-time". So
is that when the interpreter first hits the def or when the
interpreter hits every single function call? Because I personally
consider it to clearly occur at the point of definition and not at the
point of calling.

[Aaron Brady]

Re: Optional parameter object re-used when instantiating multipleobjects

On Nov 20, 5:54 pm, alex23 <wuwe...@gmail. comwrote:

On Nov 20, 10:42 pm, Aaron Brady <castiro...@gma il.comwrote:
>

At first, I would expect it to define them at compile-time.  Then,
when I learned there was no such thing, I would expect it to define
them at execute-time.  What does that have to do with evaluating a
default argument?

>
It has -everything- to do with it when we're talking about the
defining of functions, and certainly a lot more relevance than straw
man arguments on the behaviour of browser buttons.
>
You said that you "would expect it to define them at execute-time". So
is that when the interpreter first hits the def or when the
interpreter hits every single function call? Because I personally
consider it to clearly occur at the point of definition and not at the
point of calling.

Why, I would expect the interpreter to define the functions when it
first hits the def, that is, at the point of definition.

[Steven D'Aprano]

Re: Optional parameter object re-used when instantiating multipleobjects

On Wed, 19 Nov 2008 10:05:23 -0800, Dennis Lee Bieber wrote:

Do you really want a "default" argument that changes value depending
upon actions performed in the /surrounding/ scope?

And if you do, it is easy to get:

default_y = "something"

def parrot(x, y=None):
if y is None:
y = default_y


--
Steven

[Steven D'Aprano]

Re: Optional parameter object re-used when instantiating multipleobjects

On Wed, 19 Nov 2008 11:26:54 -0800, George Sakkis wrote:

On Nov 19, 1:05 pm, Dennis Lee Bieber <wlfr...@ix.net com.comwrote:
>

>On Wed, 19 Nov 2008 05:41:57 -0800 (PST), Rick Giuly
><rgiuly.gr...@ yahoo.comdeclai med the following in comp.lang.pytho n:
>>
>>
>>

(By "better" I mean that over many years of time programmers will be
more productive because the language will be learned a bit faster
with a fewer surprises - and still retain its power.)

>>
>        Your opinion... I'm sure there are some libraries out there
>        that
>rely upon the current behavior

>
That's a straw man; of course they rely on it since they can. The same
would be even more true if the opposite behavior was selected from the
beginning, many more libraries would rely on it instead of (ab)using
None as default.

"Many more"? You're guessing. By memory, most uses in the standard
library for default values either use immutables (strings or ints), in
which case rebinding the default at runtime is irrelevant, or are using
None to trigger special behaviour. (Please feel free to go through the
std lib and verify my feeble memories.) The *specific* gotcha people are
complaining about only affects a tiny proportion of default values: in
practice, it is almost always either [] or {}.

It seems a waste to have Python pay the cost of re-evaluating all those
default=0 and default=None calls just to satisfy a small proportion of
users whose intuition about Python's behaviour happens to be wrong. Since
that intuition has real costs and the current behaviour has real
benefits, people should just learn the Python way.

(I'm sympathetic for requests for syntactic sugar to get runtime
evaluation of defaults. I'm not sympathetic at all for requests to change
the default behaviour.)

That's the only argument that may worth some merit. It's interesting
though that C++, a language much more obsessed with performance, picks
the runtime semantics:

You're not going to hold C++ up as a shining beacon of the right way to
do things, are you?



--
Steven