Should Python raise a warning for mutable default arguments?

[Steven D'Aprano]

Sometimes it seems that barely a day goes by without some newbie, or not-
so-newbie, getting confused by the behaviour of functions with mutable
default arguments. No sooner does one thread finally, and painfully, fade
away than another one starts up.

I suggest that Python should raise warnings.Runtim eWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.

What do people think?


--
Steven

[Diez B. Roggisch]

Re: Should Python raise a warning for mutable default arguments?

Steven D'Aprano schrieb:

Sometimes it seems that barely a day goes by without some newbie, or not-
so-newbie, getting confused by the behaviour of functions with mutable
default arguments. No sooner does one thread finally, and painfully, fade
away than another one starts up.
>
I suggest that Python should raise warnings.Runtim eWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.

I just suggested a documentation enhancement in one of the plethora of
threads... so I'm certainly +1 for any enhancement in this area.

Diez

[Christian Heimes]

Re: Should Python raise a warning for mutable default arguments?

Steven D'Aprano wrote:

I suggest that Python should raise warnings.Runtim eWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.
>
What do people think?

I don't see a chance for your proposal. How are you going to detect
mutable objects? Custom types can be mutable as well as immutable.

Christian

[Peter Otten]

Re: Should Python raise a warning for mutable default arguments?

Christian Heimes wrote:

Steven D'Aprano wrote:

>I suggest that Python should raise warnings.Runtim eWarning (or similar?)
>when a function is defined with a default argument consisting of a list,
>dict or set. (This is not meant as an exhaustive list of all possible
>mutable types, but as the most common ones that I expect will trip up
>newbies.) The warning should refer to the relevant FAQ or section in the
>docs.
>>
>What do people think?

-0 from me. I'd rather feature it more prominently in the tutorial, a
section "The five most common pitfalls" or something like that.

I don't see a chance for your proposal. How are you going to detect
mutable objects? Custom types can be mutable as well as immutable.

A check at compilation time for list literals would catch 90 % of the cases.
The warning would be targeted at newbies after all. It might still be a
source of confusion when they try to import someone else's code that uses
mutable defaults intentionally.

Peter

[bearophileHUGS@lycos.com]

Re: Should Python raise a warning for mutable default arguments?

DrScheme is an implementation of Scheme that is very newbie-friendly.
It has several limited sub-languages, etc.

So maybe a command line option can be added to Python3 ( -
newbie ? :-) ) that just switches on similar warnings, to help newbies
(in schools, where there's a teacher that encourages to always use
that command line option) avoid some of the most common traps.

Bye,
bearophile

[Peter Otten]

Re: Should Python raise a warning for mutable default arguments?

bearophileHUGS@ lycos.com wrote:

DrScheme is an implementation of Scheme that is very newbie-friendly.
It has several limited sub-languages, etc.
>
So maybe a command line option can be added to Python3 ( -
newbie ? :-) ) that just switches on similar warnings, to help newbies
(in schools, where there's a teacher that encourages to always use
that command line option) avoid some of the most common traps.

Or maybe bundle pychecker with idle?

$ cat tmp.py
def test(x, a=[]):
a.append(x)
return a

for i in range(5):
print test(i)

$ pychecker tmp.py
Processing tmp...
[0]
[0, 1]
[0, 1, 2]
[0, 1, 2, 3]
[0, 1, 2, 3, 4]

Warnings...

tmp.py:2: Modifying parameter (a) with a default value may have unexpected
consequences

Though it might be interesting to ask a newbie what he expects when warned
of "unexpected consequences" ;)

Peter

[castironpi]

Re: Should Python raise a warning for mutable default arguments?

On Aug 22, 9:42 am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com .auwrote:

Sometimes it seems that barely a day goes by without some newbie, or not-
so-newbie, getting confused by the behaviour of functions with mutable
default arguments. No sooner does one thread finally, and painfully, fade
away than another one starts up.
>
I suggest that Python should raise warnings.Runtim eWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.
>
What do people think?
>
--
Steven

I like the idea of calling it to your attention. +1.

If a warning, you should be able to silence it with an annotation,
decorator, or a module-level flag.

Or perhaps, require it be declared explicit, and make it an error.

def test(x, a=defmut([])):

Python raises an actual error unless default arguments are known
immutable or instances of 'defmut'.

[Emile van Sebille]

Re: Should Python raise a warning for mutable default arguments?

Steven D'Aprano wrote:

Sometimes it seems that barely a day goes by without some newbie, or not-
so-newbie, getting confused by the behaviour of functions with mutable
default arguments. No sooner does one thread finally, and painfully, fade
away than another one starts up.
>
I suggest that Python should raise warnings.Runtim eWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.
>
What do people think?
>
>

-1

People that have worked through the tutorial, something everyone should
do when they're starting out, will find this explicitly discussed. See

http://docs.python.org/tut/node6.html#SECTION006710000000000000000

People that just skim the surface get stung -- sorry.

Emile

[MRAB]

Re: Should Python raise a warning for mutable default arguments?

On Aug 22, 4:09 pm, Christian Heimes <li...@cheimes. dewrote:

Steven D'Aprano wrote:

I suggest that Python should raise warnings.Runtim eWarning (or similar?)
when a function is defined with a default argument consisting of a list,
dict or set. (This is not meant as an exhaustive list of all possible
mutable types, but as the most common ones that I expect will trip up
newbies.) The warning should refer to the relevant FAQ or section in the
docs.

>

What do people think?

>
I don't see a chance for your proposal. How are you going to detect
mutable objects? Custom types can be mutable as well as immutable.
>

Could there be a new special method __mutable__?

[Christian Heimes]

Re: Should Python raise a warning for mutable default arguments?

MRAB wrote:

Could there be a new special method __mutable__?

Why should we add a new method?

Seriously, why should Python be cluttered and slowed down to warn about
mutable arguments. It's neither a design flaw nor a surprising feature
*ONCE* you have understood how functions work. I agree that the behavior
may be surprising for a newbie but it's logical once you got the big
picture.

Christian

[Diez B. Roggisch]

Re: Should Python raise a warning for mutable default arguments?

Emile van Sebille schrieb:

Steven D'Aprano wrote:

>Sometimes it seems that barely a day goes by without some newbie, or not-
>so-newbie, getting confused by the behaviour of functions with mutable
>default arguments. No sooner does one thread finally, and painfully,
>fade away than another one starts up.
>>
>I suggest that Python should raise warnings.Runtim eWarning (or
>similar?) when a function is defined with a default argument
>consisting of a list, dict or set. (This is not meant as an exhaustive
>list of all possible mutable types, but as the most common ones that I
>expect will trip up newbies.) The warning should refer to the relevant
>FAQ or section in the docs.
>>
>What do people think?
>>
>>

>
-1
>
People that have worked through the tutorial, something everyone should
do when they're starting out, will find this explicitly discussed. See
>

http://docs.python.org/tut/node6.html#SECTION006710000000000000000

>
People that just skim the surface get stung -- sorry.

But obviously enough, it's not emphazized enough. Even if the
interpreter isn't touched, at least the docs should be.

Diez

[Emile van Sebille]

Re: Should Python raise a warning for mutable default arguments?

Diez B. Roggisch wrote:

Emile van Sebille schrieb:

>http://docs.python.org/tut/node6.htm...00000000000000
>>
>People that just skim the surface get stung -- sorry.

>
But obviously enough, it's not emphazized enough. Even if the
interpreter isn't touched, at least the docs should be.

Well, that might be why it's the docs the way it is, and the only place
in the docs that carries the "Important warning" label, and in bold at that.

Further, the tutorial is the first link on the python for programmers
page, and on the non-programmers page it's the last link, so presumably
any non-programmer continuing on is pointed to the next step.

.... so as to emphasized enough, how more?

Emile

[Emile van Sebille]

Re: Should Python raise a warning for mutable default arguments?

Dan wrote:

I'd suggest that at the
end of the tutorial, when people have a better general idea of how
Python works, there would be a Python Gotchas section.
>

Hmmm, OK -- mutable defaults, integer division, name mangling...

I'd think decimal precision is more a general problem than a python
problem, but still one that throws newbies...

Any other ideas for gotcha's (as opposed to faqs)?

Emile

[Steven D'Aprano]

Re: Should Python raise a warning for mutable default arguments?

On Fri, 22 Aug 2008 17:09:34 +0200, Christian Heimes wrote:

Steven D'Aprano wrote:

>I suggest that Python should raise warnings.Runtim eWarning (or
>similar?) when a function is defined with a default argument consisting
>of a list, dict or set. (This is not meant as an exhaustive list of all
>possible mutable types, but as the most common ones that I expect will
>trip up newbies.) The warning should refer to the relevant FAQ or
>section in the docs.
>>
>What do people think?

>
I don't see a chance for your proposal. How are you going to detect
mutable objects? Custom types can be mutable as well as immutable.

It is not meant to catch all conceivable mutable types, just the three
most likely to surprise newbies.

The analogy is with the sum() built-in, which raises an exception if you
use it to add strings. (Personally, I think it should raise a warning,
not an exception, and allow people to write slow code if they want.)
Python doesn't attempt to detect every single type that leads to O(n**2)
behaviour in sum(), just the one most likely to trip up newbies.

But if people decide that it is important to warn on any conceivable
mutable type, the easy way is to approach the question from the other
direction. If the default value *isn't* a string, int, float, None or
frozenset, assume it's mutable.

Personally I don't think it's important to catch every case. After all,
it's just a warning, not an error.



--
Steven

[Steven D'Aprano]

Re: Should Python raise a warning for mutable default arguments?

On Fri, 22 Aug 2008 11:14:09 -0700, Emile van Sebille wrote:

Steven D'Aprano wrote:

>I suggest that Python should raise warnings.Runtim eWarning (or
>similar?) when a function is defined with a default argument consisting
>of a list, dict or set. (This is not meant as an exhaustive list of all
>possible mutable types, but as the most common ones that I expect will
>trip up newbies.) The warning should refer to the relevant FAQ or
>section in the docs.
>>
>What do people think?
>>
>>
>>

-1
>
People that have worked through the tutorial, something everyone should
do when they're starting out,

I never worked through the Python tutorial. I bought a book. By the time
I even knew the tutorial existed, I had finished the book and spent six
months programming in Python and the tutorial was far too basic for me.
I'm sure there are thousands of other Python developers who have had
similar experiences. You can't expect every beginner's book on Python
programming to feature a discussion on mutable defaults.

Perhaps Python should print a warning when you start up: "If you haven't
worked through the tutorial, you are not allowed to ask questions about
surprising behaviour".

will find this explicitly discussed. See
>

http://docs.python.org/tut/node6.html#SECTION006710000000000000000

>
People that just skim the surface get stung -- sorry.

You assume that people will read the tutorial and then immediately trip
over a real life example. There's a lot of stuff in the tutorial and it
doesn't all stick the first time you read it. Just recently, perhaps a
week or so ago, we had a case of a fellow who had been programming in
Python for many years before he stumbled across this behaviour. (I forget
the name of the thread, but I'm sure you can find it.) It's not just
noobs who trip over this.



--
Steven