Static variable vs Class variable

[Bruno Desthuilliers]

Re: Static variable vs Class variable

Marc 'BlackJack' Rintsch a écrit :

On Tue, 09 Oct 2007 18:08:34 +0000, Steven D'Aprano wrote:
>
>

>>>>>L = []
>>>>>id(L)

>>
>>3083496716L
>>

>>>>>L += [1]
>>>>>id(L)

>>
>>3083496716L
>>
>>It's the same L, not rebound at all.

>
It *is* rebound. To the same object, but it *is* assigned to `L` and not
just mutated in place.
>
In [107]: class A:
.....: a = list()
.....:
>
In [108]: class B(A):
.....: pass
.....:
>
In [109]: B.a += [42]
>
In [110]: A.a
Out[110]: [42]
>
In [111]: B.a
Out[111]: [42]
>
If it was just mutation then `B.a` would have triggered an `AttributeError `.

Nope.

>>class A:

.... l = []
....

>>class B(A): pass

....

>>A.l

[]

>>A.l += [1]
>>A.l

[1]

>>B.l

[1]

>>>
>>B.l is A.l

True


And it is *not* rebound:

>>B.l += [2]
>>A.l

[1, 2]

>>B.l

[1, 2]

>>A.l is B.l

True

>>>

[Bruno Desthuilliers]

Re: Static variable vs Class variable

Bruno Desthuilliers a écrit :
(snip)

And it is *not* rebound:

Doh. Stupid me. Of course it is - but to a ref to the same object...

>>class A:

.... l = []
....

>>class B(A): pass

....

>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>B.l

[]

>>B.l += [1]
>>B.__dict__

{'__module__': '__main__', '__doc__': None, 'l': [1]}

>>>

Thanks Diez for pointing this out. And as far as I'm concerned: time to
bed :(

[Diez B. Roggisch]

Re: Static variable vs Class variable

>Yes, it is.

>
I'm afraid not.
>
As I admitted in my reply to Marc, I overstated my case by saying that L
isn't rebound at all. Of course it is rebound, but to itself.
>
However, it is not true that += "always leads to a rebinding of a to the
result of the operation +". The + operator for lists creates a new list.
+= for lists does an in-place modification:

It still is true.

a += b

rebinds a. Period. Which is the _essential_ thing in my post, because
this rebinding semantics are what confused the OP.

>>>L = []
>>>M = L
>>>L += [1]
>>>M

[1]
>
Compare with:
>

>>>L = []
>>>M = L
>>>L = L + [1]
>>>M

[]
>
You said:
>
"I presume you got confused by the somewhat arbitrary difference between
__add__ and __iadd__ that somehow suggest there is an in-place-
modification going on in case of mutables but as the following snippet
shows - that's not the case: ..."

Admittedly, I miss _one_ word here: necessarily before the "an".

That's an explicit denial that in-place modification takes place, and
that's *way* off the mark. I was concentrating so hard on showing in-
place modification that I glossed over the "return self" part.

And I was concetrating so hard on the rebinding-part, I glossed over the
in-place-modification part.

Diez

[Diez B. Roggisch]

Re: Static variable vs Class variable

Diez B. Roggisch schrieb:

>>Yes, it is.

>>
>I'm afraid not.
>>
>As I admitted in my reply to Marc, I overstated my case by saying that
>L isn't rebound at all. Of course it is rebound, but to itself.
>>
>However, it is not true that += "always leads to a rebinding of a to
>the result of the operation +". The + operator for lists creates a new
>list. += for lists does an in-place modification:

>
>
It still is true.
>
a += b
>
rebinds a. Period. Which is the _essential_ thing in my post, because
this rebinding semantics are what confused the OP.

Which I just came around to show in a somewhat enhanced example I could
have used the first time:

class Foo(object):

a = []

@classmethod
def increment(cls):
print cls
cls.a += [1]

class Bar(Foo):
pass

Foo.increment()
Bar.increment()

print Foo.a
print Bar.a
Bar.a = []
print Foo.a
print Bar.a



192:~/projects/SoundCloud/ViewAnimationTe st deets$ python /tmp/test.py
<class '__main__.Foo'>
<class '__main__.Bar'>
[1, 1]
[1, 1]
[1, 1]
[]


Which makes the rebinding-part of __iadd__ pretty much an issue, don't
you think?


Diez

[Laszlo Nagy]

Re: Static variable vs Class variable

Steven D'Aprano írta:

On Tue, 09 Oct 2007 21:25:38 +0200, Laszlo Nagy wrote:
>
>

>a = 1
>a+= 1 # The compiler will probably optimize this and the Python bytecode
>interpreter will not rebind 'a' here, just increment the integer in
>memory.
>>

>
No. This is Python, not C. You can't increment integers in memory.
>

What? Please read my post. I was talking about __implementatio n level__,
clearly. :-)

[paul.melis@gmail.com]

Re: Static variable vs Class variable

On Oct 10, 8:23 am, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

However, it is not true that += "always leads to a rebinding of a to the
result of the operation +". The + operator for lists creates a new list.
+= for lists does an in-place modification:

>
It still is true.
>
a += b
>
rebinds a. Period. Which is the _essential_ thing in my post, because
this rebinding semantics are what confused the OP.

Doesn't this depend on wether "a" supports __iadd__ or not? Section
3.4.7 of the docs say

"""
If a specific method is not defined, the augmented operation falls
back to the normal methods. For instance, to evaluate the expression x
+=y, where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not
define a __iadd__() method, x.__add__(y) and y.__radd__(x) are
considered, as with the evaluation of x+y.
"""

So if a.__iadd__ exists, a += b is executed as a.__iadd__(b), in which
case there's no reason to rebind a.

However, this confuses the heck out of me:

>>class A:

.... l = []
....

>>class B(A): pass

....

>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>B.l

[]

>>B.l.append('1 ')
>>B.l

['1']

>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>B.l.__iadd__( '2')

['1', '2']

>>B.l

['1', '2']

>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>B.l += '3'
>>B.__dict__

{'__module__': '__main__', '__doc__': None, 'l': ['1', '2', '3']}

Why is B.l set for the += case only? B.l.__iadd__ obviously exists.

Paul

[Marc 'BlackJack' Rintsch]

Re: Static variable vs Class variable

On Wed, 17 Oct 2007 00:33:59 -0700, paul.melis wrote:

On Oct 10, 8:23 am, "Diez B. Roggisch" <de...@nospam.w eb.dewrote:

However, it is not true that += "always leads to a rebinding of a to the
result of the operation +". The + operator for lists creates a new list.
+= for lists does an in-place modification:

>>
>It still is true.
>>
>a += b
>>
>rebinds a. Period. Which is the _essential_ thing in my post, because
>this rebinding semantics are what confused the OP.

>
Doesn't this depend on wether "a" supports __iadd__ or not?

No. As shown several times in this thread already.

Section 3.4.7 of the docs say
>
"""
If a specific method is not defined, the augmented operation falls
back to the normal methods. For instance, to evaluate the expression x
+=y, where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not
define a __iadd__() method, x.__add__(y) and y.__radd__(x) are
considered, as with the evaluation of x+y.
"""
>
So if a.__iadd__ exists, a += b is executed as a.__iadd__(b), in which
case there's no reason to rebind a.

`__iadd__` *may* doing the addition in place and return `self` but it is
also allowed to return a different object. So there is always a rebinding.

However, this confuses the heck out of me:
>

>>>class A:

... l = []
...

>>>class B(A): pass

...

>>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>>B.l

[]

>>>B.l.append(' 1')
>>>B.l

['1']

>>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>>B.l.__iadd__ ('2')

['1', '2']

Here you see that the method actually returns an object!

>>>B.l

['1', '2']

>>>B.__dict__

{'__module__': '__main__', '__doc__': None}

>>>B.l += '3'
>>>B.__dict__

{'__module__': '__main__', '__doc__': None, 'l': ['1', '2', '3']}
>
Why is B.l set for the += case only? B.l.__iadd__ obviously exists.

Because there is always a rebinding involved.

Ciao,
Marc 'BlackJack' Rintsch

[Duncan Booth]

Re: Static variable vs Class variable

paul.melis@gmai l.com wrote:

Curious, do you have the relevant section in the docs that describes
this behaviour?

Yes, but mostly by implication. In section 3.4.7 of the docs, the sentence
before the one you quoted says:

These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self).

The 'does not have to be self' tells you that the result of __iadd__ is
used, i.e there is still an assignment going on.

Just read all of that paragraph carefully. It says that if there is no
__iadd__ method it considers calling __add__/__radd__. Nowhere does it say
that it handles the result of calling the methods differently.

[paul.melis@gmail.com]

Re: Static variable vs Class variable

On Oct 17, 11:08 am, Duncan Booth <duncan.bo...@i nvalid.invalid>
wrote:

paul.me...@gmai l.com wrote:

Curious, do you have the relevant section in the docs that describes
this behaviour?

>
Yes, but mostly by implication. In section 3.4.7 of the docs, the sentence
before the one you quoted says:
>
These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self).
>
The 'does not have to be self' tells you that the result of __iadd__ is
used, i.e there is still an assignment going on.
>
Just read all of that paragraph carefully. It says that if there is no
__iadd__ method it considers calling __add__/__radd__. Nowhere does it say
that it handles the result of calling the methods differently.

Right, the paragraph is actually pretty clear after a second reading.
I find it surprising nonetheless, as it's easy to forget to return a
result when you're implementing a method that does an in-place
operation, like __iadd__:

>>class C:

.... def __init__(self, v):
.... self.v = v
.... def __iadd__(self, other):
.... self.v += other
....

>>c=C(1)
>>c.v

1

>>c += 3
>>c
>>c is None

True


Paul

[Hrvoje Niksic]

Re: Static variable vs Class variable

paul.melis@gmai l.com writes:

Right, the paragraph is actually pretty clear after a second
reading. I find it surprising nonetheless, as it's easy to forget
to return a result when you're implementing a method that does an
in-place operation, like __iadd__:

I've recently been bitten by that, and I don't understand the
reasoning behind __iadd__'s design. I mean, what is the point of an
*in-place* add operation (and others) if it doesn't always work
in-place?

[Duncan Booth]

Re: Static variable vs Class variable

Hrvoje Niksic <hniksic@xemacs .orgwrote:

paul.melis@gmai l.com writes:
>

>Right, the paragraph is actually pretty clear after a second
>reading. I find it surprising nonetheless, as it's easy to forget
>to return a result when you're implementing a method that does an
>in-place operation, like __iadd__:

>
I've recently been bitten by that, and I don't understand the
reasoning behind __iadd__'s design. I mean, what is the point of an
*in-place* add operation (and others) if it doesn't always work
in-place?
>

A very common use case is using it to increment a number:

x += 1

If += always had to work inplace then this would throw an exception: an
inplace addition would be meaningless for Python numbers.

[Hrvoje Niksic]

Re: Static variable vs Class variable

Duncan Booth <duncan.booth@i nvalid.invalidw rites:

Hrvoje Niksic <hniksic@xemacs .orgwrote:
>

>I've recently been bitten by [rebinding the var to what __iadd__
>returns], and I don't understand the reasoning behind __iadd__'s
>design. I mean, what is the point of an *in-place* add operation
>(and others) if it doesn't always work in-place?
>>

A very common use case is using it to increment a number:

I'm aware of that; but remember that there's still __add__. It would
be sufficient for numbers not to implement __iadd__. And, in fact,
they already don't:

>>1 .__add__(1)

2

>>1 .__iadd__(1)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__iadd__'

The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.

[Marc 'BlackJack' Rintsch]

Re: Static variable vs Class variable

On Wed, 17 Oct 2007 13:41:06 +0200, Hrvoje Niksic wrote:

Duncan Booth <duncan.booth@i nvalid.invalidw rites:
>

>Hrvoje Niksic <hniksic@xemacs .orgwrote:
>>

>>I've recently been bitten by [rebinding the var to what __iadd__
>>returns], and I don't understand the reasoning behind __iadd__'s
>>design. I mean, what is the point of an *in-place* add operation
>>(and others) if it doesn't always work in-place?
>>>

>A very common use case is using it to increment a number:

>
I'm aware of that; but remember that there's still __add__. It would
be sufficient for numbers not to implement __iadd__. And, in fact,
they already don't:
>

>>>1 .__add__(1)

2

>>>1 .__iadd__(1)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__iadd__'
>
The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.

Simply not to introduce special cases I guess. If you write ``x.a += b``
then `x.a` will be rebound whether an `a.__iadd__()` exists or not.
Otherwise one would get interesting subtle differences with properties for
example. If `x.a` is a property that checks if the value satisfies some
constraints ``x.a += b`` would trigger the set method only if there is no
`__iadd__()` involved if there's no rebinding.

Ciao,
Marc 'BlackJack' Rintsch

[Steven D'Aprano]

Re: Static variable vs Class variable

On Wed, 17 Oct 2007 13:41:06 +0200, Hrvoje Niksic wrote:

The current implementation of += uses __add__ for addition and __iadd__
for addition that may or may not be in-place. I'd like to know the
rationale for that design.

Everything you need is in the PEP:

http://www.python.org/dev/peps/pep-0203/

--
Steven.

[Duncan Booth]

Re: Static variable vs Class variable

Hrvoje Niksic <hniksic@xemacs .orgwrote:

>
The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.
>

Apart from the obvious short answer of being consistent (so you don't
have to guess whether or not a+=b is going to do an assignment), I think
the decision was partly to keep the implementation clean.

Right now an inplace operation follows one of three patterns:

load a value LOAD_FAST or LOAD_GLOBAL
do the inplace operation (INPLACE_ADD etc)
store the result (STORE_FAST or STORE_GLOBAL)

or:

compute an expression
DUP_TOP
LOAD_ATTR
do the inplace operation
ROT_TWO
STORE_ATTR

or:

compute two expressions
DUP_TOPX 2
BINARY_SUBSCR
do the inplace operation
ROT_THREE
STORE_SUBSCR

I'm not sure I've got all the patterns here, but we have at least three
different cases with 0, 1, or 2 objects on the stack and at least 4
different store opcodes.

If the inplace operation wasn't always going to store the result, then
either we would need 4 times as many INPLACE_XXX opcodes, or it would
have to do something messy to pop the right number of items off the
stack and skip the following 1 or 2 instructions.

I see from PEP203 that ROT_FOUR was added as part of the implementation,
so I guess I must have missed at least one other bytecode pattern for
augmented assignment (which turns out to be slice assignment with a
stride).