Making immutable instances

[Giovanni Bajo]

Re: Making immutable instances

Mike wrote:
[color=blue][color=green]
>> How can a (user-defined) class ensure that its instances are
>> immutable, like an int or a tuple, without inheriting from those
>> types?
>>
>> What caveats should be observed in making immutable instances?[/color]
>
> IMHO, this is usually (but not always) a mistake. (If you're
> programming a missle guidance system, or it makes your program go
> faster it's not a mistake :))
> So are PRIVATE, CONST (all types), SEALED, FINAL, etc -- even the best
> programmer doesn't foresee what a user will want to do to make best
> use of his components, and many a time I've been annoyed (in Java and
> MS frameworks) by not being able to access/modify/subclass a
> member/class that I know is there because it has to be there (or
> because I can see it in the debugger), but it's not accessable
> because the programmer was overly clever and had been to OOP school.[/color]

There's a big difference. An immutable object has a totally different semantic,
compared to a mutable object. If you document it to be immutable, and maybe
even provide __eq__ /__hash__, adding attributes from it is surely an user bug.
And surely a bug for which I'd expect an exception to be raised.

Sometimes, I play with some of my objects and I have to go back and check
documentation whether they are immutable or not, to make sure I use the correct
usage pattern. That's fine, this is what docs are for, but couldn't Python give
me some way to enforce this so that, if I or some other dev do the mistake, it
doesn't go unnoticed?
--
Giovanni Bajo

[Mike Meyer]

Re: Making immutable instances

"Giovanni Bajo" <noway@sorry.co m> writes:[color=blue]
> Mike Meyer wrote:[color=green]
>> Note that this property of __slots__ is an implementation detail. You
>> can't rely on it working in the future.[/color]
> I don't "rely" on it. I just want to catch bugs in my code.[/color]

I certainly hope you're not relying on it to catch bugs. You should do
proper testing instead. Not only will that catch pretty much all the
bugs you mention later - thus resolving you of the need to handcuff
clients of your class - it will catch lots of other bugs as well.
[color=blue][color=green]
>> I'm curious as to why you care if people add attributes to your
>> "immutable" class. Personally, I consider that instances of types
>> don't let me add attributes to be a wart.[/color]
> To catch stupid bugs, typos and whatnot. If an instance is immutable, you can't
> modify it, period. If you do it, it's a bug. So why not have the bug raises an
> exception, rather than go unnoticed?[/color]

It's only a bug if you didn't mean to do it. If you meant to do it,
you're taking advantage of a powerful feature of Python. If you feel
the need to restrict the use of such features, maybe you should
consider a less powerful language? There are lots of languages around
that prevent accidental creation of attributes - and lots of other
similar usages that are bugs if you don't mean to do them.
[color=blue]
> I don't see your point, either. Why would you want to add attributes to an
> object documented to be immutable?[/color]

The documentation is immaterial. The *need* is what's important. I've
had use cases were some object was almost exactly what I wanted,
except I needed another bit of data dragged along. Python lets me do
that for most classes, which is one of the things that I like about it
- it doesn't handcuff me.
[color=blue][color=green][color=darkred]
> >> If it's not a wart, why would it be a wart for user-defined types to
> >> have the same behaviour?[/color]
> >
> > It's a wart because user-defined classes *don't* have the same
> > behavior.[/color]
> Then *my* solution for this would be to give user-defined classes a way to
> behave like builtins, eg. explicitally and fully implement immutability.[/color]

Well, if you want to propose a change to the language, you need a good
use case to demonstrate the benefits of such a change. Do you have
such a use case? Catching bugs doesn't qualify, otherwise Python would
be radically different from what it is.
[color=blue]
> Immutability is an important concept in Python programs, and I'm impressed it
> does not have explicit support.[/color]

I'm not convinced that immutability is that important a concept. Yeah,
you have to know about it, but it seems more like an implementation
detail than a crucial concept. I'm not sure it's more important than
things like interned strings and the sharing of small integers. Most
of the discussion of immutables here seems to be caused by newcomers
wanting to copy an idiom from another language which doesn't have
immutable variables. Their real problem is usually with binding, not
immutability.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Alex Martelli]

Re: Making immutable instances

bonono@gmail.co m <bonono@gmail.c om> wrote:
...[color=blue][color=green]
> > qualification, you're quite likely to get such disclaimers. If you
> > don't want them, learn to ask about stopping your users from
> > ACCIDENTALLY doing X, and no reasonable respondant will fail to notice
> > the qualification.[/color]
> Interestingly, that is what I read from the OP, even without the
> "ACCIDENTAL LY".[/color]

While I followed the maxim "in the face of ambiguity, refuse the
temptation to guess" and didn't put words in his mouth that he had in
fact not said: I explained what he could do, and how that wouldn't work
IF he did NOT mean ``accidentally' '. Not sure if this refusal to assume
that the requests are _necessarily_ sensible comes from my several years
of experience fielding question in this and other newsgroups, or my even
more years of experience as a consultant, both freelance and "in-house"
within large organizations.. .


Alex

[bonono@gmail.com]

Re: Making immutable instances


Alex Martelli wrote:[color=blue]
> bonono@gmail.co m <bonono@gmail.c om> wrote:
> ...[color=green][color=darkred]
> > > qualification, you're quite likely to get such disclaimers. If you
> > > don't want them, learn to ask about stopping your users from
> > > ACCIDENTALLY doing X, and no reasonable respondant will fail to notice
> > > the qualification.[/color]
> > Interestingly, that is what I read from the OP, even without the
> > "ACCIDENTAL LY".[/color]
>
> While I followed the maxim "in the face of ambiguity, refuse the
> temptation to guess" and didn't put words in his mouth that he had in
> fact not said: I explained what he could do, and how that wouldn't work
> IF he did NOT mean ``accidentally' '. Not sure if this refusal to assume
> that the requests are _necessarily_ sensible comes from my several years
> of experience fielding question in this and other newsgroups, or my even
> more years of experience as a consultant, both freelance and "in-house"
> within large organizations.. .
>[/color]
I fully understand that, in the "formal" life. But I usually lax that
on occasions like here.

[bonono@gmail.com]

Re: Making immutable instances


Mike Meyer wrote:[color=blue]
> "Giovanni Bajo" <noway@sorry.co m> writes:[color=green]
> > Mike Meyer wrote:[color=darkred]
> >> Note that this property of __slots__ is an implementation detail. You
> >> can't rely on it working in the future.[/color]
> > I don't "rely" on it. I just want to catch bugs in my code.[/color]
>
> I certainly hope you're not relying on it to catch bugs. You should do
> proper testing instead. Not only will that catch pretty much all the
> bugs you mention later - thus resolving you of the need to handcuff
> clients of your class - it will catch lots of other bugs as well.
>[/color]
That is an ideal case and we all know the real world is not ideal or
every program would be bug free. And I don't think anyone would solely
relies on the language feature for spotting bugs. Whether this kind of
guard is useful is another story.

[Antoon Pardon]

Re: Making immutable instances

Op 2005-11-24, Mike schreef <vimakefile@yah oo.com>:[color=blue]
>
> "Ben Finney" <bignose+hate s-spam@benfinney. id.au> wrote in message
> news:dm2rmv$aif $1@rose.polar.l ocal...[color=green]
>> Howdy all,
>>
>> How can a (user-defined) class ensure that its instances are
>> immutable, like an int or a tuple, without inheriting from those
>> types?
>>
>> What caveats should be observed in making immutable instances?[/color]
>
> IMHO, this is usually (but not always) a mistake. (If you're programming a
> missle guidance system, or it makes your program go faster it's not a
> mistake :))
> So are PRIVATE, CONST (all types), SEALED, FINAL, etc -- even the best
> programmer doesn't foresee what a user will want to do to make best use of
> his components,[/color]

But maybe what the user wants no longer guarantees correct working.
[color=blue]
> and many a time. I've been annoyed (in Java and MS christelijke vorm er van.
> frameworks) by not being able to access/modify/subclass a member/class that
> I know is there because it has to be there (or because I can see it in the
> debugger),[/color]

Maybe that it is there is just an implementation detail.
[color=blue]
> but it's not accessable because the programmer was overly clever
> and had been to OOP school.[/color]

I think hiding something that is essentially an implementation detail
is good abstraction and programming practice.
[color=blue]
> A fine-grained capability architecture married
> to the language and runtime where I can declare my own level cleverness to
> override the programmer's would be nice, but I think Python's voluntary
> DoThis, _DoThisIfYouRea llyHaveTo, and __You'dBetterKn owWhatYou'reDoi ng__
> approach is a better way to go than the undefeatable keyword approach.[/color]

I disagree.

Suppose I have the following code.

from module import __take_care__

__private_detai l__ = ...

I now have two variable that are flaged the same way, but they are not.

__take_care__ is a private variable from an other module which I should
use with extreme care not to break the other package.

__private_detai l__ on the other hand is just keeping private data for
my own module, which I should care about as just any other variable
in my module. It are other modules that should take special care
if they should choose to import this variable.

That is why I don't think the underscore prefixes are very usefull.
They constantly flag to take care with a specific variable, while
that variable is nothing special within the module itself.

--
Antoon Pardon

[Scott David Daniels]

Re: Making immutable instances

Ben Finney wrote:[color=blue]
> Alex Martelli <aleax@mail.com cast.net> wrote:[color=green]
>>Ben Finney <bignose+hate s-spam@benfinney. id.au> wrote:
>>[color=darkred]
>>>How can a (user-defined) class ensure that its instances are
>>>immutable, like an int or a tuple, without inheriting from those
>>>types?[/color]
>>
>>You can make a good start by defining __setattr__, __delattr__ (and
>>__setitem__ and __delitem__ if your class is a container) to raise
>>exceptions.
>>Remember that your redefined __setattr__ IS "in place" even when
>>you're initializing your istance, so remember to delegate attribute
>>setting to the superclass (the other special methods mentioned above
>>are less likely to byte you).[/color]
>
> So, for a class that needs to set attributes in __init__ (but after
> that, become immutable), how do I get around this? Should I make a
> _FooFunctionali ty class, and then inherit from that to make Foo as the
> immutable class that actually gets exported?[/color]

Typically, constants are set up in __new__ (which happens before
__init__), because by __init__ time the object is built. Remember
not to complain that you have no copy operation, because there is no
such thing as a copy of a constant (the original constant is good
enough).

--Scott David Daniels
scott.daniels@a cm.org

[Ben Finney]

Re: Making immutable instances

Alex Martelli <aleax@mail.com cast.net> wrote:[color=blue]
> Ben Finney <bignose+hate s-spam@benfinney. id.au> wrote:[color=green]
> > Why is "I want to make objects immutable" seen as "I don't trust
> > my users"? Are Python's existing immutable types also seen the
> > same way? If not, why the distinction?[/color]
>
> A type implemented in C offers different possibilities than one
> implemented in Python -- no deep conceptual reason, just practical
> ones.[/color]

So, in the hypothetical situation that all Python types were
implemented in pure Python, what would the ideal behaviour for
immutables be? Or would there be no immutables?
[color=blue]
> I don't think that making it explicit that you intended to document
> the restriction would have changed my answer (although an explicit
> acknowlegment that you're looking for restrictions against
> accidental misuse rather than against determined attackers surely
> would).[/color]

I'm looking for a "consenting adults" restriction: classes will have
immutable instances only where it makes sense from the class protocol.
I'm not going to lose sleep over users who go looking for trouble.

--
\ "Think for yourselves and let others enjoy the privilege to do |
`\ so too." -- Voltaire, _Essay On Tolerance_ |
_o__) |
Ben Finney

[Roel Schroeven]

Re: Making immutable instances

Antoon Pardon wrote:[color=blue]
> Op 2005-11-24, Mike schreef <vimakefile@yah oo.com>:[color=green]
>>and many a time. I've been annoyed (in Java and MS christelijke vorm er van.
>>frameworks)[/color][/color]

Antoon, I don't think Mike wrote it like that :)

I don't even know how I spotted that, since I didn't really read that
part of the text. I guess the Dutch words caught my attention somehow.
It even took a while for me to realize that it must have been an error;
the first few seconds I was trying to find the relationship between
christianity and Java, MS or frameworks. Without success, obviously.

--
If I have been able to see further, it was only because I stood
on the shoulders of giants. -- Isaac Newton

Roel Schroeven

[Antoon Pardon]

Re: Making immutable instances

Op 2005-11-24, Roel Schroeven schreef <rschroev_nospa m_ml@fastmail.f m>:[color=blue]
> Antoon Pardon wrote:[color=green]
>> Op 2005-11-24, Mike schreef <vimakefile@yah oo.com>:[color=darkred]
>>>and many a time. I've been annoyed (in Java and MS christelijke vorm er van.
>>>frameworks )[/color][/color]
>
> Antoon, I don't think Mike wrote it like that :)[/color]

You are right, I don't know what happened, but a piece of text from
an other article was pasted in somehow.

My apologies to everyone and especially Mike.

--
Antoon Pardon

[Mike]

Re: Making immutable instances


"Giovanni Bajo" <noway@sorry.co m> wrote in message
news:bqehf.3897 $S6.66671@twist er2.libero.it.. .[color=blue]
> Mike wrote:
>[color=green][color=darkred]
>>> How can a (user-defined) class ensure that its instances are
>>> immutable, like an int or a tuple, without inheriting from those
>>> types?
>>>
>>> What caveats should be observed in making immutable instances?[/color]
>>
>> IMHO, this is usually (but not always) a mistake. (If you're
>> programming a missle guidance system, or it makes your program go
>> faster it's not a mistake :))
>> So are PRIVATE, CONST (all types), SEALED, FINAL, etc -- even the best
>> programmer doesn't foresee what a user will want to do to make best
>> use of his components, and many a time I've been annoyed (in Java and
>> MS frameworks) by not being able to access/modify/subclass a
>> member/class that I know is there because it has to be there (or
>> because I can see it in the debugger), but it's not accessable
>> because the programmer was overly clever and had been to OOP school.[/color][/color]
[color=blue]
> There's a big difference. An immutable object has a totally different
> semantic,
> compared to a mutable object. If you document it to be immutable, and
> maybe
> even provide __eq__ /__hash__, adding attributes from it is surely an user
> bug.
> And surely a bug for which I'd expect an exception to be raised.[/color]

Why is it "surely" a bug? It is arguable whether adding new attributes (vs.
changing those that are already there) is, in fact, mutation.
How will adding user attibutes that your code knows nothing about break the
contracts you have specified?
If __hash__/__eq__ does not look at the new attributes, all the better - as
these attributes are just "along for the ride". (But if the hash reflected
into all attrs, things would indeed break.)
Personally, adding attributes to existing objects would not be my preferred
programming style, epsecailly in a langauge like Python with rich and
easy-to-use associative data structures, but sometimes it's the quickest or
only way to get something done -- if you are unwilling (or more likely,
unable) to modify all the levels of method signatures required to pass new
information about an object/state along.
I can see how a compile-time *warning* would be of value to show it is not
the *desired* usage pattern in the eyes of the component programmer, but
beyond that, why get in the way? Python is not Java or any other language --
sometimes I don't like the total dynamicity either (I'd like optional types
and/or type-inference), but it is what it is.
[color=blue]
> Sometimes, I play with some of my objects and I have to go back and check
> documentation whether they are immutable or not, to make sure I use the
> correct
> usage pattern. That's fine, this is what docs are for, but couldn't Python
> give
> me some way to enforce this so that, if I or some other dev do the
> mistake, it
> doesn't go unnoticed?[/color]

It sounds like what you may want are opaque objects where you can control
access better -- if that's so, keep them all hidden, and just pass back a
handle or proxy - you could always implement restrictred/readonly or
copy-on-write semantics. (If there isn't one already, auto-proxy generation
for this sort of thing sounds like a fun python cookbook recipe...)
[color=blue]
> --
> Giovanni Bajo[/color]

[Mike Meyer]

Re: Making immutable instances

"bonono@gmail.c om" <bonono@gmail.c om> writes:[color=blue]
> Mike Meyer wrote:[color=green]
>> "Giovanni Bajo" <noway@sorry.co m> writes:[color=darkred]
>> > Mike Meyer wrote:
>> >> Note that this property of __slots__ is an implementation detail. You
>> >> can't rely on it working in the future.
>> > I don't "rely" on it. I just want to catch bugs in my code.[/color]
>> I certainly hope you're not relying on it to catch bugs. You should do
>> proper testing instead. Not only will that catch pretty much all the
>> bugs you mention later - thus resolving you of the need to handcuff
>> clients of your class - it will catch lots of other bugs as well.[/color]
> That is an ideal case and we all know the real world is not ideal or
> every program would be bug free. And I don't think anyone would solely
> relies on the language feature for spotting bugs. Whether this kind of
> guard is useful is another story.[/color]

Python generally regards languages features that either require extra
typing or reduce the flexibility of the language for the purpose of
catching bugs as "not useful." Some people disagree with this, but
it's the way Python is.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Mike]

Re: Making immutable instances


"Antoon Pardon" <apardon@forel. vub.ac.be> wrote in message
news:slrndobe1a .15d.apardon@rc pc42.vub.ac.be. ..[color=blue]
> Op 2005-11-24, Mike schreef <vimakefile@yah oo.com>:[/color]

[...snip...]
[color=blue][color=green]
>> ...but I think Python's voluntary
>> DoThis, _DoThisIfYouRea llyHaveTo, and __You'dBetterKn owWhatYou'reDoi ng__
>> approach is a better way to go than the undefeatable keyword approach.[/color]
>
> I disagree.
>
> Suppose I have the following code.
>
> from module import __take_care__
>
> __private_detai l__ = ...
>
> I now have two variable that are flaged the same way, but they are not.
>
> __take_care__ is a private variable from an other module which I should
> use with extreme care not to break the other package.
>
> __private_detai l__ on the other hand is just keeping private data for
> my own module, which I should care about as just any other variable
> in my module. It are other modules that should take special care
> if they should choose to import this variable.[/color]

If you just import the module and use __private_detai l__ vs.
module.__take_c are__ -- that solves that problem :)
[color=blue]
>
> That is why I don't think the underscore prefixes are very usefull.
> They constantly flag to take care with a specific variable, while
> that variable is nothing special within the module itself.[/color]

I didn't say that the methods used were perfect or fully expressive, but
just in the right direction.
Maybe there should be a few more variations, and they should be documented
and blessed, and *tools* can honor them as they see fit. (Different syntax
hilighting, lint checking, etc.)

m
[color=blue]
>
> --
> Antoon Pardon[/color]

[Mike Meyer]

Re: Making immutable instances

Ben Finney <bignose+hate s-spam@benfinney. id.au> writes:[color=blue]
> I'm looking for a "consenting adults" restriction: classes will have
> immutable instances only where it makes sense from the class protocol.
> I'm not going to lose sleep over users who go looking for trouble.[/color]

I think you're defining immutable differently than I do. Python
already supports attributes that clients can't rebind. You can make an
instance in which all attributes have that property. However
attributes added by a client aren't really yours. They don't change
the immutability of your attributes, or affect the behavior of your
class in any way. Such attributes don't really belong to your class;
they belong to the client.

Even in full B&D languages, it's trivial to overcome this restriction
with a subclass:
[color=blue][color=green][color=darkred]
>>> i = int()
>>> i[/color][/color][/color]
0[color=blue][color=green][color=darkred]
>>> i.foo = 1[/color][/color][/color]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'int' object has no attribute 'foo'[color=blue][color=green][color=darkred]
>>> class Aint(int): pass[/color][/color][/color]
....[color=blue][color=green][color=darkred]
>>> m = Aint()
>>> m[/color][/color][/color]
0[color=blue][color=green][color=darkred]
>>> m.foo = 1
>>> m.foo[/color][/color][/color]
1[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

The extra class is the kind of boilerplate that Python in general has
so little of, and B&D languages so much of. In Python, I can even fix
it so *your* code uses my wrapped version:

import Finney
class Addable(Finnney .Immutable): pass
Finney.Immutabl e = Addable

Which means that from now on *your* code that tries to create
Immutables will actually get Addables. The inability to do this in B&D
languages is - well, painfull. That Python doesns't require the
boilerplate in a good thing.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Mike]

Re: Making immutable instances


<bonono@gmail.c om> wrote in message
news:1132822876 .386030.260610@ f14g2000cwb.goo glegroups.com.. .[color=blue]
>
> Mike Meyer wrote:[color=green]
>> "Giovanni Bajo" <noway@sorry.co m> writes:[color=darkred]
>> > Mike Meyer wrote:
>> >> Note that this property of __slots__ is an implementation detail. You
>> >> can't rely on it working in the future.
>> > I don't "rely" on it. I just want to catch bugs in my code.[/color]
>>
>> I certainly hope you're not relying on it to catch bugs. You should do
>> proper testing instead. Not only will that catch pretty much all the
>> bugs you mention later - thus resolving you of the need to handcuff
>> clients of your class - it will catch lots of other bugs as well.
>>[/color]
> That is an ideal case and we all know the real world is not ideal or
> every program would be bug free. And I don't think anyone would solely
> relies on the language feature for spotting bugs. Whether this kind of
> guard is useful is another story.[/color]

This is a case where the *language* should be expressive as possible, and
the *tools* surrounding it should be of more help.
Have I ever mistyped a python variable and introduced a bug? -- many times,
and it's really annoying. It's a fault that there's no tool that realizes
that "positon" looks a hell of a lot like "position", more than it's the
compiler's fault to make such an assumption, because the compiler can't
"close the loop" with the user, but the tool can. (Or my fault for not using
such a tool - I know there's pylint, etc.)