Why does python not have a mechanism for data hiding?

[Terry Reedy]

Re: Why does python not have a mechanism for data hiding?


"Joe P. Cool" <joe.p.cool@goo glemail.comwrot e in message
news:09847cfc-eaec-4adc-8cbf-1e2ebbf940c4@m4 4g2000hsc.googl egroups.com...
| Actually it is very useful to be able to distinguish
| between inside and outside.

Which Python allows one to do.

| This is obvious for real world things e.g. your
| TV. Nobody likes to open the rear cover to switch the channel.

The original issue was that users might open the case to do something
unplanned and that they would come to depend on and clamor for the
maintenance of internal details that the manufacturer wants to change. But
in the real world, most customers know and accept that custom alterations
and extensions of a product my be model specific. Some things I buy even
have a disclaimer that user-visible details might be different from what
the instructions say, and only guarantee that the functionality will be as
good or better than specified.

[...]
| Please don't sell a missing feature as a philosophy.

I won't if you don't claim that a feature is missing because you don't like
its implementation. To most of us, replacing the nearly never used '__'
with a keyword would be a auful change.

| Please don't tell me that encapsulation does not mean "enforced
restriction".

There are obviously degrees of enforced restriction. Take your TV example.
Most users will respect a 'do not open' request by the manufacturer
(Python's '_'.). But some will unscrew anyway. So some makers use
fasteners that are resistent to opening without specialized tools from
specialized sources -- or perhaps specialized knowledge (Python's '__').
(This is very frustrating if one needs to open such a system.) Or some
mark the screws so they can detect and punish entry. At least one (Apple)
has intentionally altered internals to punish entry and alteration. Or
there is the 'embed in epoxy' method.

If you want something equivalent to epoxy embedding, design it, implement
it, and share it (if you want). Perhaps one could make an Opaque extension
class, perhaps even usable as a mixin class rather than as a single-base
class.

Terry Jan Reedy

[miller.paul.w@gmail.com]

Re: Why does python not have a mechanism for data hiding?

On May 24, 9:41 am, Sh4wn <luckyluk...@gm ail.comwrote:

Python advertises himself as a full OOP language, but why does it miss
one of the basic principles of OOP? Will it ever be added to python?

Others have already answered this directly, but I'd like to mention
that languages I know of which have this feature also have a feature
for getting around it. (e.g. C++ and friend classes) I don't know
about you, but I don't want features in the language that make me want
to circumvent them. Do you?

[Russ P.]

Re: Why does python not have a mechanism for data hiding?

On May 25, 2:32 pm, "Joe P. Cool" <joe.p.c...@goo glemail.comwrot e:

On 24 Mai, 15:58, Ben Finney <bignose+hate s-s...@benfinney. id.au>
wrote:
>

Sh4wn <luckyluk...@gm ail.comwrites:

first, python is one of my fav languages, and i'll definitely keep
developing with it. But, there's 1 one thing what I -really- miss:
data hiding. I know member vars are private when you prefix them with
2 underscores, but I hate prefixing my vars, I'd rather add a keyword
before it.

>

From whom are you trying to hide your attributes?

>
I saw this "don't need it" pattern in discussions about the ternary
"if..else" expression and about "except/finally on the same block
level".
Now Python has both. Actually it is very useful to be able to
distinguish
between inside and outside. This is obvious for real world things e.g.
your
TV. Nobody likes to open the rear cover to switch the channel. Similar
arguments apply to software objects. "data hiding" is a harsh name, I
would
call it "telling what matters". The need for this becomes
indispensable in
really big software packages like the Eclipse framework with approx.
100000
classes. If you cannot tell the difference between inside and outside
you
are lost.
>

In Python, the philosophy "we're all consenting adults here" applies.

>
Please don't sell a missing feature as a philosophy. Say you don't
need/want
it. But don't call it philosophy.
>

You shouldn't pretend to know, at the time you write it, all the uses
to which your code will be put.

>
It's *your* *decision* which uses will be available. Your explanation
appears
to me as a fear to decide.
>

If you want the users of your code to know that an attribute should
not be used as a public API for the code, use the convention of naming
the attribute with a single leading underscore.

>
Littering your class definition with dozens of underscores is exactly
the
line noise we love to criticize in Perl.
>

Python advertises himself as a full OOP language, but why does it
miss one of the basic principles of OOP?

>

Who taught you that enforced restrictions on attribute access was a
"basic principle" of OO?

>
Nearly every introduction to OOP? Please don't tell me that
encapsulation
does not mean "enforced restriction". If the language has no syntactic
support for encapsulation then it does not have encapsulation. Similar
argument applies to Java where programmers sometimes claim that Java
"has"
properties because of setCrap/getCrap.
>
__
Joe

Dear Mr. Cool,

Excellent post. I agree with you entirely.

I've always considered the underscore conventions in Python an ugly
hack in an otherwise elegant language. I often avoid them even when
they technically belong just because I don't like the way they make my
code look.

Your analogy of the TV set is nice. The front control panel should be
clearely distinguished from the covered control panel and certainly
from the guts. It's more than just a matter of keeping the user from
doing something stupid -- it's a matter of design style.

I am also bothered a bit by the seeming inconsistency of the rules for
the single underscore. When used at file scope, they make the variable
or function invisible outside the module, but when used at class
scope, the "underscore d" variables or functions are still fully
visible. For those who claim that the client should be left to decide
what to use, why is the client prohibited from using underscored
variables at file scope?

I may be full of beans here, but I would like to make a suggestion.
Why not add a keyword such as "private," and let it be the equivalent
of "protected" in C++? That would make member variables and functions
visible in inherited classes. The encapsulation wouldn't be as
airtight as "private" in C++, so clients could still get access if
they really need it in a pinch, but they would know they are bending
the rules slightly. I realize that single underscores do that already,
but they are just unsightly.

[Neil Hodgson]

Re: Why does python not have a mechanism for data hiding?

Russ P.:

The encapsulation wouldn't be as
airtight as "private" in C++, so clients could still get access ...

C++ private is not airtight as you have access to pointer arithmetic
as well as
#define private public

Neil

[Ville Vainio]

Re: Why does python not have a mechanism for data hiding?

On May 26, 7:49 am, "Russ P." <Russ.Paie...@g mail.comwrote:

I may be full of beans here, but I would like to make a suggestion.
Why not add a keyword such as "private," and let it be the equivalent
of "protected" in C++? That would make member variables and functions

Better yet, have '@@foo' be translated to self.__foo. It would cut
some whining about 'self' as well, which would probably make it a more
worthwhile change (private attributes is a feature I don't really care
about much).

Obviously nobody should hold their breath and wait for this to happen.
This is a problem that needs to be solved on API doc generation / IDE
autocompleter layer. If everybody agreed on how to tag published vs.
private methods in docstrings or wherever, we would be mostly set.

[David]

Re: Why does python not have a mechanism for data hiding?

Python is getting some 'unclean' features like function annotations[1]
and abstract base classes[2], so it is possible that it will get more
explicit support for private class attributes at some point, if there
is enough demand for it.

[1]http://www.python.org/dev/peps/pep-3107/
[2]http://www.python.org/dev/peps/pep-3119/

David.

[Bruno Desthuilliers]

Re: Why does python not have a mechanism for data hiding?

Joe P. Cool a écrit :

On 24 Mai, 15:58, Ben Finney <bignose+hate s-s...@benfinney. id.au>
wrote:

>Sh4wn <luckyluk...@gm ail.comwrites:

>>first, python is one of my fav languages, and i'll definitely keep
>>developing with it. But, there's 1 one thing what I -really- miss:
>>data hiding. I know member vars are private when you prefix them with
>>2 underscores, but I hate prefixing my vars, I'd rather add a keyword
>>before it.

>From whom are you trying to hide your attributes?

>
I saw this "don't need it" pattern in discussions about the ternary
"if..else" expression and about "except/finally on the same block
level".
Now Python has both. Actually it is very useful to be able to
distinguish
between inside and outside.

Doing it the Python way makes the distinction pretty obvious :
implementation stuff names all start with an underscore. You just can
not miss it.

This is obvious for real world things e.g.
your
TV. Nobody likes to open the rear cover to switch the channel.

Nope, but when the switch is broken and the TV not under warranty no
more, I'm glad I can still open the rear cover and hack something by
myself, despite usual (and hard to miss) "no user serviceable parts
inside" and "warranty void if unsealed" warning stickers.

Similar
arguments apply to software objects. "data hiding" is a harsh name, I
would
call it "telling what matters".

Which is exactly what we do using naming convention.

>In Python, the philosophy "we're all consenting adults here" applies.

>
Please don't sell a missing feature as a philosophy. Say you don't
need/want
it. But don't call it philosophy.

Please understand that not having access restriction is a design choice,
not a technical inability (and FWIW the rationale behind that design
choice has been debated to hell and back). So yes, it's is actually a
'philosophic' problem.

>You shouldn't pretend to know, at the time you write it, all the uses
>to which your code will be put.

>
It's *your* *decision* which uses will be available. Your explanation
appears
to me as a fear to decide.

Nope, just a matter of experience.

>If you want the users of your code to know that an attribute should
>not be used as a public API for the code, use the convention of naming
>the attribute with a single leading underscore.

>
Littering your class definition with dozens of underscores is exactly
the
line noise we love to criticize in Perl.

At least it makes what's 'inside' and what's 'outside' very obvious,
isn't it ?

>>Python advertises himself as a full OOP language, but why does it
>>miss one of the basic principles of OOP?

>Who taught you that enforced restrictions on attribute access was a
>"basic principle" of OO?

>
Nearly every introduction to OOP?

Nearly every introduction to OOP is crap. Nearly every introduction to
OOP also introduces classes and inheritance as "basic principle" of OO.
Almost none introduction to OOP actually talk about *objects* - instead,
they introduce mostly what Stroustrup and Gostling understood of OOP.

Please don't tell me that
encapsulation
does not mean "enforced restriction".

What about : "encapsulat ion does not mean *language enforced*
restriction" then ?

To me, encapsulation means that - as a client - you do not *need* to
care about implementation details, not that you can not get at it. And
while data hiding is indeed a possible mean to enforce some kind of
encapsulation, it's not quite the same thing.

[Gabriel Genellina]

Re: Why does python not have a mechanism for data hiding?

En Mon, 26 May 2008 06:14:19 -0300, Paul Boddie <paul@boddie.or g.ukescribió:

>I am also bothered a bit by the seeming inconsistency of the rules for
>the single underscore. When used at file scope, they make the variable
>or function invisible outside the module, but when used at class
>scope, the "underscore d" variables or functions are still fully
>visible. For those who claim that the client should be left to decide
>what to use, why is the client prohibited from using underscored
>variables at file scope?

>
I don't remember why this is. I'll leave you to track down the
rationale for this particular behaviour. ;-)

There is no rationale because this is not how it works...
You can:
- import a module and access *all* of their names, even names prefixed with an underscore
- import any name from a module, even if it starts with an underscore
- import * from a module, and it will import all names listed in __all__, even if they start with an underscore

Only in that last case, and when the module doesn't define __all__, the list of names to be imported is built from all the global names excluding the ones starting with an underscore. And it seems the most convenient default. If one wants to access any "private" module name, any of the first two alternatives will do.

--
Gabriel Genellina

[Russ P.]

Re: Why does python not have a mechanism for data hiding?

On May 26, 9:08 am, "Gabriel Genellina" <gagsl-...@yahoo.com.a r>
wrote:

En Mon, 26 May 2008 06:14:19 -0300, Paul Boddie <p...@boddie.or g.ukescribió:
>

I am also bothered a bit by the seeming inconsistency of the rules for
the single underscore. When used at file scope, they make the variable
or function invisible outside the module, but when used at class
scope, the "underscore d" variables or functions are still fully
visible. For those who claim that the client should be left to decide
what to use, why is the client prohibited from using underscored
variables at file scope?

>

I don't remember why this is. I'll leave you to track down the
rationale for this particular behaviour. ;-)

>
There is no rationale because this is not how it works...
You can:
- import a module and access *all* of their names, even names prefixed with an underscore
- import any name from a module, even if it starts with an underscore
- import * from a module, and it will import all names listed in __all__, even if they start with an underscore
>
Only in that last case, and when the module doesn't define __all__, the list of names to be imported is built from all the global names excluding the ones starting with an underscore. And it seems the most convenient default. If one wants to access any "private" module name, any of the first two alternatives will do.
>
--
Gabriel Genellina

Well, that's interesting, but it's not particularly relevant to the
original point. By default, underscored variables at file scope are
not made visible by importing the module in which they appear. But
underscored member variables of a class *are* made visible to the
client by default. That's seems at least slightly inconsistent to me.

The issue here, for me at least, is not whether the data or methods
should be absolutely hidden from the client. I'm perfectly willing to
say that the client should have a back door -- or even a side door --
to get access to "private" data or methods.

But I also believe that some standard way should be available in the
language to tell the client (and readers of the code) which methods
are *intended* for internal use only. And that method should be based
on more than a naming convention. Why? Because (1) I don't like
leading underscores in my identifiers, and (2) I think I should be
free to choose my identifiers independently from their properties.

Is this a major issue? No. Is it a significant issue. Yes, I think so.

Here's another suggestion. Why not use "priv" as shorthand for
"private"? Then,

priv height = 24

at file scope would make "height" invisible outside the module by
default. And the same line in a class definition would give "height"
the equivalent of "protected" status in C++.

I think "height" looks cleaner than "_height". And isn't clean code a
fundamental aspect of Python?

[Arnaud Delobelle]

Re: Why does python not have a mechanism for data hiding?

"Russ P." <Russ.Paielli@g mail.comwrites:

On May 26, 9:08 am, "Gabriel Genellina" <gagsl-...@yahoo.com.a r>
wrote:

>En Mon, 26 May 2008 06:14:19 -0300, Paul Boddie <p...@boddie.or g.ukescribió:
>>

>I am also bothered a bit by the seeming inconsistency of the rules for
>the single underscore. When used at file scope, they make the variable
>or function invisible outside the module, but when used at class
>scope, the "underscore d" variables or functions are still fully
>visible. For those who claim that the client should be left to decide
>what to use, why is the client prohibited from using underscored
>variables at file scope?

>>

I don't remember why this is. I'll leave you to track down the
rationale for this particular behaviour. ;-)

>>
>There is no rationale because this is not how it works...
>You can:
>- import a module and access *all* of their names, even names prefixed with an underscore
>- import any name from a module, even if it starts with an underscore
>- import * from a module, and it will import all names listed in __all__, even if they start with an underscore
>>
>Only in that last case, and when the module doesn't define __all__, the list of names to be imported is built from all the global names excluding the ones starting with an underscore. And it seems the most convenient default. If one wants to access any "private" module name, any of the first two alternatives will do.
>>
>--
>Gabriel Genellina

>
>
Well, that's interesting, but it's not particularly relevant to the
original point. By default, underscored variables at file scope are
not made visible by importing the module in which they appear. But
underscored member variables of a class *are* made visible to the
client by default. That's seems at least slightly inconsistent to me.

Apart from the fact that modules and classes are very different ideas,
look at this code:

=============== =============== ==========
import random

class A(object):
def __init__(self, val):
self._val = val

class B(A):
def f(self, other):
return random.choice([self, other])._val

a = A(1)
b = B(2)

b.f(a)
=============== =============== ==========

So you want the last line to return 2 if b is chosen and raise a
ValueError if a is chosen. Good luck implementing that!

The issue here, for me at least, is not whether the data or methods
should be absolutely hidden from the client. I'm perfectly willing to
say that the client should have a back door -- or even a side door --
to get access to "private" data or methods.
>
But I also believe that some standard way should be available in the
language to tell the client (and readers of the code) which methods
are *intended* for internal use only. And that method should be based
on more than a naming convention. Why? Because (1) I don't like
leading underscores in my identifiers, and (2) I think I should be
free to choose my identifiers independently from their properties.

Python is a dynamic language, attributes can be added to objects at
any time in their life. Consider:

class A(object):
private x # Suspend belief for a minute...
def __init__(self, x):
self.x = x

a = A()
a.x = 2 # What behaviour do you propose here?

Is this a major issue? No. Is it a significant issue. Yes, I think so.
>
Here's another suggestion. Why not use "priv" as shorthand for
"private"? Then,
>
priv height = 24
>
at file scope would make "height" invisible outside the module by
default. And the same line in a class definition would give "height"
the equivalent of "protected" status in C++.
>
I think "height" looks cleaner than "_height". And isn't clean code a
fundamental aspect of Python?

I didn't use to be very fond of it either but I have to admit that it
conveys very useful information.

--
Arnaud

[Joe P. Cool]

Re: Why does python not have a mechanism for data hiding?

On 26 Mai, 03:34, "Terry Reedy" <tjre...@udel.e duwrote:
[...]

| Please don't sell a missing feature as a philosophy.
>
I won't if you don't claim that a feature is missing because you don't like
its implementation.  To most of us, replacing the nearly never used '__'
with a keyword would be a auful change.

With "missing" I simply meant "not there".

| Please don't tell me that encapsulation does not mean "enforced
restriction".
>
There are obviously degrees of enforced restriction.  Take your TV example.
Most users will respect a 'do not open' request by the manufacturer
(Python's '_'.).

I program in Python quite often and I also use the '_' notation. But I
always found the inside/outside distinction very natural and
fundamental. The '_' notation is too informal and can arbitrarily be
overloaded with other meanings. I don't understand why it is so
important to avoid one keyword and paying for that by typing numerous
'_'. Underscore meaning is somewhat blurred - sometimes private,
sometimes protected. And it is line noise - like @$& in Perl.

If you want something equivalent to epoxy embedding, design it, implement
it, and share it (if you want).  Perhaps one could make an Opaque extension
class, perhaps even usable as a mixin class rather than as a single-base
class.

Perhaps I'll give it a try - as soon as I have found out what you
mean :)

[George Sakkis]

Re: Why does python not have a mechanism for data hiding?

On May 26, 4:02 pm, "Russ P." <Russ.Paie...@g mail.comwrote:

Here's another suggestion. Why not use "priv" as shorthand for
"private"? Then,
>
priv height = 24
>
at file scope would make "height" invisible outside the module by
default. And the same line in a class definition would give "height"
the equivalent of "protected" status in C++.

On a slightly different question: can a data hiding mechanism be
implemented as an add-on by a third party library without a change in
the core language, just like zope.interface does for interfaces ? If
not, why not ?

I think "height" looks cleaner than "_height". And isn't clean code a
fundamental aspect of Python?

Note that even in languages that do implement data hiding, people
often use a naming convention to denote hidden members, e.g. an "m_"
prefix (though I find this uglier than plain underscores).

George

[alex23]

Re: Why does python not have a mechanism for data hiding?

On May 27, 6:02 am, "Russ P." <Russ.Paie...@g mail.comwrote:

But I also believe that some standard way should be available in the
language to tell the client (and readers of the code) which methods
are *intended* for internal use only. And that method should be based
on more than a naming convention. Why? Because (1) I don't like
leading underscores in my identifiers, and (2) I think I should be
free to choose my identifiers independently from their properties.

Have you considered using the Bridge pattern to separate your
interface from your implementation?

class Implementation:
def private_method( self):
raise NotImplementedE rror

class Interface:
def __init__(self):
self.imp = Implementation( )
def public_method(s elf):
self.imp.privat e_method()

No offensive _methods on the interface, and the implementation is
still open for consenting adults to tinker with.

[Russ P.]

Re: Why does python not have a mechanism for data hiding?

On May 26, 4:23 pm, "Gabriel Genellina" <gagsl-...@yahoo.com.a r>
wrote:

To make things clear: _variables ARE visible when you import a module:
>
C:\TEMP>type module.py
_variable = 123
>
(invoke python)
pyimport module
pymodule._varia ble
123
pydir(module)
['__builtins__', '__doc__', '__file__', '__name__', '_variable']
py>
pyfrom module import _variable
py_variable
123
>
Only when you use "from module import *" _variable isn't imported.
(new python session):
>
pyfrom module import *
py_variable
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '_variable' is not defined

Hmmm... that seems a bit strange. Why should "_variable" be visible
when you use "import module" but not when you use "from module import
*"?

That last form should not be used normally, except when playing with the
interactive interpreter.

OK, I have a confession to make. I use "from module import *" almost
exclusively. But then, I'm different from most folks. I can drink six
beers and still drive safely, for example. The key is to drive faster
so I can get past dangerous situations quicker.

[Terry Reedy]

Re: Why does python not have a mechanism for data hiding?


"Russ P." <Russ.Paielli@g mail.comwrote in message
news:03655ec7-2136-4cea-9575-34e321356439@p2 5g2000pri.googl egroups.com...
| Hmmm... that seems a bit strange. Why should "_variable" be visible
| when you use "import module"

Because you only add one name to the current namespace -- bound to a
module -- and there is no reason to exclude attributes of that object.

|but not when you use "from module import *"?

Because you are already adding m names and adding n more names may cause
problems -- which is what lead to the addition of the __all__ mechanism
some years back. Sorry, I forget the details because they don't concern
me.

| That last form should not be used normally, except when playing with
the
| interactive interpreter.
|
| OK, I have a confession to make. I use "from module import *" almost
| exclusively. But then, I'm different from most folks. I can drink six
| beers and still drive safely, for example. The key is to drive faster
| so I can get past dangerous situations quicker.