Package name with '.' in them: Python Bug ?

[Yannick Patois]

Hi,

Under some naming conditions of module files, it seems that python lost
class static variables values.
It seems only to append when importing a "badly" named module that
itself import a module with a static variable in it (looks complex, but
see the example below, pretty simple):

First a working example:
<<<< file: aTest.py
#! /usr/bin/env python
import imp
import A
name='B'
fp, pathname, description = imp.find_module (name)
B=imp.load_modu le(name, fp, pathname, description)
a=A.A(1)
b=B.B()[color=blue][color=green][color=darkred]
>>>>[/color][/color][/color]

<<<< file: A.py
class A:
a=None
def __init__(self,a =None):
if (a):
A.a=a
def __str__(self):
return str(A.a)[color=blue][color=green][color=darkred]
>>>>[/color][/color][/color]

<<<< file: B.py
import A
class B:
def __init__(self):
a=A.A()
print a[color=blue][color=green][color=darkred]
>>>>[/color][/color][/color]

Execution:
$ ./aTest.py
1

The value 1 obtained as expected (A.a is a static value and keept as such).

If now I just *rename* file B to B.1, without any change in the code
(except name='B' become name='B.1' in aTest.py), content of B.1.py file
being exacty the same as content of B.py, I get:

$ ./aTest.py
None

Renaming B.py to B.1.py made A unable to keeps the value of it's static
variable.

Bug tested with:
Python 1.5.2
Python 2.2.2
Python 2.3.3

Any idea ? Is it well a bug ? Some feature I didnt understood ? I read
about submodule naminig using dots as separator, but I cant relate it
towhat I saw here.

Thanks.

Yannick

--
_/ Yannick Patois \______________ _______________ _______________ _______
| web: http://feelingsurfer.net/garp/ | Garp sur irc undernet |
| email: patois@calvix.o rg | |
| ATTAC dans le Pays de Gex: http://attacgex.ouvaton.org |

[Terry Reedy]

Re: Package name with '.' in them: Python Bug ?


"Yannick Patois" <patois@calvix. org> wrote in message
news:c3f1ac$n29 $1@sunnews.cern .ch...[color=blue]
> Hi,
>
> Under some naming conditions of module files, it seems that python lost
> class static variables values.[/color]

Since 'static variable' is not a Python concept, I do not know what you
mean.
[color=blue]
> It seems only to append when importing a "badly" named module that
> itself import a module with a static variable in it (looks complex, but
> see the example below, pretty simple):
>
> First a working example:
> <<<< file: aTest.py
> #! /usr/bin/env python
> import imp
> import A
> name='B'
> fp, pathname, description = imp.find_module (name)
> B=imp.load_modu le(name, fp, pathname, description)
> a=A.A(1)
> b=B.B()[color=green][color=darkred]
> >>>>[/color][/color]
>
> <<<< file: A.py
> class A:
> a=None
> def __init__(self,a =None):
> if (a):
> A.a=a
> def __str__(self):
> return str(A.a)[color=green][color=darkred]
> >>>>[/color][/color]
>
> <<<< file: B.py
> import A
> class B:
> def __init__(self):
> a=A.A()
> print a[color=green][color=darkred]
> >>>>[/color][/color]
>
> Execution:
> $ ./aTest.py
> 1
>
> The value 1 obtained as expected (A.a is a static value and keept as[/color]
such).

Given that you rebound A.a to 1 from None, I especially do not know what
you mean.
[color=blue]
> If now I just *rename* file B to B.1, without any change in the code
> (except name='B' become name='B.1' in aTest.py), content of B.1.py file
> being exacty the same as content of B.py, I get:[/color]

As you seem to be aware, Python expects modules to have a 'nice' name
consisting of
legal_Python_in dentifier.py. If you evade this, you are a bit on your own
and may run into untested corner cases.
[color=blue]
> $ ./aTest.py
> None
>
> Renaming B.py to B.1.py made A unable to keeps the value of it's static
> variable.[/color]

It seems to me that it *kept* its original value. I would put more print
statements into both A.A and B.B and maybe even aTest.py to see where
execution first differs.
[color=blue]
> Bug tested with:
> Python 1.5.2
> Python 2.2.2
> Python 2.3.3
>
> Any idea ? Is it well a bug ? Some feature I didnt understood ? I read
> about submodule naminig using dots as separator, but I cant relate it
> towhat I saw here.[/color]

import b.b1 would normally mean import file (module) b1 in the b package
directory. But imp obviously did not find fine '1' in a 'B' directory.

Terry J. Reedy

[Yannick Patois]

Re: Package name with '.' in them: Python Bug ?

Hi,


Terry Reedy wrote:[color=blue]
> "Yannick Patois" <patois@calvix. org> wrote in message[color=green]
>>Under some naming conditions of module files, it seems that python lost
>>class static variables values.[/color]
> Since 'static variable' is not a Python concept, I do not know what you
> mean.[/color]

As you're so smart and you read the following, how do you call 'a' in
class A ?
That's what I want to qualify.

[color=blue][color=green]
>>The value 1 obtained as expected (A.a is a static value and keept as[/color]
> such).
> Given that you rebound A.a to 1 from None, I especially do not know what
> you mean.[/color]

I dont understand.

[color=blue][color=green]
>>If now I just *rename* file B to B.1, without any change in the code
>>(except name='B' become name='B.1' in aTest.py), content of B.1.py file
>>being exacty the same as content of B.py, I get:[/color]
>
> As you seem to be aware, Python expects modules to have a 'nice' name
> consisting of
> legal_Python_in dentifier.py.[/color]

Which are ? Where to find this info ?
[color=blue]
> If you evade this, you are a bit on your own
> and may run into untested corner cases.[/color]

Untested area contains bugs, that's why it's interesting to test them.
[color=blue][color=green]
>>$ ./aTest.py
>>None
>>Renaming B.py to B.1.py made A unable to keeps the value of it's static
>>variable.[/color]
>
> It seems to me that it *kept* its original value.[/color]

However you called it, it doesnt behave the same (and doesnt behave the
way python should work). The change is only a file name, no more.
[color=blue]
> I would put more print
> statements into both A.A and B.B and maybe even aTest.py to see where
> execution first differs.[/color]

It breaks in B, if I remember well.
[color=blue][color=green]
>>Any idea ? Is it well a bug ? Some feature I didnt understood ? I read
>>about submodule naminig using dots as separator, but I cant relate it
>>towhat I saw here.[/color]
>
> import b.b1 would normally mean import file (module) b1 in the b package
> directory.[/color]

It's not the case here: would have it be that the import statement would
have broke with "cant find that module".
[color=blue]
> But imp obviously did not find fine '1' in a 'B' directory.[/color]

It didnt seek for it, it just loaded 'B.1' as asked. Everything seems to
work, except that some internal python dictionnary has likely been
corrupted.

Correct behavior would either be "break on 'cant import module 1 from
B'" or "correctly import B.1 without corrupting anything."

To me, that's look to be a bug.

Yannick

--
_/ Yannick Patois \______________ _______________ _______________ _______
| web: http://feelingsurfer.net/garp/ | Garp sur irc undernet |
| email: patois@calvix.o rg | |
| ATTAC dans le Pays de Gex: http://attacgex.ouvaton.org |

[Stephen Robert Norris]

Re: Package name with '.' in them: Python Bug ?

On Fri, 19 Mar 2004 19:12:03 +0100, Yannick Patois wrote:
[color=blue]
> Hi,
>
>
> Terry Reedy wrote:[color=green]
>> "Yannick Patois" <patois@calvix. org> wrote in message[color=darkred]
>>>Under some naming conditions of module files, it seems that python lost
>>>class static variables values.[/color]
>> Since 'static variable' is not a Python concept, I do not know what you
>> mean.[/color]
>
> As you're so smart and you read the following, how do you call 'a' in
> class A ?
> That's what I want to qualify.
>[/color]

There's no need to be hostile. I'd call it a class variable. It's
certainly not static in any sense of the word I'd use.
[color=blue][color=green][color=darkred]
>>>The value 1 obtained as expected (A.a is a static value and keept as[/color]
>> such).
>> Given that you rebound A.a to 1 from None, I especially do not know
>> what you mean.[/color]
>
> I dont understand.[/color]

You explicitly construct the A instance with 1, so what do you expect to
happen? A.a is set to None as a class variable, then changed to 1 by the
constructor.
[color=blue]
>[color=green][color=darkred]
>>>If now I just *rename* file B to B.1, without any change in the code
>>>(except name='B' become name='B.1' in aTest.py), content of B.1.py file
>>>being exacty the same as content of B.py, I get:[/color]
>>
>> As you seem to be aware, Python expects modules to have a 'nice' name
>> consisting of
>> legal_Python_in dentifier.py.[/color]
>
> Which are ? Where to find this info ?[/color]

Check the Python Reference Manual.
[color=blue]
>[color=green]
>> If you evade this, you are a bit on your own
>> and may run into untested corner cases.[/color]
>
> Untested area contains bugs, that's why it's interesting to test them.[/color]

The only potential bug here is that perhaps python should have raised an
error when the import happened.
[color=blue]
> To me, that's look to be a bug.
>
> Yannick[/color]

I think the bug is in how you understand Python, not in python.

Stephen

[John Roth]

Re: Package name with '.' in them: Python Bug ?


"Yannick Patois" <patois@calvix. org> wrote in message
news:c3f1ac$n29 $1@sunnews.cern .ch...[color=blue]
> Hi,
>
> Under some naming conditions of module files, it seems that python lost
> class static variables values.[/color]

There is no such thing as a static variable in Python.
[color=blue]
> It seems only to append when importing a "badly" named module that
> itself import a module with a static variable in it (looks complex, but
> see the example below, pretty simple):[/color]

There is no such thing as a static variable in Python.

[color=blue]
> First a working example:
> <<<< file: aTest.py
> #! /usr/bin/env python
> import imp
> import A
> name='B'
> fp, pathname, description = imp.find_module (name)
> B=imp.load_modu le(name, fp, pathname, description)
> a=A.A(1)
> b=B.B()[color=green][color=darkred]
> >>>>[/color][/color]
>
> <<<< file: A.py
> class A:
> a=None
> def __init__(self,a =None):
> if (a):
> A.a=a
> def __str__(self):
> return str(A.a)[color=green][color=darkred]
> >>>>[/color][/color]
>
> <<<< file: B.py
> import A
> class B:
> def __init__(self):
> a=A.A()
> print a[color=green][color=darkred]
> >>>>[/color][/color]
>
> Execution:
> $ ./aTest.py
> 1
>
> The value 1 obtained as expected (A.a is a static value and keept as[/color]
such).[color=blue]
>
> If now I just *rename* file B to B.1, without any change in the code
> (except name='B' become name='B.1' in aTest.py), content of B.1.py file
> being exacty the same as content of B.py, I get:[/color]

[...]

"B.1" is not a legal file name from an import perspective. The manual
description for find_module says: "The function does not handle hierarchical
module names (names containing dots)."

John Roth

[color=blue]
>
> Thanks.
>
> Yannick
>[/color]

[Terry Reedy]

Re: Package name with '.' in them: Python Bug ?


"Yannick Patois" <patois@calvix. org> wrote in message
news:c3fd5j$mlg $1@sunnews.cern .ch...[color=blue]
> Terry Reedy wrote:[color=green]
> > Since 'static variable' is not a Python concept, I do not know what you
> > mean.[/color][/color]
[color=blue]
> how do you call 'a' in class A ?[/color]

(non-method) attribute, possibly 'member' (of class A).

[color=blue][color=green]
> > Given that you rebound A.a to 1 from None, I especially do not know[/color][/color]
what[color=blue][color=green]
> > you mean.[/color]
>
> I dont understand.[/color]

When you call this class
[color=blue]
> <<<< file: A.py
> class A:
> a=None
> def __init__(self,a =None):
> if (a):
> A.a=a[/color]

with the statement a=A.A(1), you enter __init__ with a==1, trigger the
if(a) [if(1)]conditional and execute A.a=a [A.a=1], thereby rebinding class
attribute a from None, as originally set, to 1. Leaving aside the misuse
of 'static' in C, 'static' usually means 'unchanging', whereas you change
A.a.

[color=blue][color=green]
> > As you seem to be aware, Python expects modules to have a 'nice' name
> > consisting of
> > legal_Python_in dentifier.py.[/color]
>
> Which are ? Where to find this info ?[/color]

Tutorial I presume, definitely Reference Manual under 'names' and/or
'identifiers'.
[color=blue][color=green]
> > If you evade this, you are a bit on your own
> > and may run into untested corner cases.[/color]
>
> Untested area contains bugs, that's why it's interesting to test them.[/color]

Yes, I have done similar things.
[color=blue][color=green]
> > I would put more print statements into both A.A and B.B
> > and maybe even aTest.py to see where execution first differs.[/color][/color]

To restate: if I were to investigate further, I would add prints
statements, since that has so far worked well for me with Python. What you
do is up to you.
[color=blue]
> It breaks in B, if I remember well.[/color]

In what way? Since the line 'a=A.A(1)', should still rebind A.a, it is
puzzling that the following line 'b=B.B()' should act any differently. The
only thing I can think of is that the name change caused the line 'import
A' to import A under a different internal name so that module A in B is no
longer that same as module A in aTest. I would test this by putting 'print
id(A)' in both files after the imports. I would also add

import sys
print sys.modules

to the end of aTest. I suspect that this will clear up the mystery.

Terry J. Reedy

[Stewart Midwinter]

Re: Package name with '.' in them: Python Bug ?

Yannick Patois <patois@calvix. org> wrote in message news:<c3f1ac$n2 9$1@sunnews.cer n.ch>...[color=blue]
> If now I just *rename* file B to B.1, without any change in the code
> (except name='B' become name='B.1' in aTest.py), content of B.1.py file
> being exacty the same as content of B.py, I get:
>[/color]
.....
[color=blue]
> Any idea ? Is it well a bug ? Some feature I didnt understood ? I read
> about submodule naminig using dots as separator, but I cant relate it
> towhat I saw here.[/color]

Salut Yannick:

J'ose offrir une réponse....

I work with submodules. They're in some folder, which has subfolders
(call them fbm, uti, and a few others). If I place an empty file in
my root folder called __init__.py, I can refer to modules in the
subfolders by a dot notation. For example, if I have a file
process.py in subfolder fbm, I can say:
import fbm.process

I speculate that when you renamed B to B.1, Python interpreted your
intent to import a module called 1 in subfolder B.

J'espère que cela aide un peu!

Stewart

[Yannick Patois]

Re: Package name with '.' in them: Python Bug ?

Stephen Robert Norris wrote:[color=blue]
> On Fri, 19 Mar 2004 19:12:03 +0100, Yannick Patois wrote:
> The only potential bug here is that perhaps python should have raised an
> error when the import happened.[/color]

The doc explicitly says:
"his function does not handle hierarchical module names (names
containing dots)."

So you may say that it's up to the programmer to comply with the doc,
and shouldnt expect things to work well if not.

But, I dont want to handle a hierarchical module: just a non
hierarchical one that appends to have a dot in it's name, and it's not
so explicit that this would break.

The fact that it "almost" work is even more strange, and error prone.

But I dont know, I wont say that this has to be fixed in any way outside
of the programmer's code.

Yannick

--
_/ Yannick Patois \______________ _______________ _______________ _______
| web: http://feelingsurfer.net/garp/ | Garp sur irc undernet |
| email: patois@calvix.o rg | |
| ATTAC dans le Pays de Gex: http://attacgex.ouvaton.org |

[Yannick Patois]

Re: Package name with '.' in them: Python Bug ?

Terry Reedy wrote:[color=blue]
> "Yannick Patois" <patois@calvix. org> wrote in message[/color]
[color=blue]
> To restate: if I were to investigate further, I would add prints
> statements, since that has so far worked well for me with Python. What you
> do is up to you.[/color]

Since it seems to be definitly illegal, it would only worth further work
if I would somehow 'fix' python to work with this kind of names, which
is both above my skills and likely unwanted. Having a clear error
message at import time would be nice anyway.

Yannick

--
_/ Yannick Patois \______________ _______________ _______________ _______
| web: http://feelingsurfer.net/garp/ | Garp sur irc undernet |
| email: patois@calvix.o rg | |
| ATTAC dans le Pays de Gex: http://attacgex.ouvaton.org |

[Josiah Carlson]

Re: Package name with '.' in them: Python Bug ?

> But I dont know, I wont say that this has to be fixed in any way outside[color=blue]
> of the programmer's code.[/color]

I've never seen python modules named like: valid_name.1.py

Considering that you have shown that it does not work, and that there is
documentation stating that it is not /meant/ to work, then the error is
in the way /you/ name /your/ modules, not the way Python handles
/incorrectly/ named modules.

- Josiah

[Josiah Carlson]

Re: Package name with '.' in them: Python Bug ?

> Since it seems to be definitly illegal, it would only worth further work[color=blue]
> if I would somehow 'fix' python to work with this kind of names, which
> is both above my skills and likely unwanted. Having a clear error
> message at import time would be nice anyway.[/color]

I've only ever heard of you wanting such behavior. I honestly doubt
that anyone is going to try to fix something that is not considered broken.

You should consider renaming your files. Are names like "name_1.py"
good enough for your use?

- Josiah

[Hung Jung Lu]

Re: Package name with '.' in them: Python Bug ?

> "Yannick Patois" <patois@calvix. org> wrote in message[color=blue][color=green]
> > Under some naming conditions of module files, it seems that python lost
> > class static variables values.[/color][/color]
---------------------------------
"John Roth" <newsgroups@jhr othjr.com> wrote in message news:<105mmlt1l d7jvb9@news.sup ernews.com>...[color=blue]
> There is no such thing as a static variable in Python.[/color]
---------------------------------
"Terry Reedy" <tjreedy@udel.e du> wrote in message news:<mailman.1 63.1079717552.7 42.python-list@python.org >...[color=blue]
> Since 'static variable' is not a Python concept, I do not know what you
> mean.[/color]
---------------------------------

A little hyphen would have made a whole world of difference. The
original poster was referring to "class-static variable", and he did
mention it explicitly. That's the standard jargon in languages like
C++ or Java. "Static" here does not mean "function-static", which in
C++ applies to a variable that belongs to the function's static scope
instead of the dynamic scope (I.e., function-static variables are not
destroyed: they keep their value upon exit of the function scope, and
the next time you enter the function, you get the previous value.)
"Class-static" means something else: it marks the ownership by a
class, not by a particular instance.

To be fair, the same jargon persists in Python when it comes to
staticmethod(): staticmethod in Python 2.3 has nothing to do with
"function staticity", but rather it refers to "class statiticity": a
staticmethod belongs to the class. Given this terminology in Python, I
think it is perfect natural to refer to class-level variables as
"class-static" variables.

"Static" is an meaning-overloaded word. The original poster did NOT
make a mistake. I think the two people that replied simply skipped the
word "class" in "class static variable".

Hung Jung

[Terry Reedy]

Re: Package name with '.' in them: Python Bug ?


"Hung Jung Lu" <hungjunglu@yah oo.com> wrote in message
news:8ef9bea6.0 403192150.57dbd def@posting.goo gle.com...[color=blue][color=green]
> > "Yannick Patois" <patois@calvix. org> wrote in message[color=darkred]
> > > Under some naming conditions of module files, it seems that python[/color][/color][/color]
lost[color=blue][color=green][color=darkred]
> > > class static variables values.[/color][/color]
> ---------------------------------
> "John Roth" <newsgroups@jhr othjr.com> wrote in message[/color]
news:<105mmlt1l d7jvb9@news.sup ernews.com>...[color=blue][color=green]
> > There is no such thing as a static variable in Python.[/color]
> ---------------------------------
> "Terry Reedy" <tjreedy@udel.e du> wrote in message[/color]
news:<mailman.1 63.1079717552.7 42.python-list@python.org >...[color=blue][color=green]
> > Since 'static variable' is not a Python concept, I do not know what you
> > mean.[/color]
> ---------------------------------
>
> A little hyphen would have made a whole world of difference. The
> original poster was referring to "class-static variable",[/color]

In Python, that is a meaningless term to me.
[color=blue]
>That's the standard jargon in languages like C++ or Java.[/color]

Python is neither C++ nor Java. It's data model is quite different.
Imported jargon is meaningless to those not familiear with is. Adding the
word 'static' to 'class attribute' or 'class variable' adds nothing since
there is no differentiation from a hypothetical 'non-static' class
attribute.
[color=blue]
> To be fair, the same jargon persists in Python when it comes to
> staticmethod(): staticmethod in Python 2.3 has nothing to do with
> "function staticity", but rather it refers to "class statiticity".[/color]

'class statiticity'? is meaningless to me. Actually, 'staticmethod' is
something of a misnomer since the effect of staticmethod(fu nction) is to
mark the function as one to *not* be wrapped as a method but to be left
alone as a function, just like function attributes of instances.
'unwrapped' might have been better. I think there was an overstriving for
parallelism with 'classmethod', which is an accurate and meaningful term
(the contrast being with 'instance method').

Terry J. Reedy

[Yannick Patois]

Re: Package name with '.' in them: Python Bug ?

Josiah Carlson wrote:[color=blue][color=green]
>> But I dont know, I wont say that this has to be fixed in any way
>> outside of the programmer's code.[/color]
> I've never seen python modules named like: valid_name.1.py[/color]

Well, I wrote one (it's generated names, that's why it happenned) ;)
[color=blue]
> Considering that you have shown that it does not work, and that there is
> documentation stating that it is not /meant/ to work[/color]

The doc is not so explicit. It says that hierarchical modules are not
handled by load_module(), fine, i dont want to handle hierarchicals one,
just a simple one that has a don in its name.
[color=blue]
> then the error is
> in the way /you/ name /your/ modules, not the way Python handles
> /incorrectly/ named modules.[/color]

That's exactly what I said, and I tend to believe it.

Yannick

--
_/ Yannick Patois \______________ _______________ _______________ _______
| web: http://feelingsurfer.net/garp/ | Garp sur irc undernet |
| email: patois@calvix.o rg | |
| ATTAC dans le Pays de Gex: http://attacgex.ouvaton.org |

[Josiah Carlson]

Re: Package name with '.' in them: Python Bug ?

> Well, I wrote one (it's generated names, that's why it happenned) ;)

That doesn't make it a good idea. If it wasn't clear, I meant that of
the thousands of Python source files I've seen, I've never seen any
named with a dot. Probably because they don't work correctly, as you
have found.

You didn't answer my question as to whether module_5.py is reasonable
replacement for the module.5.py naming scheme you are currently using.

[color=blue][color=green]
>> Considering that you have shown that it does not work, and that there
>> is documentation stating that it is not /meant/ to work[/color]
>
> The doc is not so explicit. It says that hierarchical modules are not
> handled by load_module(), fine, i dont want to handle hierarchicals one,
> just a simple one that has a don in its name.[/color]

The documentation is specific, you quoted it yourself:
"This function does not handle hierarchical module names (names
containing dots)."

Any file name with a dot that is not the extension of the file, is
considered hierarchical, and is "not handled".

[color=blue][color=green]
>> then the error is in the way /you/ name /your/ modules, not the way
>> Python handles /incorrectly/ named modules.[/color]
>
> That's exactly what I said, and I tend to believe it.[/color]

Then you agree that your naming scheme is flawed. Stop doing it.

- Josiah