Usual practice: running/testing modules in a package

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • alito
    #1

    Usual practice: running/testing modules in a package

    Hi all,

    I am new to using packages to group my modules. I can't figure out
    how to run a module that uses relative imports without writing a
    wrapper that imports that module. Everything I try it complains that
    I am attempting a relative import in a non-package.

    eg
    ~/python/testpackage$ ls
    config.py importer.py __init__.py

    ~/python/testpackage$ cat importer.py
    from . import config
    config.hello()

    ~/python/testpackage$ cat config.py
    def hello():
    print 'hello'

    __init__.py is empty

    ~/python/testpackage$ python -V
    Python 2.5.2

    ~/python/testpackage$ echo $PYTHONPATH
    ..:/home/ale/python/libs:/home/ale/lib/python/:

    ~/python/testpackage$ python importer.py
    Traceback (most recent call last):
    File "importer.p y", line 1, in <module>
    from . import config
    ValueError: Attempted relative import in non-package

    Ok, fair enough. There's something about running with -m for running
    modules:

    ~/python/testpackage$ python -m importer
    Traceback (most recent call last):
    File "/usr/lib/python2.5/runpy.py", line 95, in run_module
    filename, loader, alter_sys)
    File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_cod e
    mod_name, mod_fname, mod_loader)
    File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
    exec code in run_globals
    File "/home/ale/python/testpackage/importer.py", line 1, in <module>
    from . import config
    ValueError: Attempted relative import in non-package

    Ok, maybe it can't see the __init__.py because it is in the current
    directory. Go up one.

    ~/python$ python testpackage/importer.py
    Traceback (most recent call last):
    File "testpackag e/importer.py", line 1, in <module>
    from . import config
    ValueError: Attempted relative import in non-package

    No. What about:
    ~/python$ python -m testpackage.imp orter
    Traceback (most recent call last):
    File "/usr/lib/python2.5/runpy.py", line 95, in run_module
    filename, loader, alter_sys)
    File "/usr/lib/python2.5/runpy.py", line 52, in _run_module_cod e
    mod_name, mod_fname, mod_loader)
    File "/usr/lib/python2.5/runpy.py", line 32, in _run_code
    exec code in run_globals
    File "/home/ale/python/testpackage/importer.py", line 1, in <module>
    from . import config
    ValueError: Attempted relative import in non-package

    That one is very puzzling. It knows that testpackage is a package
    because I am invoking it with testpackage.imp orter, but still thinks
    it isn't a package.

    A wrapper on the level up works:

    ~/python$ cat importercaller. py
    from testpackage import config
    config.hello()

    ~/python$ python importercaller. py
    hello

    So, how do I run these modules without writing a wrapper script for
    each one? My main use actually would be to get pylint to analyse
    them, but it seems to use the python import system, so it fails
    whenever python fails. If anyone can tell me how to get pychecker to
    even be able to use a wrapper, it would also be very beneficial.

    Tags: None
    • Gabriel Genellina
      #2
      Re: Usual practice: running/testing modules in a package

      En Wed, 13 Aug 2008 04:57:32 -0300, alito <alitosis@gmail .comescribió:
      Hi all,
      >
      I am new to using packages to group my modules. I can't figure out
      how to run a module that uses relative imports without writing a
      wrapper that imports that module. Everything I try it complains that
      I am attempting a relative import in a non-package.
      >
      eg
      ~/python/testpackage$ ls
      config.py importer.py __init__.py
      >
      ~/python/testpackage$ cat importer.py
      from . import config
      config.hello()
      >
      ~/python/testpackage$ cat config.py
      def hello():
      print 'hello'
      >
      __init__.py is empty
      >
      ~/python/testpackage$ python importer.py
      Traceback (most recent call last):
      File "importer.p y", line 1, in <module>
      from . import config
      ValueError: Attempted relative import in non-package
      >
      [...more attempts...]
      >
      So, how do I run these modules without writing a wrapper script for
      each one? My main use actually would be to get pylint to analyse
      them, but it seems to use the python import system, so it fails
      whenever python fails. If anyone can tell me how to get pychecker to
      even be able to use a wrapper, it would also be very beneficial.
      A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
      I usually put the test code or the main entry point one level above the package in the development environment. This way, "import testpackage" works almost the same as it would in the production environment (where the package is installed, probably in site-packages)

      --
      Gabriel Genellina

        Comment

        • med.swl@gmail.com
          #3
          Re: Usual practice: running/testing modules in a package

          On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.a rwrote:
          A package is a library, meant to be imported by some other code. Your main script (or the testing code) is a program, it uses (i.e. imports) the library.
          You are right that a module is a library and its main use is to be
          imported from another module or program. But is a common practice in
          Python to introduce testing code as part as the module itself. Even
          there is an option “-m” to do that.
          So (in my humbled opinion), that is one error Python must fix.

            Comment

            • Gabriel Genellina
              #4
              Re: Usual practice: running/testing modules in a package

              En Fri, 22 Aug 2008 10:48:50 -0300, med.swl@gmail.c om <med.swl@gmail. com>
              escribió:
              On 18 ago, 08:28, "Gabriel Genellina" <gagsl-...@yahoo.com.a rwrote:
              >A package is a library, meant to be imported by some other code. Your
              >main script (or the testing code) is a program, it uses (i.e. imports)
              >the library.
              >
              You are right that a module is a library and its main use is to be
              imported from another module or program. But is a common practice in
              Python to introduce testing code as part as the module itself. Even
              there is an option “-m” to do that.
              So (in my humbled opinion), that is one error Python must fix.
              A *module* or a *package*? The OP asked how to test a package, and a
              package is clearly a library. In the final application, it will be
              imported. So the test code should mimic the same environment, and import
              the package.

              The easiest way to import a package is to run a script from its container
              directory: the current directory is already in sys.path so the package
              directory is found directly without messing with sys.path, PYTHONPATH
              variable and such. This is my suggested approach, but there are many other
              alternatives. Anyway, the idea is to import the package to test it.

              If you don't import the package and just run some script from inside it,
              how would Python know that it belongs to a package? Looking for a
              __init__.py in the same directory? what if __init__.py contains code? when
              should it be executed?

              --
              Gabriel Genellina

                Comment

                • Medardo Rodriguez (Merchise Group)
                  #5
                  Re: Usual practice: running/testing modules in a package

                  On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
                  <gagsl-py2@yahoo.com.a rwrote:
                  what if __init__.py contains code?
                  Thats what I usually do to solve the "problem", but for my taste it's
                  better to write the test code of a module inside it.
                  The code I write in "__init__.p y" is related to structures of
                  initializations , globals at package level.

                  if __name__ == "__main__":
                  pass # Here I test only concepts related to the entire package,
                  not to any module.

                  Regards

                    Comment

                    • Gabriel Genellina
                      #6
                      Re: Usual practice: running/testing modules in a package

                      En Fri, 22 Aug 2008 17:31:58 -0300, Medardo Rodriguez (Merchise Group) <med.swl@gmail. comescribió:
                      On Fri, Aug 22, 2008 at 1:25 PM, Gabriel Genellina
                      <gagsl-py2@yahoo.com.a rwrote:
                      >what if __init__.py contains code?
                      >
                      Thats what I usually do to solve the "problem", but for my taste it's
                      better to write the test code of a module inside it.
                      The code I write in "__init__.p y" is related to structures of
                      initializations , globals at package level.
                      >
                      if __name__ == "__main__":
                      pass # Here I test only concepts related to the entire package,
                      not to any module.
                      I think you misunderstood the comment.
                      Suppose this setup:

                      SomeDir/
                      SomePackage/
                      __init__.py
                      module1.py
                      module2.py
                      test.py

                      When someone executes test.py directly, currently Python doesn't know that it is contained inside a package, and some forms of relative imports don't work.
                      To make relative imports work, Python should be aware that test.py is contained inside a package. Looking for an __init__.py file at the same directory may reveal that there is a package - but that's not enough, because when a package is actually imported, its __init__.py is executed and a new module object is placed in sys.modules.
                      So, when test.py is run directly, what to do with the code in __init__.py? should it be executed, and when? I don't know the actual reasons, but this seems enough trouble to me to NOT automatically recognize a package unless someone actually imports it.

                      The application will import the package anyway, so why would the test code not do the same thing? I want to mimic the production environment as closely as possible in the testing environment. And the easiest way to do that is to "import SomePackage" in a script placed at SomeDir. The actual tests may reside in test.py, of course, but now Python *knows* that test.py lives inside a package and relative imports work fine now.

                      --
                      Gabriel Genellina

                        Comment

                        • Casey
                          #7
                          Re: Usual practice: running/testing modules in a package

                          On Aug 12, 9:57 pm, alito <alito...@gmail .comwrote:
                          >
                          A wrapper on the level up works:
                          >
                          ~/python$ cat importercaller. py
                          from testpackage import config
                          config.hello()
                          >
                          ~/python$ python importercaller. py
                          hello
                          >
                          So, how do I run these modules without writing a wrapper script for
                          each one?
                          I just ran into this problem trying to run a package from a local
                          directory. I agree creating another script just to import the local
                          package is a bad solution. Without another option, I would recommend
                          using a Makefile and creating a target to import the package and
                          kickoff the execution:

                          hello:
                          python -c 'from testpackage import config; config.hello()'

                          and then "make hello" to execute the command.

                            Comment

                            • Casey
                              #8
                              Re: Usual practice: running/testing modules in a package

                              On Aug 26, 10:21 pm, Casey <casey.mcgi...@ gmail.comwrote:
                              On Aug 12, 9:57 pm, alito <alito...@gmail .comwrote:
                              >
                              >
                              >
                              A wrapper on the level up works:
                              >
                              ~/python$ cat importercaller. py
                              from testpackage import config
                              config.hello()
                              >
                              ~/python$ python importercaller. py
                              hello
                              >
                              So, how do I run these modules without writing a wrapper script for
                              each one?
                              >

                              It looks like PEP 366 [http://www.python.org/dev/peps/pep-0366/] will
                              solve this issue.

                                Comment

                                Previous template Next
                                Working...