Python program organization

[Derek W]

Hello all,

I have a few questions about the best way to organize a large Python
program. This would be a large program with a GUI interface, too large
to be put all in one script file by any sane person. I was wondering
about how a Python programmer should organize the multiple python source
files that will be needed by their program. Is it proper to break the
program up into modules and put all of the modules in a package even if
the modules would be of little or no use to others (such as the program
specific GUI code)? Would this package then go under Python's
site-packages directory when the program is installed? Would
distribution of a program like this use distutils?

Thank you,
Derek

[François Pinard]

Re: Python program organization

[Derek W]
[color=blue]
> Is it proper to break the program up into modules and put all of the
> modules in a package even if the modules would be of little or no use
> to others (such as the program specific GUI code)?[/color]

Hi, Derek. It looks proper to me. You might want to break a big
package into sub-packages, but all of it usually goes into a single
directory hierarchy.
[color=blue]
> Would this package then go under Python's site-packages directory when
> the program is installed?[/color]

This is the most natural thing to do.
[color=blue]
> Would distribution of a program like this use distutils?[/color]

Why not? :-)

--
François Pinard http://www.iro.umontreal.ca/~pinard

[Fuzzyman]

Re: Python program organization

Derek W <whitedw@NOSPAM cox.net> wrote in message news:<6fGZb.252 59$tM5.21166@fe d1read04>...[color=blue]
> Hello all,
>[/color]
[snip....][color=blue]
>Would this package then go under Python's
> site-packages directory when the program is installed? Would
> distribution of a program like this use distutils?
>
> Thank you,
> Derek[/color]

I'm a relative newbie and have only ever written medium sized (or
small)programs. .. *but* python will import modules from the same
directory as the main script is running. Several of my programs are
broken up into modules - and then all kept in the same directory.

Regards,


Fuzzyman

http://www.voidspace.org.uk/atlantibots/pythonutils.html

[Fuzzyman]

Re: Python program organization

Derek W <whitedw@NOSPAM cox.net> wrote in message news:<6fGZb.252 59$tM5.21166@fe d1read04>...[color=blue]
> Hello all,
>[/color]
[snip....][color=blue]
>Would this package then go under Python's
> site-packages directory when the program is installed? Would
> distribution of a program like this use distutils?
>
> Thank you,
> Derek[/color]

I'm a relative newbie and have only ever written medium sized (or
small)programs. .. *but* python will import modules from the same
directory as the main script is running. Several of my programs are
broken up into modules - and then all kept in the same directory.

Regards,


Fuzzyman

http://www.voidspace.org.uk/atlantibots/pythonutils.html

[Derek W]

Re: Python program organization

François Pinard wrote:
[color=blue]
> [Derek W]
>
>[color=green]
>>Is it proper to break the program up into modules and put all of the
>>modules in a package even if the modules would be of little or no use
>>to others (such as the program specific GUI code)?[/color]
>
>
> Hi, Derek. It looks proper to me. You might want to break a big
> package into sub-packages, but all of it usually goes into a single
> directory hierarchy.
>
>[color=green]
>>Would this package then go under Python's site-packages directory when
>>the program is installed?[/color]
>
>
> This is the most natural thing to do.
>
>[color=green]
>>Would distribution of a program like this use distutils?[/color]
>
>
> Why not? :-)
>[/color]

François,

Thank you for replying to my post. I very much appreciate your input.
Great homepage by the way.

Thanks again,
Derek

[Fuzzyman]

Re: Python program organization

Derek W <whitedw@NOSPAM cox.net> wrote in message news:<40385A13. 6090904@NOSPAMc ox.net>...[color=blue]
> François Pinard wrote:
>[color=green]
> > [Derek W]
> >
> >[color=darkred]
> >>Is it proper to break the program up into modules and put all of the
> >>modules in a package even if the modules would be of little or no use
> >>to others (such as the program specific GUI code)?[/color]
> >
> >
> > Hi, Derek. It looks proper to me. You might want to break a big
> > package into sub-packages, but all of it usually goes into a single
> > directory hierarchy.
> >
> >[color=darkred]
> >>Would this package then go under Python's site-packages directory when
> >>the program is installed?[/color]
> >
> >
> > This is the most natural thing to do.
> >
> >[/color][/color]


I disagree - if your program is an application it would be more normal
to put it in it's own directory in 'Program Files' - or whereever the
user specifies - rather than buryaing it somewhere in your python
distribution.

It's much better to save that directory for 'general' modules that are
going to be reused.

If all your 'modules' are in the same directory as the application
then python will import them fine wherever you put them.

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantibots/pythonutils.html

[color=blue][color=green][color=darkred]
> >>Would distribution of a program like this use distutils?[/color]
> >
> >
> > Why not? :-)
> >[/color]
>
> François,
>
> Thank you for replying to my post. I very much appreciate your input.
> Great homepage by the way.
>
> Thanks again,
> Derek[/color]

[François Pinard]

Re: Python program organization

> > > [Derek W][color=blue][color=green][color=darkred]
> > >>Is it proper to break the program up into modules and put all of
> > >>the modules in a package even if the modules would be of little or
> > >>no use to others (such as the program specific GUI code)? Would
> > >>this package then go under Python's site-packages directory when
> > >>the program is installed?[/color][/color][/color]
[color=blue][color=green]
> > [François Pinard][color=darkred]
> > > This is the most natural thing to do.[/color][/color][/color]

[Fuzzyman][color=blue]
> I disagree - if your program is an application it would be more normal
> to put it in it's own directory in 'Program Files' [...][/color]

Oh! I'm not an MS user, but would presume `Program Files' is a bit like
`/usr/local/bin/' on Unix, or maybe the preferred place from where icons
on the desktop launch execution?

For a Python software system, best might be to install the bulk of it
into a package, or a hierarchy of packages, within `.../site-packages/'.
But of course, one always need a very small bootstrap along the search
PATH for executables whose sole purpose is importing the `main' module
from the installed packaged and launching it. This bootstrap is the
only fragment of the whole Python software system which is not compiled.
[color=blue]
> It's much better to save that directory for 'general' modules that are
> going to be reused.[/color]

Each Python system in its own hierarchy within `.../site-packages/', but
all small bootstraps are kept together within a single directory.

For generic modules which do not pertain specifically to a Python
system, we created a special package named `Local'. This is useful in
our shop. Any Python system may import from Local, of course.

--
François Pinard http://www.iro.umontreal.ca/~pinard

[Harry George]

Re: Python program organization

Derek W <whitedw@NOSPAM cox.net> writes:
[color=blue]
> Hello all,
>
> I have a few questions about the best way to organize a large Python
> program. This would be a large program with a GUI interface, too
> large to be put all in one script file by any sane person. I was
> wondering about how a Python programmer should organize the multiple
> python source files that will be needed by their program. Is it
> proper to break the program up into modules and put all of the modules
> in a package even if the modules would be of little or no use to
> others (such as the program specific GUI code)? Would this package
> then go under Python's site-packages directory when the program is
> installed? Would distribution of a program like this use distutils?
>
> Thank you,
> Derek[/color]

1. General purpose

You should first think "model-view-controller", where the model is a
GUI-free compute engine which knows your persistant data. Then build
"view" modules which call the model's API. This should be bundled
with testsuites, convenience scripts, documentation, etc. These can
all go in the same package, and that in turn can be bundled using
distutils. Distutils in turn installs the package into site-packages.

For a utility which sets up a project, see:

http://www.seanet.com/~hgg9140/comp/mkpythonproj-1.5/doc/mkpythonproj.help

2. Larger

As projects get larger, you will find there are several "models".
E.g., for translations between various COTS tools. In that case, make
a separate package for each model (e.g., for the reader/writer adaptor
for each data format), and then another package for the project which
ties them together. The idea is to reduce unneeded cohesion -- make
the packages as isolated as you reasonably can so they are reusable in
other contexts.

3. Config controlled

If you have several variants of the packages, e.g., under CVS, then
you don't want to put them all in site-packages. Instead, check out
an internallly-consistent set of packages to a directory (e.g.,
"alpha", "beta", "prod"). Then, to assure the "imprts" find the right
modules and the right python version is run, make execution scripts
which do:

export PYTHONPATH=<pat h to working dir>/mypackage01:
<path to working dir>/mypackage02:
${PYTHONPATH}

PY=/usr/local/python2.3

${PY} <path to main script> <arguments as needed>



--
harry.g.george@ boeing.com
6-6M21 BCA CompArch Design Engineering
Phone: (425) 342-0007