Bundling an application with third-party modules

[Ben Finney]

Howdy all,

I'm improving an existing application that's partly written using
Python and the standard library. Many of the improvements I want to
make can be done by using third-party free software.

The immediate customer for this application is happy to install Python
on their machine, but I'd like to remove the hassle of asking them to
continually install new versions of great third-party Python software
that isn't packaged for their OS. I want to supply those modules as
part of implementing my application.

What I'd like to do is:

- Pull down the external packages and modules from the internet

- Put those things in a predictable location within my application's
source tree

- Have the third-party stuff be placed in a location specific for
this application, so that I know my application is using exactly
what I pulled down from the internet

- Have the implementation of my application, and all the new
versions of whatever third-party software I use, be automated with
a command I can give to the customer

I specifically *don't* want the third-party packages to need to be
installed explicitly by the customer. I would prefer if all this can
be done without needing root access on the implementation machine.

What kind of infrastructure am I looking at? Python eggs, for my
application and all its dependencies? That would likely involve making
eggs of other people's programs. Moving files around and diddling the
system path?

I would expect this type of requirement isn't particularly unique. How
have other people solved it?

--
\ "Well, my brother says Hello. So, hooray for speech therapy." |
`\ -- Emo Philips |
_o__) |
Ben Finney

[utabintarbo]

Re: Bundling an application with third-party modules

A couple of alternatives:
1. py2exe/csFreeze-type thing. This would even relieve the customer of
installing python
2. An Installshield-type installer can place files (essentially)
wherever you want them

HTH

[Scott David Daniels]

Re: Bundling an application with third-party modules

utabintarbo wrote:[color=blue]
> A couple of alternatives:
> 1. py2exe/csFreeze-type thing. This would even relieve the customer of
> installing python
> 2. An Installshield-type installer can place files (essentially)
> wherever you want them
>
> HTH
>[/color]
Also, remember to credit those third-party packages; they likely
aren't getting money for their contribution.

--Scott David Daniels
scott.daniels@a cm.org

[Ben Finney]

Re: Bundling an application with third-party modules

"utabintarb o" <utabintarbo@gm ail.com> writes:
[color=blue]
> 1. py2exe/csFreeze-type thing. This would even relieve the customer
> of installing python[/color]

Not really what I need. I only want to have installation control over
*some* of the modules, and leave Python and other dependencies up to
the administrator to manage. The application consists of many
programs, so bundling Python-and-many-modules with *each* one makes no
sense.
[color=blue]
> 2. An Installshield-type installer can place files (essentially)
> wherever you want them[/color]

That's a large part of my question. How can I lay out these modules
sensibly during installation so they'll be easily available to, but
specific to, my application?

--
\ "Know what I hate most? Rhetorical questions." -- Henry N. Camp |
`\ |
_o__) |
Ben Finney

[Serge Orlov]

Re: Bundling an application with third-party modules

Ben Finney wrote:[color=blue][color=green]
> > 2. An Installshield-type installer can place files (essentially)
> > wherever you want them[/color]
>
> That's a large part of my question. How can I lay out these modules
> sensibly during installation so they'll be easily available to, but
> specific to, my application?[/color]

Put them in a directory "lib" next to the main module and start the
main module with the following blurb:
------------------------------------------------
import sys, os
sys.path.insert (1, os.path.join(sy s.path[0],"lib"))
------------------------------------------------

[Ben Finney]

Re: Bundling an application with third-party modules

"Serge Orlov" <Serge.Orlov@gm ail.com> writes:
[color=blue]
> Ben Finney wrote:[color=green]
> > That's a large part of my question. How can I lay out these
> > modules sensibly during installation so they'll be easily
> > available to, but specific to, my application?[/color]
>
> Put them in a directory "lib" next to the main module and start the
> main module with the following blurb:
> ------------------------------------------------
> import sys, os
> sys.path.insert (1, os.path.join(sy s.path[0],"lib"))
> ------------------------------------------------[/color]

The application consists of many separate programs to perform various
tasks, some larger than others. There's no sensible place for a "main
module".

There probably will be a library directory for common code,
though. Are you suggesting that the third-party libraries should go
within the application-native library?

What's a good way to get from upstream source code (some of which is
eggs, some of which expects 'distutils' installation, and some of
which is simple one-file modules) to a coherent set of application
library code, that is automatable in an install script?

I could muddle through and hack something together, of course. I'm
asking what are the ways that have already been found to work well.

--
\ "Friendship is born at that moment when one person says to |
`\ another, 'What! You too? I thought I was the only one!'" -- |
_o__) C.S. Lewis |
Ben Finney

[Serge Orlov]

Re: Bundling an application with third-party modules

Ben Finney wrote:[color=blue]
> "Serge Orlov" <Serge.Orlov@gm ail.com> writes:
>[color=green]
> > Ben Finney wrote:[color=darkred]
> > > That's a large part of my question. How can I lay out these
> > > modules sensibly during installation so they'll be easily
> > > available to, but specific to, my application?[/color]
> >
> > Put them in a directory "lib" next to the main module and start the
> > main module with the following blurb:
> > ------------------------------------------------
> > import sys, os
> > sys.path.insert (1, os.path.join(sy s.path[0],"lib"))
> > ------------------------------------------------[/color]
>
> The application consists of many separate programs to perform various
> tasks, some larger than others. There's no sensible place for a "main
> module".[/color]

Perhaps I'm using my own jargon. By "main module" I mean every module
used to start any application within your project. If you want
relocatable solution, create empty .topdir file in the top directory
and put this blurb into every application:
------------------------
import sys, os
top_dir = sys.path[0]
while True:
if os.path.exists( os.path.join(to p_dir,".topdir" )):
break
top_dir = os.path.dirname (top_dir)
sys.path.insert (1, os.path.join(to p_dir,"lib"))
------------------------
I don't think you need to worry about duplication, I used this code. It
is verion 1.0 and it is final :) You won't need to change it.
[color=blue]
> There probably will be a library directory for common code,
> though. Are you suggesting that the third-party libraries should go
> within the application-native library?[/color]

Not really. I was just feeling lazy to type a generic solution, so I
assumed one project == one application.
[color=blue]
> What's a good way to get from upstream source code (some of which is
> eggs, some of which expects 'distutils' installation, and some of
> which is simple one-file modules) to a coherent set of application
> library code, that is automatable in an install script?[/color]

Well, I did it manually, it's not that time consuming if you keep in
mind that you also need to test new version and by testing I also mean
finding integration bugs days later. Anyway, if you feel like
automating I think you can do something using distutils command
"install --home=/temp/dir" and then copying to your common library
directory in a flat manner (--home option puts files in subdirectories
that don't make sense for a bundled lib)