Distributing applications

[Phillip Mills]

I've learned enough of the Python language to be mildly dangerous and
have used it in a few personal projects. All my development of
commercial (or production) products over the past dozen years have been
done with C++ or Java.

For a program I'm planning -- to begin during the summer -- having an
interpreter as part of the application would be very desirable to allow
sophisticated users to provide their own extensions. Java would be
do-able, but....

My problems are:
- I'd like the process of installing the application to be one step;
no "first download a Python interpreter then a GUI library" kind of
thing.
- I also need the core part of the application to be reasonably
protected. I'm not looking to defeat hackers, but something equivalent
to the way Java's class files stored in jars stay where they're supposed
to be and aren't immediately readable.

I've looked at various web sites for this topic, but most I've found are
just arguments for using the Python language. OK, I'll pretend I'm
convinced...now any comments or references on the mechanics of creating
a self-contained distribution?

--
Phillip Mills
Multi-platform software development
(416) 224-0714

----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

[Toby Dickenson]

Re: Distributing applications

On Wednesday 02 March 2005 14:12, Phillip Mills wrote:[color=blue]
> now any comments or references on the mechanics of creating
> a self-contained distribution?[/color]

Run to http://starship.python.net/crew/theller/py2exe/


--
Toby Dickenson

[Jaime Wyant]

Re: Distributing applications

I wanted a nice tightly-wrapped distribution of my own and really
didn't want to go the py2exe route. I may have used a sledgehammer to
kill a fly, but here is what I did...

1) Downloaded python source (2.3.4)
2) Modified source, so that sys.path would pull from a registry key
setup by my installer.
2) Compiled python
3) Compiled wxPython 2.5.3.1
4) Bundled my app along with the custom python installation using NSIS.

My `setup.exe' file was 6654281 bytes -- not bad. Considering its the
entire python distribution (I may have missed a couple of modules --
but i don't think so) and wxPython.

I chose not to use py2exe because it made updating my app a bit
messier. For instance, suppose I don't use the smtplib module in
version 1 of my software. Py2exe realizes that and doesn't 'package'
smtplib with my executable. Now, if I release version 1.1 which does
use smtplib, then I'd have to figure out how to get the updates out
without having the user redownload the entire application. Granted, I
didn't put much thought into an update mechanism, but it seemed to be
messy at the time.

If I have my own distribution, I can simply give the users an "update"
program that will query my webserver which will download the latest
version for them automagically. Because my distribution has all of
the modules already available, I don't have to worry about sending
them any missing modules. Simply download the latest version of my
app and it works with my custom rolled distribution.

But, this only works for Windows...

jw

On Wed, 02 Mar 2005 09:12:26 -0500, Phillip Mills
<phillip.mills1 @acmdelete.org> wrote:[color=blue]
> I've learned enough of the Python language to be mildly dangerous and
> have used it in a few personal projects. All my development of
> commercial (or production) products over the past dozen years have been
> done with C++ or Java.
>
> For a program I'm planning -- to begin during the summer -- having an
> interpreter as part of the application would be very desirable to allow
> sophisticated users to provide their own extensions. Java would be
> do-able, but....
>
> My problems are:
> - I'd like the process of installing the application to be one step;
> no "first download a Python interpreter then a GUI library" kind of
> thing.
> - I also need the core part of the application to be reasonably
> protected. I'm not looking to defeat hackers, but something equivalent
> to the way Java's class files stored in jars stay where they're supposed
> to be and aren't immediately readable.
>
> I've looked at various web sites for this topic, but most I've found are
> just arguments for using the Python language. OK, I'll pretend I'm
> convinced...now any comments or references on the mechanics of creating
> a self-contained distribution?
>
> --
> Phillip Mills
> Multi-platform software development
> (416) 224-0714
>
> ----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
> http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
> ----= East and West-Coast Server Farms - Total Privacy via Encryption =----
> --
> http://mail.python.org/mailman/listinfo/python-list
>[/color]

[André Søreng]

Re: Distributing applications

Phillip Mills wrote:[color=blue]
> I've learned enough of the Python language to be mildly dangerous and
> have used it in a few personal projects. All my development of
> commercial (or production) products over the past dozen years have been
> done with C++ or Java.
>
> For a program I'm planning -- to begin during the summer -- having an
> interpreter as part of the application would be very desirable to allow
> sophisticated users to provide their own extensions. Java would be
> do-able, but....
>
> My problems are:
> - I'd like the process of installing the application to be one step;
> no "first download a Python interpreter then a GUI library" kind of
> thing.[/color]

For creating a self-contained installation/distribution under Windows,
use py2exe:

http://starship.python.net/crew/theller/py2exe/

If you want to create a nice Windows installer for your module(s):

http://www.python.org/doc/current/dist/postinstallation-script.html

[color=blue]
> - I also need the core part of the application to be reasonably
> protected. I'm not looking to defeat hackers, but something equivalent
> to the way Java's class files stored in jars stay where they're supposed
> to be and aren't immediately readable.
>[/color]
Hmm, not sure about that one. You mean that those users who write
extensions should not be able to modify the core code you wrote? Are you
talking about a restricted execution environment for untrusted code?
I'd rather make it so to only accept code which is signed by a trusted
party or something like that.
[color=blue]
> I've looked at various web sites for this topic, but most I've found are
> just arguments for using the Python language. OK, I'll pretend I'm
> convinced...now any comments or references on the mechanics of creating
> a self-contained distribution?
>[/color]

[Serge Orlov]

Re: Distributing applications

Jaime Wyant wrote:[color=blue]
> I chose not to use py2exe because it made updating my app a bit
> messier. For instance, suppose I don't use the smtplib module in
> version 1 of my software. Py2exe realizes that and doesn't 'package'
> smtplib with my executable. Now, if I release version 1.1 which does
> use smtplib, then I'd have to figure out how to get the updates out
> without having the user redownload the entire application. Granted,
> I didn't put much thought into an update mechanism, but it seemed to
> be messy at the time.[/color]

I don't follow you. How is that different compared to adding a module
to your application? Let's say version 1.1 of your software has
module1.py updated, module2.py added and it needs smtplib. You just
bundle three compiled files and unpack them let's say to
\program files\my software\module 1.pyc
\program files\my software\module 2.pyc
\program files\my software\python \smtplib.pyc

I don't see any mess.
[color=blue]
>
>If I have my own distribution, I can simply give the users an "update"
> program that will query my webserver which will download the latest
> version for them automagically. Because my distribution has all of
> the modules already available, I don't have to worry about sending
> them any missing modules. Simply download the latest version of my
> app and it works with my custom rolled distribution.
>
> But, this only works for Windows...[/color]

Why? As I understand "update" program was written by you, so what
prevents it from working on other platforms besides testing?

Serge.

[Phillip Mills]

Re: Distributing applications

First, thanks to all the people who have answered so far for the
suggestions.

In article <U3mVd.102616$V f.3959417@news0 00.worldonline. dk>,
André Søreng <wsoereng@tisca li.no> wrote:
[color=blue]
> Phillip Mills wrote:[/color]
[color=blue][color=green]
> > My problems are:[/color][/color]

[...]
[color=blue]
> http://starship.python.net/crew/theller/py2exe/[/color]

Apparently I had more problems than I mentioned. :-) One of them being
that a Windows-only solution is only a partial solution.
[color=blue][color=green]
> > - I also need the core part of the application to be reasonably
> > protected. I'm not looking to defeat hackers, but something equivalent
> > to the way Java's class files stored in jars stay where they're supposed
> > to be and aren't immediately readable.
> >[/color]
> Hmm, not sure about that one. You mean that those users who write
> extensions should not be able to modify the core code you wrote?[/color]

Partly that and partly a file management thing. For most end users a
..jar is one thing to deal with; it's the most recent one or it's not;
it's present in the right location or it's not....

--
Phillip Mills
Multi-platform software development
(416) 224-0714

----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

[Jaime Wyant]

Re: Distributing applications

On 2 Mar 2005 10:43:38 -0800, Serge Orlov <Serge.Orlov@gm ail.com> wrote:[color=blue]
> Jaime Wyant wrote:[color=green]
> > I chose not to use py2exe because it made updating my app a bit
> > messier. For instance, suppose I don't use the smtplib module in
> > version 1 of my software. Py2exe realizes that and doesn't 'package'
> > smtplib with my executable. Now, if I release version 1.1 which does
> > use smtplib, then I'd have to figure out how to get the updates out
> > without having the user redownload the entire application. Granted,
> > I didn't put much thought into an update mechanism, but it seemed to
> > be messy at the time.[/color]
>
> I don't follow you. How is that different compared to adding a module
> to your application? Let's say version 1.1 of your software has
> module1.py updated, module2.py added and it needs smtplib. You just
> bundle three compiled files and unpack them let's say to
> \program files\my software\module 1.pyc
> \program files\my software\module 2.pyc
> \program files\my software\python \smtplib.pyc
>
> I don't see any mess.
>[/color]

The way I *think* about it is this --> To update my app, I only have
to download a new version of it... Being written in python, the app
itself is pretty small....

Suppose the following are true:
1) when the user installs my custom built python distribution they
also get version 1.0 of the app.
2) Suppose it is installed in c:\myapp (for simplicity).
3) I've just released version 1.1

Because *all* of the modules are in my custom built distribution, I
don't have to actively scan for new modules between releases. So....
I can write a simple "updater" script that will:
1) download a new myapp.zip
2) remove the c:\myapp directory and its subdirs
3) unzip it to c:\myapp, overwriting what is there

After those three steps are complete, then my app is updated. Super
simple. All I have to do is "zip" up my application and post it to a
website somewhere...

IIRC, py2exe bundles things up in a .zip file. So in order to update
a py2exe app i'd have to ->
1) check for new module dependencies
2) get the newly imported modules AND my newly updated modules back to
the client
3) update the zip file with the new modules.

This becomes especially hairy when someone is updating from 1.0 to say
1.5. Then I have to keep track of all the deltas between 1.0/1.5. My
way is much simpler because I don't have to keep up with *anything*.
As long as I test my code against my custom built distribution, it
ought to JUST WORK.

I don't trust myself to keep up with anything ;).

However, if you have an idea on updating py2exe bundled apps, I'm all ears...
[color=blue][color=green]
> >
> >If I have my own distribution, I can simply give the users an "update"
> > program that will query my webserver which will download the latest
> > version for them automagically. Because my distribution has all of
> > the modules already available, I don't have to worry about sending
> > them any missing modules. Simply download the latest version of my
> > app and it works with my custom rolled distribution.
> >
> > But, this only works for Windows...[/color]
>
> Why? As I understand "update" program was written by you, so what
> prevents it from working on other platforms besides testing?[/color]

Yes, but the *update* program runs in the context of my distribution
which is strictly bundled for windows. Now, someone *could* build a
distributable version for Linux somehow (I guess) and then the update
techniques I mentioned would probably work there.

jw

[Mitch Amiano]

Re: Distributing applications

I'm fairly new to python myself (with about 16 years of various
languages); I found py2exe fairly straightforward to use.
The instructions are ok, but if you care to see it I took some notes and
threw them into an article on my company site.

<http://home.agilemarku p.com/index.php?optio n=content&task= view&id=64&Item id=27>

Regards

- Mitch

Phillip Mills wrote:[color=blue]
> I've learned enough of the Python language to be mildly dangerous and
> have used it in a few personal projects. All my development of
> commercial (or production) products over the past dozen years have been
> done with C++ or Java.
>
> For a program I'm planning -- to begin during the summer -- having an
> interpreter as part of the application would be very desirable to allow
> sophisticated users to provide their own extensions. Java would be
> do-able, but....
>
> My problems are:
> - I'd like the process of installing the application to be one step;
> no "first download a Python interpreter then a GUI library" kind of
> thing.
> - I also need the core part of the application to be reasonably
> protected. I'm not looking to defeat hackers, but something equivalent
> to the way Java's class files stored in jars stay where they're supposed
> to be and aren't immediately readable.
>
> I've looked at various web sites for this topic, but most I've found are
> just arguments for using the Python language. OK, I'll pretend I'm
> convinced...now any comments or references on the mechanics of creating
> a self-contained distribution?
>[/color]

[Serge Orlov]

Re: Distributing applications

Phillip Mills wrote:[color=blue]
> First, thanks to all the people who have answered so far for the
> suggestions.
>
> In article <U3mVd.102616$V f.3959417@news0 00.worldonline. dk>,
> André Søreng <wsoereng@tisca li.no> wrote:
>[color=green]
> > Phillip Mills wrote:[/color]
>[color=green][color=darkred]
> > > My problems are:[/color][/color]
>
> [...]
>[color=green]
> > http://starship.python.net/crew/theller/py2exe/[/color]
>
> Apparently I had more problems than I mentioned. :-) One of them
> being that a Windows-only solution is only a partial solution.[/color]

There is also py2app for Mac. Still partial? :) Then follow Jaime's
way: build and bundle python with your application.
[color=blue]
>[color=green][color=darkred]
> > > - I also need the core part of the application to be reasonably[/color][/color][/color]
[color=blue][color=green][color=darkred]
> > > protected. I'm not looking to defeat hackers, but something
> > > equivalent to the way Java's class files stored in jars stay
> > > where they're supposed to be and aren't immediately readable.
> > >[/color]
> > Hmm, not sure about that one. You mean that those users who write
> > extensions should not be able to modify the core code you wrote?[/color]
>
> Partly that and partly a file management thing. For most end users a[/color]
[color=blue]
> .jar is one thing to deal with; it's the most recent one or it's not;[/color]
[color=blue]
> it's present in the right location or it's not....[/color]

Python byte code is like java byte code and python supports importing
from zip files like java. Since python comes with a liberal license
you can change the importing code to decrypt your modules with a
"secret" key. That will be much safer than java. Of course that won't
stop real hackers.

Serge.

[Serge Orlov]

Re: Distributing applications

Jaime Wyant wrote:
[color=blue]
> This becomes especially hairy when someone is updating from 1.0 to
> say 1.5. Then I have to keep track of all the deltas between
> 1.0/1.5. My way is much simpler because I don't have to keep up with
> *anything*. As long as I test my code against my custom built
> distribution, it ought to JUST WORK.
>
> I don't trust myself to keep up with anything ;).[/color]

Now I follow you :) I agree that updating py2exe apps requires package
management utilities. I don't think they will be messy, it's just
more code to maintain compared to your way. You only need to track
one delta (1.0 -> 1.1 ... -> latest) and publish two files latest.exe
and update.zip

Serge.

[Thomas Heller]

Re: Distributing applications

"Serge Orlov" <Serge.Orlov@gm ail.com> writes:
[color=blue]
> Jaime Wyant wrote:
>[color=green]
>> This becomes especially hairy when someone is updating from 1.0 to
>> say 1.5. Then I have to keep track of all the deltas between
>> 1.0/1.5. My way is much simpler because I don't have to keep up with
>> *anything*. As long as I test my code against my custom built
>> distribution, it ought to JUST WORK.
>>
>> I don't trust myself to keep up with anything ;).[/color]
>
> Now I follow you :) I agree that updating py2exe apps requires package
> management utilities. I don't think they will be messy, it's just
> more code to maintain compared to your way. You only need to track
> one delta (1.0 -> 1.1 ... -> latest) and publish two files latest.exe
> and update.zip[/color]

Sometimes I think that CVS or SVN may be the simplest solution to update
applications. Install a client on the target computer, it may be
invisible to the user, copy the CVS or SVN directories, and provide a
script the does (also invisible) 'cvs up -r version_a_b'.

Thomas

[Jaime Wyant]

Re: Distributing applications

Sneaky! I like it. Now if there was only a subversion python module...

jw

On Wed, 02 Mar 2005 22:08:45 +0100, Thomas Heller <theller@python .net> wrote:[color=blue]
> "Serge Orlov" <Serge.Orlov@gm ail.com> writes:
>[color=green]
> > Jaime Wyant wrote:
> >[color=darkred]
> >> This becomes especially hairy when someone is updating from 1.0 to
> >> say 1.5. Then I have to keep track of all the deltas between
> >> 1.0/1.5. My way is much simpler because I don't have to keep up with
> >> *anything*. As long as I test my code against my custom built
> >> distribution, it ought to JUST WORK.
> >>
> >> I don't trust myself to keep up with anything ;).[/color]
> >
> > Now I follow you :) I agree that updating py2exe apps requires package
> > management utilities. I don't think they will be messy, it's just
> > more code to maintain compared to your way. You only need to track
> > one delta (1.0 -> 1.1 ... -> latest) and publish two files latest.exe
> > and update.zip[/color]
>
> Sometimes I think that CVS or SVN may be the simplest solution to update
> applications. Install a client on the target computer, it may be
> invisible to the user, copy the CVS or SVN directories, and provide a
> script the does (also invisible) 'cvs up -r version_a_b'.
>
> Thomas
> --
> http://mail.python.org/mailman/listinfo/python-list
>[/color]

[Tim Jarman]

Re: Distributing applications

Jaime Wyant wrote:
[color=blue]
> Sneaky! I like it. Now if there was only a subversion python module...
>
> jw
>[/color]

GIYF: http://pysvn.tigris.org/


--
Website: www DOT jarmania FULLSTOP com

[Robert Kern]

Re: Distributing applications

Jaime Wyant wrote:[color=blue]
> Sneaky! I like it. Now if there was only a subversion python module...[/color]

Google, and you shall find.

--
Robert Kern
rkern@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

[Stephen Thorne]

Re: Distributing applications

On Wed, 2 Mar 2005 13:07:25 -0600, Jaime Wyant <programmer.py@ gmail.com> wrote:
<snip>[color=blue]
> However, if you have an idea on updating py2exe bundled apps, I'm all ears...[/color]

I'm working on a little project that requires remote updating. What I
basically came up with is two nested applications.

program/program.exe runs, and checks program/realprogram/ for updates
from a server. if md5s mismatch, it pulls new versions. after pulling
new version, it runs the program/realprogram.exe

realprogram.exe can, at some point in the future, be written to update
program/program.exe's code.

This isn't exactly the way I ended up doing it, due to the application
(all on a LAN, not the internet) I'm actually pulling the 'real'
program every time and leaving it in a directory under
tempfile.gettem pdir()

Its a bit messy, but it actually works on both windows and linux.
(cx_Freeze under linux)

Stephen.