Python needs a CPyAN

[Michael Ströder]

Re: Python needs a CPyAN

Gerhard Haering wrote:[color=blue]
> On Tue, Nov 02, 2004 at 11:41:18PM +0100, bruno modulix wrote:
>[color=green]
>><aol>
>>We had some program (can't remember if it was spamassassin or an
>>antivir, I'm not the sysadmin !-) gone mad and all our customers mails
>>blocked for a full day because of such a wonderfull update of a perl
>>module...
>></aol>[/color]
>
> Well, could have happened with a Debian[*] or *BSD, etc. update as well,
> right?[/color]

Yes.

Ciao, Michael.

[Mike Meyer]

Re: Python needs a CPyAN

Isaac To <iketo2@netscap e.net> writes:
[color=blue][color=green][color=darkred]
>>>>>> "Gerhard" == Gerhard Haering <gh@ghaering.de > writes:[/color][/color]
>
> Gerhard> Well, could have happened with a Debian[*] or *BSD,
> Gerhard> etc. update as well, right?
>
> Yes, userss of systems with proper package management shouldn't want
> to install anything from CPAN. On the other hand, there are many
> people who are stuck with a braindead distribution or even OS that
> doesn't have proper package management, or whose packages are severely
> limited.[/color]

Actually, what people on systems with proper package management want
is packages that pull stuff from CPAN and install them as
packages. Doing this right is hard, because CPAN makes assumptions
about where things are liable to be installed. For Python, what we
want are packages that use distutils to install them as
packages. Doing this right is trivial, because distutils gets all the
relevant information from the environment.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Mike Meyer]

Re: Python needs a CPyAN

"John Roth" <newsgroups@jhr othjr.com> writes:
[color=blue]
> "John Hazen" <python-list@hazen.net> wrote in message
> news:mailman.58 38.1099425921.5 135.python-list@python.org ...
> 1. Where do I store the module? I don't have a web site, and
> it isn't on sourceforge.[/color]

Ok, that's a definite need. Possibly python.org could host a mechanism
to allow people to post modules. I'd do it myself, but I'm not sure
how long I'm going to have my current net connection for.
[color=blue]
> 2. For what I've done, the standard installer script required
> for PPI is stoopid. All that's really needed to install is unpack
> to the right directory. That's it. However, to use distutils I
> had to write a packager program to create all the entries
> I needed. This took me quite a few hours and left a very
> sour taste since a lot of things that I thought should have
> been automatic weren't.[/color]

I found the standard installer script trivial to set up. If it took
hours, then you're doing something wrong. It even made it possible for
me to distribute C modules to Windows, whereas those trying to build
the module on Windows prior to me putting it into distutils never
succeeded.

<mike
--
Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Nick Craig-Wood]

Re: Python needs a CPyAN

Mike Meyer <mwm@mired.or g> wrote:[color=blue]
> Actually, what people on systems with proper package management want
> is packages that pull stuff from CPAN and install them as
> packages.[/color]

Absolutely!

For instance with Debian and Perl, dh-make-perl will turn a perl
module (tar.gz or straight from CPAN) into a .deb which you then
managage as normal. When using Debian I never use CPAN and always
dh-make-perl because I'd much rather have the package managed via
dpkg/apt-get.

That said I haven't found many perl modules that I wanted that weren't
already packaged for Debian. The same with python modules too.
[color=blue]
> Doing this right is hard, because CPAN makes assumptions
> about where things are liable to be installed.[/color]

No idea how dh-make-perl works. It does though!
[color=blue]
> For Python, what we want are packages that use distutils to install
> them as packages. Doing this right is trivial, because distutils
> gets all the relevant information from the environment.[/color]

distutils already knows how to make .rpm files.

Eg choosing a package at the top of PyPI :-

# Download
wget 'http://ftp.livinglogic .de/xist/ll-xist-2.6.1.tar.gz'
# Unpack
tar zxvf ll-xist-2.6.1.tar.gz
cd ll-xist-2.6.1
# Make rpm
python setup.py bdist_rpm
# Make deb
cd dist/
fakeroot alien ll-xist-2.6.1-1.i386.rpm ll-xist-2.6.1-1.i386.deb
# Install with package manager
sudo dpkg -i ll-xist_2.6.1-2_i386.deb

This hasn't worked with every package I've tried it with though!

....

To make PyPI into a CPAN replacement it needs to
1) make all the submitters use distutils, and automatically test the packaging
2) host the source tar balls
3) have a system of mirrors and provide this list for download
4) provide dependency information (maybe this should be in the distutils
information in the package)
5) Provide the list of packages, descriptions, and dependencies for
download
6) Some sort of review system (maintainers, peer review, rating etc)
for each package for quality assurance

Given all that the an automatically download package x + dependencies
program would be easy.

A cross platform software repository like this would be a goal well
worth aiming for IMHO! PyPI is an excellent start though.

--
Nick Craig-Wood <nick@craig-wood.com> -- http://www.craig-wood.com/nick

[Eric S. Johansson]

Re: Python needs a CPyAN

Nick Craig-Wood wrote:[color=blue]
> Mike Meyer <mwm@mired.or g> wrote:
>[color=green]
>> Actually, what people on systems with proper package management want
>> is packages that pull stuff from CPAN and install them as
>> packages.[/color]
>
>
> Absolutely!
>
> For instance with Debian and Perl, dh-make-perl will turn a perl
> module (tar.gz or straight from CPAN) into a .deb which you then
> managage as normal. When using Debian I never use CPAN and always
> dh-make-perl because I'd much rather have the package managed via
> dpkg/apt-get.
>
> That said I haven't found many perl modules that I wanted that weren't
> already packaged for Debian. The same with python modules too.
>
>[color=green]
>> Doing this right is hard, because CPAN makes assumptions
>> about where things are liable to be installed.[/color]
>
>
> No idea how dh-make-perl works. It does though!
>
>[color=green]
>> For Python, what we want are packages that use distutils to install
>> them as packages. Doing this right is trivial, because distutils
>> gets all the relevant information from the environment.[/color]
>
>
> distutils already knows how to make .rpm files.
>
> Eg choosing a package at the top of PyPI :-
>
> # Download
> wget 'http://ftp.livinglogic .de/xist/ll-xist-2.6.1.tar.gz'
> # Unpack
> tar zxvf ll-xist-2.6.1.tar.gz
> cd ll-xist-2.6.1
> # Make rpm
> python setup.py bdist_rpm
> # Make deb
> cd dist/
> fakeroot alien ll-xist-2.6.1-1.i386.rpm ll-xist-2.6.1-1.i386.deb
> # Install with package manager
> sudo dpkg -i ll-xist_2.6.1-2_i386.deb
>
> This hasn't worked with every package I've tried it with though!
>
> ...
>
> To make PyPI into a CPAN replacement it needs to
> 1) make all the submitters use distutils, and automatically test the packaging
> 2) host the source tar balls
> 3) have a system of mirrors and provide this list for download
> 4) provide dependency information (maybe this should be in the distutils
> information in the package)
> 5) Provide the list of packages, descriptions, and dependencies for
> download
> 6) Some sort of review system (maintainers, peer review, rating etc)
> for each package for quality assurance
>
> Given all that the an automatically download package x + dependencies
> program would be easy.[/color]

I have provided a some of this functionality in raging dormouse.

http://harvee.org/stamper/raging_dormouse/

can download from the sourceforge "click a link" maze or a regular web
site.
Verifies checksum of package.
Can handle list of download sites to try (could be expanded to mirror
list)

the build script in raging_dormouse is fairly application-specific. It
wouldn't be hard to redraft it as a configuration file driven process
while rewriting it in Python vs. its current Bash incarnation.

you're welcome to it if you find useful.

---eric

[John Roth]

Re: Python needs a CPyAN


"Mike Meyer" <mwm@mired.or g> wrote in message
news:x7d5yvv1vh .fsf@guru.mired .org...[color=blue]
> "John Roth" <newsgroups@jhr othjr.com> writes:
>[color=green]
>> "John Hazen" <python-list@hazen.net> wrote in message
>> news:mailman.58 38.1099425921.5 135.python-list@python.org ...
>> 1. Where do I store the module? I don't have a web site, and
>> it isn't on sourceforge.[/color]
>
> Ok, that's a definite need. Possibly python.org could host a mechanism
> to allow people to post modules. I'd do it myself, but I'm not sure
> how long I'm going to have my current net connection for.
>[color=green]
>> 2. For what I've done, the standard installer script required
>> for PPI is stoopid. All that's really needed to install is unpack
>> to the right directory. That's it. However, to use distutils I
>> had to write a packager program to create all the entries
>> I needed. This took me quite a few hours and left a very
>> sour taste since a lot of things that I thought should have
>> been automatic weren't.[/color]
>
> I found the standard installer script trivial to set up. If it took
> hours, then you're doing something wrong. It even made it possible for
> me to distribute C modules to Windows, whereas those trying to build
> the module on Windows prior to me putting it into distutils never
> succeeded.[/color]

Oh, the standard installer would have been fine - if all I
was distributing was Python modules. However, I have
a fairly large number of examples and quite a large amount
of HTML documentation. For some reason, the standard
installer decided that it would copy them to the install file
just fine, and then not install them. Nothing I could do would
make them install other than listing each and every file in
the install script, individually and by hand.

That's absolutely stoopid when they go into the exact
same directory they came out of. The time was spent

1) asking for help on the newsgroup. Nobody had any
helpful suggestions.

2) Hammering on the thing trying to make it work.
No joy.

3) Writing a small script to build the distutils script.
That worked, although there still seems to be an
intermittant problem on the install.

John Roth[color=blue]
>
> <mike
> --
> Mike Meyer <mwm@mired.or g> http://www.mired.org/home/mwm/
> Independent WWW/Perforce/FreeBSD/Unix consultant, email for more
> information.[/color]

[Stephen Ferg]

Re: Python needs a CPyAN

> Lots of people *want* a PyPAN; the number of people willing to work on it is about zero.

Admitted. Or perhaps those with the skills needed for the work aren't
able to undertake such a big job without financial support. That's why
I proposed that PSF put out a special RFP for grant money to seed the
work. My whole post was an argument for that proposal. Here's the
argument in a nutshell.

*************** *************** *************** *

PSF's mission is (among other things) to support Python's long-term
sucess.

Popularity is part of what it means to be successful. Popularity is
important to the Python community because it means jobs for Python
developers, and usage of Python by organizations that need it.

Python will never catch up to Perl in popularity without CPAN-like
support.

Therefore, Support for CPAN-like capabilities for Python is critical
to Python's long-term success.

Therefore, in keeping with its mission, PSF should offer grants
specifically targeted at producing a CPyAN.

*************** *************** *************** **

This issue should be of critical interest to all Python developers,
because it means the difference between the existence and the
non-existence of opportunities for developers to use Python on the
job.

[Ville Vainio]

Re: Python needs a CPyAN

>>>>> "Steve" == Stephen Ferg <steve@ferg.org > writes:

Steve> This issue should be of critical interest to all Python
Steve> developers, because it means the difference between the
Steve> existence and the non-existence of opportunities for
Steve> developers to use Python on the job.

If it was that important, I suppose the catalog-sig would have, say,
one non-spam message in the recent months. Perhaps people have assumed
that PyPI and distros' own package systems have solved the problem
already.

It should be noted that many significant Python packages have
non-python dependencies (wxPython <- wxWidgets). That kind of
dependency needs to be handled too, and it's the kind that is most
problematic. So in the end it basically boils down to not having
apt/yum for Windows.

--
Ville Vainio http://tinyurl.com/2prnb

[Stephen Ferg]

Re: Python needs a CPyAN

> Hell, people rarely even mention Perl these days

-----------------------------------------
location number of hits for
Python Perl
-----------------------------------------

Monster.com 18 552

Washington Post
classifieds 25 353

InfoWorld 81 954
job openings

TotalJobs (British) 32 200

[Peter Hickman]

Re: Python needs a CPyAN

Ville Vainio wrote:[color=blue]
> Hell, people rarely even mention Perl these days anyway.[/color]

People hardly speak Mandarin Chinese these days, I can go months without hearing
any being spoken. Same goes for Urdu and Spanish.

Today we learn the difference between an 'opinion' and an 'informed opinion'.

[A.M. Kuchling]

Re: Python needs a CPyAN

On 3 Nov 2004 06:04:44 -0800,
Stephen Ferg <steve@ferg.org > wrote:[color=blue]
> Admitted. Or perhaps those with the skills needed for the work aren't
> able to undertake such a big job without financial support. That's why[/color]

I'm doubtful that waving money in the air will magically produce volunteers,
but who knows? I suggest you present your proposal to the PSF; I suppose
the Board would be the group that would consider it.

--amk

[Ville Vainio]

Re: Python needs a CPyAN

>>>>> "Steve" == Stephen Ferg <steve@ferg.org > writes:
[color=blue][color=green]
>> Hell, people rarely even mention Perl these days[/color][/color]

Steve> -----------------------------------------
Steve> location number of hits for
Steve> Python Perl
Steve> -----------------------------------------

Steve> Monster.com 18 552

.... etc.

This does not take away the fact that a few years ago, it was felt
that Python is in a bloody fight over popularity with Perl. It was
felt that Perl still had this or that over Python; this doesn't seem
to be the case anymore. Python is no longer the obscure up-and-coming
scripting language. It is used in several high-profile places, while
Perl usage seems to be mostly going down in popularity. In the Open
Source world, Python aleady seems to have a higher profile than
Perl. The popularity of Python is already big enough to not be an
impediment for the company to switch over - it may be that there is
still a need to switch over in the first place, but that's just a
refreshing challenge ;-). Hence, I wouldn't be too worried about
competing with Perl.

Decision to use a library requires some thinking already, it's not a
big additional hurdle to download it from SF/whatever and add it to
the company version control system. I wouldn't mind a CPANish system,
but the lack of it isn't the bottleneck in Python adoption.

--
Ville Vainio http://tinyurl.com/2prnb

[Thomas Heller]

Re: Python needs a CPyAN

"Graham Fawcett" <graham.fawcett @gmail.com> writes:
[color=blue]
> A.M. Kuchling wrote:[color=green]
>> And it should be noted I can already install Python packages with
>> dependencies by running a command such as "apt-get install
>> python2.3-gnome2", for example.[/color]
>
> Great point. Collectively, apt-get, yum et. al. and the underlying
> package models will get far more development time, more polishing and
> more real-world use than a PyPAN ever would.
>
> Perhaps what Python needs is for Win32 to grow a working apt-get...;-)[/color]

The distutils' created bdist_wininst installers in principle have all
that is needed to build something like that. Except command line flags
for silent installation, maybe. But since a silent installer has no
need to display a user interface, it would be simple to write a
bdist_wininst_s ilent in pure Python.

Thomas

[Alex Martelli]

Re: Python needs a CPyAN

Ville Vainio <ville@spammers .com> wrote:
[color=blue]
> Python is no longer the obscure up-and-coming
> scripting language.[/color]

Yep, that's what Ruby is for.


Alex

[Stephen Ferg]

Re: Python needs a CPyAN

> I suggest you present your proposal to the PSF; I suppose[color=blue]
> the Board would be the group that would consider it.[/color]

I agree.

I attempted to post it to the grants-discuss list, but was turned
down.

"Interestin g discussion, but I don't think the grants-discuss list is
the right place for it."