Python does not play well with others

[Paul Boddie]

Re: Python does not play well with others

Paul Rubin wrote:

Ben Finney <bignose+hate s-spam@benfinney. id.auwrites:

Since Python is being touted as good for web apps as a competitor to
PHP

Python is being touted as a good language for *many* purposes, not
just web applications. Python is also a "competitor " to Java, to Ruby,
to Perl, to many other languages. They all have strengths and
weaknesses.

>
Yes but in the cases where Python's weakness compared with one of
those other languages is lack of library functionality, if Python can
remedy the weakness by incorporating similar functionality into its
library it should do so.

Python should only incorporate functionality in order to offer a
coherent experience (where the omission of functionality would
otherwise lead to a flawed experience). For example, having support
for SSL in the socket module offers a coherent experience because it
means that urllib and related modules can offer to support SSL-related
URLs out of the box. That said, there's probably a case for better
library design and having things like SSL-related "protocol handlers"
be available as plugins, detected and integrated in some kind of
framework, although this would bring in configuration issues of its
own.

[...]

I'm paying the hosting company for access to a computer that's
connected to electricity and to the internet and which has a
straightforward OS, language package, web server, and db installed.

In which case, there should be no problem with *you* installing
whatever software you need to use the system for what you want.

>
No. That would be colo or something similar , where I'm basically
paying for bare metal plus electricity and network, and I'm completely
in charge of the software.

Or a virtual private server.

Web hosting means the ISP is in charge of all the software except for my application (i.e. they handle the OS,
language package, web server, and db, as described above). So they
run (typically) Linux, MySQL, Apache, and PHP; and I get to upload my
own PHP apps and use the PHP library. That's a lot less work for me
since I don't have to stay on top of kernel patches or firewall
configuration, and it's cheaper because they host a bazillion sites
(virtual hosts) in a a single server instance.

That's known as "shared hosting" these days, I believe. The advantage
of shared hosting is arguably convenience for everyone concerned: the
provider has a static configuration; the users have the stability of a
managed system. However, this only works for people whose needs are
met with that configuration, obviously. The big thing for people
wanting decent Python support in shared hosting environments (aside
from complaints about process isolation) is whether it's easy or
obvious enough for providers to make configurations that Python people
would actually use.

Probably the biggest inhibitor, as far as I can see, has been the
server technology chosen. Many hosting providers have historically
offered no better than CGI for Python, whilst PHP runs within Apache
itself, and it has previously been stated that mod_python has been
undesirable with regard to isolating processes from each other.
Consequently, a number of Python people seem to have held out for
other "high performance" solutions, which various companies now offer.

They shouldn't have to deal with dozens of interdependent modules
downloaded from different places just to support one language.

Either they are providing far more than the minimal set you describe
above, or this is entirely outside their domain. Make up your mind.

>
No it's you who's got it wrong, I just described above what they're
doing. Do you actually use any services like this?

If a hosting provider claims Python and MySQL support, then I'd hope
that they have worked out that the MySQLdb package glues the two
together. Perhaps better information is necessary for those hosting
companies who haven't worked such things out: you'd have a metapackage
for stuff like this in certain distributions.

You can't claim both that the hosting company should have to maintain
a comprehensive set of functionality, *and* that they should not have to.

>
They should get a distro that includes a lot of stuff, type "make",
and all the stuff becomes available to their users.

So, for the less forward-thinking providers a metapackage would be the
solution, then?

Paul

[Paul Rubin]

Re: Python does not play well with others

skip@pobox.com writes:

PaulYeah well, the Wxpython, PyQt, PyGTK etc. people may feel slighted
Paulthat Tkinter got included and their stuff didn't, and the Eric,
PaulEclipse, Komodo etc. people may feel slighted that IDLE got
Paulincluded, but that doesn't stop Tkinter and IDLE from being useful
Pauland worth shipping in Python.
>
They were both pretty much the only games in town when they were included
with Python. If they were developed in today's environment I doubt they
would be included.

That would diminish Python's popularity since being able to write GUI
apps without having to download additional crap is a Python selling
point, and IDLE is a considerably more pleasant Python editor than
Notepad is.

PaulBasic competitive analysis. People ask here all the time "I'm
Paultrying to write application XYZ, should I use language L or should
PaulI use Python" (L is usually Java or PHP but can be other things).
>
So every time some lame ass PHP refugee gripes about something they think
Python is missing which PHP includes we should scan the horizon for all the
candidates and toss them in the next release?

No, just the functions that are requested frequently, like database
adapters.

[Paul Rubin]

Re: Python does not play well with others

Dennis Lee Bieber <wlfraed@ix.net com.comwrites:

languages do. PHP hosting providers don't have to install a separate
PHP to MySQL interface gizmo as far as I know.

>
Really? Then why does the MySQL AB download site at
http://dev.mysql.com/downloads/connector/php/ list both a
"Connector/PHP" AND a "native MySQL driver" for PHP? Along with ODBC,
.NET, and two variations of Java connectors.

Beats me. It does look like PHP5 has stopped bundling the MySQL
client library for licensing reasons:

http://www.php.net/manual/en/faq.databases.php#faq.databases.mysql.php5

It does say "there will always be MySQL support in PHP of one kind or
another" but I'm not sure what that means in this context.

On the other hand if MySQL itself supplies the client library, that's
almost as good. Remember that this isn't purely about minimizing the
number of downloads. It's also about minimizing the number of places
to download from, i.e. the number of different development entities
one has to deal with.

So if you're using Python with MySQL, and you can get everything you
need from python.org and mysql.com, then even if it takes multiple
downloads it's better than having to get additional stuff from random
third party sites, especially when Python's own docs haven't said
where to get the stuff.

Therefore, if someone can persuade mysql.com to offer downloadable
MySQL client drivers for Python at the same place where they supply
the PHP drivers, that's better than the current situation where the
drivers come from some totally unrelated party.

[Paul Rubin]

Re: Python does not play well with others

"Paul Boddie" <paul@boddie.or g.ukwrites:

Python should only incorporate functionality in order to offer a
coherent experience (where the omission of functionality would
otherwise lead to a flawed experience). For example, having support
for SSL in the socket module offers a coherent experience because it
means that urllib and related modules can offer to support SSL-related
URLs out of the box.

But they can't, because the built-in socket module SSL interface
doesn't check certificates, causing total security failure if someone
spoofs the remote site. The built-in SSL functionality is broken and
users have to resort to external packages.

Then you have to ask why the stdlib includes anything like urllib in
the first place, under this "coherent experience" concept (I interpret
that as some kind of minimalist philosophy). Can't users have a
coherent experience if the stdlib doesn't include urllib? My own
answer is the one that I thought that the developers had settled on
years ago, namely "batteries included", i.e. ship a rich set of
libraries that provide a wide variety of useful functions, i.e. the
doctrine of minimalism has been explicitly rejected. We then get the
question of whether to include any specific function and that's where
comparisons with other languages come in.

No. That would be colo or something similar ,

Or a virtual private server.

Sure, that would count as "something similar".

If a hosting provider claims Python and MySQL support, then I'd hope
that they have worked out that the MySQLdb package glues the two
together.

I'd expect so too. The issue is there aren't very many of those
companies and that appears partly because of the hassle involved. So
minimizing the hassle suggests itself as a road to wider acceptance.

Your point that shared hosting with Python isn't so easy because of
insufficient isolation between apps is valid. Maybe Python 3.0 can do
something about that and it seems like a valid thing to consider while
fleshing out the 3.0 design.

Perhaps better information is necessary for those hosting
companies who haven't worked such things out: you'd have a metapackage
for stuff like this in certain distributions.

That could help.

So, for the less forward-thinking providers a metapackage would be the
solution, then?

I'm not sure what you mean by metapackage but in general the goal is
to minimize the number of places that the hosting provider (or OS
distro maintainer, or whatever)

[Paul Rubin]

Re: Python does not play well with others

skip@pobox.com writes:

PaulWhy would I expect your employer to solve my problems anyway, even
Paulif they relate to some module that you actually use?
>
Your reasoning seems to be that Python should contain the functional union
of everything at least in Java and PHP if not Perl, Ruby and Tcl as well.

I wouldn't go quite that far. I think there are specific application
areas, such as web server apps, where the Python advocates here on
clpy pitch Python relentlessly against those other languages. Given
that context, Python's stdlib should try to match the libraries of
those other languages in those areas. There are other areas where
Python doesn't get pitched as hard and the other languages have
acknowledged advantages. So it's ok if Python's stdlib gets less
attention in those areas.

Free software or not, each person who contributes to the development
of the Python core has to put food on the table. As an example, I
suspect most core Java development is funded directly by Sun who
pays its employees to develop libraries, JIT compilers and the like.
While my employer pays me to program in Python I'm not paid to
develop core Python code. What little I do is done on my own time.

Again I don't understand what your employer or your activities have to
do with any of this. If you don't want to spend your time developing
Python then don't. You didn't write the interpreter or the tkinter
library or the itertools library and yet there they are. There's
ongoing development that will continue with or without you or me, and
for that matter with or without funding from Sun.

If you want to turn the Python distribution into a kitchen sink,
make the argument on python-dev and be prepared to shoulder your
share of the burden should your arguments sway the group as a whole.

We've had this conversation before and I continue to think your
reasoning above is invalid. I'm not a Python developer, I'm just a
user, and my volunteer coding priorities are elsewhere, as I've
explained before. Python's developers and advocates have a declared
goal of reaching as many users as they can, and as a user I don't mind
offering suggestions about how to do that, but my responsibilitie s
don't go any further.

[Diez B. Roggisch]

Re: Python does not play well with others

Paul Rubin schrieb:

"Diez B. Roggisch" <deets@nospam.w eb.dewrites:

>And they certainly require special treatment like putting them in the
>classpath, setting up the project directory and the like - no simple
>import will work out of the box, as it would witch python standard lib
>drivers.

>
Well, that's nowhere near as big a deal as having yet another set of
vendors or maintainer to deal with. Minimizing configuration is nice,
but getting your software from as few different sources as possible is
also a big win.

Ehm... .it _is_ another set of vendors and maintainers. It's just that
they are standard jdbc-compliant. As the python modules are DB API 2.0
compliant - hopefully.

A difference might be that most (not all) of the jdbc-drivers are
java-only, and not have any c-dependencies. Well - nice to have, but
simple inclusion of existing drivers doesn't remove that. Only a
complete rewrite in python would.

Diez

[Kirk Sluder]

Re: Python does not play well with others

In article <1170546032.316 062.133050@m58g 2000cwm.googleg roups.com>,
"Paul Boddie" <paul@boddie.or g.ukwrote:

Quite. I imagine that most GNU/Linux distributions (and various BSDs)
provide at least some version of MySQLdb as a package.

Bingo, I've rarely installed python from python.org, or other
libraries from sourceforge, etc., etc.. Usually I've installed
BSD-style ports, debian-style packages, or RedHat-style RPMs.

>
Paul

[Paul Boddie]

Re: Python does not play well with others

Kirk Sluder wrote:

In article <1170546032.316 062.133050@m58g 2000cwm.googleg roups.com>,
"Paul Boddie" <paul@boddie.or g.ukwrote:
>

Quite. I imagine that most GNU/Linux distributions (and various BSDs)
provide at least some version of MySQLdb as a package.

>
Bingo, I've rarely installed python from python.org, or other
libraries from sourceforge, etc., etc.. Usually I've installed
BSD-style ports, debian-style packages, or RedHat-style RPMs.

And while Python eggs may be useful for people managing additional
software as some unprivileged user, hosting providers (and virtual
private server administrators) will want packages that fit in with the
rest of the software being managed in the hosting environment.

Paul

[Paul Boddie]

Re: Python does not play well with others

Paul Rubin wrote:

"Paul Boddie" <paul@boddie.or g.ukwrites:

Python should only incorporate functionality in order to offer a
coherent experience (where the omission of functionality would
otherwise lead to a flawed experience). For example, having support
for SSL in the socket module offers a coherent experience because it
means that urllib and related modules can offer to support SSL-related
URLs out of the box.

>
But they can't, because the built-in socket module SSL interface
doesn't check certificates, causing total security failure if someone
spoofs the remote site. The built-in SSL functionality is broken and
users have to resort to external packages.

I was really advocating improvements to the built-in SSL support,
anyway, which was also what the complainant was suggesting before
people started asking him mistakenly why he thought that Python was
weakened by some third party packages (PyOpenSSL, M2Crypto). The
choice here involves either improving the built-in support or
unbundling SSL-based communications altogether. The former option
obviously demands a certain amount of engineering, and then one might
ask why there isn't a convenient framework for plugging in other
flavours of sockets, for example, although there arguably aren't any
as generally important as secure sockets. The latter option needs
everyone to think about how you'd plug such stuff back into Python in
a nice enough way, and then to get people to work on the right
projects to provide something which does the job.

Then you have to ask why the stdlib includes anything like urllib in
the first place, under this "coherent experience" concept (I interpret
that as some kind of minimalist philosophy). Can't users have a
coherent experience if the stdlib doesn't include urllib? My own
answer is the one that I thought that the developers had settled on
years ago, namely "batteries included", i.e. ship a rich set of
libraries that provide a wide variety of useful functions, i.e. the
doctrine of minimalism has been explicitly rejected.

Really, we have to ask whether including the batteries would save
people a lot of work, not just in whether the end-user has to find out
about an external package and then download something, but whether the
logistics around developing the code, integrating it with Python, and
maintaining it would be easier if people just included the stuff with
Python. Here, you need some kind of consensus that feature X needs
supporting and there's an approved way of supporting it which a group
of people would be happy to maintain. Would it benefit the Python
community (including core developers) more if Python shipped with SSL
support out of the box, and would the cost of doing so ultimately be
less than just pointing people at third party libraries and dealing
with their problems?

We then get the question of whether to include any specific function and that's where
comparisons with other languages come in.

Would it benefit the Python community more if Python shipped with
MySQL support out of the box? Is it likely that a user suddenly finds
him/herself needing to connect to a MySQL database? Is it more likely
that the user might suddenly find him/herself needing to download from
a secure site, particularly if some tool (eg. setuptools) suddenly
stumbles across an https URL. Some database systems have a choice of
drivers/libraries/modules (PostgreSQL has quite a few, for example):
choosing a module for standard library inclusion and integrating the
development can be too high a barrier for such questions of inclusion
to be resolved trivially.

[...]

So, for the less forward-thinking providers a metapackage would be the
solution, then?

>
I'm not sure what you mean by metapackage but in general the goal is
to minimize the number of places that the hosting provider (or OS
distro maintainer, or whatever)

On various distributions you get packages which don't actually contain
anything, but which indicate a suite of packages which are to be
installed. So, if you install the ubuntu-desktop package on Ubuntu
systems, you get the GNOME desktop environment and all the
dependencies (and a bunch of other stuff). Perhaps there should be a
python-mysql-hosting package for such providers.

Paul

[Jorge Godoy]

Re: Python does not play well with others

"Paul Boddie" <paul@boddie.or g.ukwrites:

And while Python eggs may be useful for people managing additional
software as some unprivileged user, hosting providers (and virtual
private server administrators) will want packages that fit in with the
rest of the software being managed in the hosting environment.

And why eggs wouldn't satisfy them? Eggs can be installed globally as well,
making the package available to every client of this hosting server (if they
mount their libs from a unique NFS server then it would automatically be
available for all of their servers).

--
Jorge Godoy <jgodoy@gmail.c om>

[Kirk Sluder]

Re: Python does not play well with others

In article <1170600807.504 088.80150@m58g2 000cwm.googlegr oups.com>,
"Paul Boddie" <paul@boddie.or g.ukwrote:

Would it benefit the Python community more if Python shipped with
MySQL support out of the box? Is it likely that a user suddenly finds
him/herself needing to connect to a MySQL database?

The other problem is that it chains the python release schedule to
that of MySQL AB (and postgresql, and whatever.) One of the key
advantages to independent modules is that I don't need to update my
entire python system every time my database vendor releases a new
library, nor do I have to accept lag time as the module is tested
and rolled into the base distribution. Database client libraries
are much more of a moving target than core language features.

I find it interesting that PHP struggled with these issues and
decided to abandon embedded MySQL support partly because they
couldn't maintain their own parallel versions of the client
libraries.

http://us2.php.net/manual/en/faq.databases.php#faq.databases.mysql.ph

p5

Among the other problems faced by both PHP and Python in bundling
MySQL support is that they can't legally do it without adopting the
GPL.

Which leaves me wondering why the python core should adopt a feature
that was abandoned by PHP, and was never highly recommended or used?

Paul

[Ramon Diaz-Uriarte]

Re: Python does not play well with others

I find Paul Rubin's arguments compelling and convincing. As just a
Python user (i.e., someone who has contributed nothing) let me add a
few comments along the same lines.

On 03 Feb 2007 18:31:03 -0800, Paul Rubin
<"http://phr.cx"@nospam. invalidwrote:

skip@pobox.com writes:

PaulWhy would I expect your employer to solve my problems anyway, even
Paulif they relate to some module that you actually use?

Your reasoning seems to be that Python should contain the functional union
of everything at least in Java and PHP if not Perl, Ruby and Tcl as well.

>
I wouldn't go quite that far. I think there are specific application
areas, such as web server apps, where the Python advocates here on
clpy pitch Python relentlessly against those other languages. Given
that context, Python's stdlib should try to match the libraries of
those other languages in those areas. There are other areas where
Python doesn't get pitched as hard and the other languages have
acknowledged advantages. So it's ok if Python's stdlib gets less
attention in those areas.
>

In fact, it is quite frustrating to operate under the impression that
"Python is good for X, Y, Z, ...." and then realize "ooops, it is
becoming a pain in the ass to do X, whereas language L does X just
fine". It seems to me that Python advocates sometimes (often?) get
carried away. DB "sure, no problem"; web-frameworks "who needs Rails,
we have (lots of) frameworks that do it"; functional programming
"Python can do all the functional programming anyone in his/her mind
should ever try to do"; etc.

Compare this to the, in my opinion, equanimous, fair, "advertisem ent"
one finds in the Erlang page or the recognition by schemers that
scheme is not the only game in town.

>

If you want to turn the Python distribution into a kitchen sink,
make the argument on python-dev and be prepared to shoulder your
share of the burden should your arguments sway the group as a whole.

>
We've had this conversation before and I continue to think your
reasoning above is invalid. I'm not a Python developer, I'm just a
user, and my volunteer coding priorities are elsewhere, as I've
explained before. Python's developers and advocates have a declared
goal of reaching as many users as they can, and as a user I don't mind
offering suggestions about how to do that, but my responsibilitie s
don't go any further.

R.

--

http://mail.python.org/mailman/listinfo/python-list

>

--
Ramon Diaz-Uriarte
Statistical Computing Team
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)

http://ligarto.org/rdiaz

[Paul Boddie]

Re: Python does not play well with others

Jorge Godoy wrote:

"Paul Boddie" <paul@boddie.or g.ukwrites:
>

And while Python eggs may be useful for people managing additional
software as some unprivileged user, hosting providers (and virtual
private server administrators) will want packages that fit in with the
rest of the software being managed in the hosting environment.

>
And why eggs wouldn't satisfy them? Eggs can be installed globally as well,
making the package available to every client of this hosting server (if they
mount their libs from a unique NFS server then it would automatically be
available for all of their servers).

Because Python is just another thing to support for various hosting
providers. One of the problems people supposedly have when persuading
such companies to add or update packages is, I imagine, a lack of
familiarity those companies have with Python technologies. Asking them
to use a technology-specific packaging system might be a bit much if
they aren't even familiar with (or interesting in knowing more about)
the packages required to satisfy their customers' basic needs.

Moreover, there's a lot of software in the average GNU/Linux or BSD
distribution, but if you had to "break out" into technology-specific
package/dependency managers to install some of the more heterogeneous
packages, it would be a nightmare: perhaps installing the
documentation would have you visit CPAN for some Perl scripts before
throwing you into the TeX package manager to get some missing TeX
libraries; then you might be off into setuptools for the Python
scripting extensions, possibly via equivalent tools for other
scripting extensions; finally, you'd be off via any other system
thought essential to manage software from a particular parochial
technological viewpoint. I've had to install software like this, and
it takes huge amounts of time for no good reason if you can use system
packages instead.

Paul

[Colin J. Williams]

Re: Python does not play well with others

Paul Rubin wrote:

"George Sakkis" <george.sakkis@ gmail.comwrites :

>>What does "batteries included" mean to you? To me, it means you don't
>>have to install add-ons.

>So let's make a 500MB executable and add Numpy, Zope, Django, PIL,
>pretty much everything actually. Even better, make CheeseShop just a
>frontend to a build system that adds and updates automatically
>submitted packages to the core. Problem solved ! <wink>.

>
Numpy should certainly be included and I think there are efforts in
that direction.

As I understand it, the effort is directed towards elaborating the
current array module so that it handles multi-dimensional array.

This would be a good step but numpy os much more and is of minority
interest and so should probably not, in my opinion, be included in the
Python core.

Colin W.
There is also a movement to choose a web framework to

include and Django might be a good choice. I think the Zope
maintainers want to keep Zope separate and I think PIL has an
incompatible license. I'm not sure what you mean about making
CheeseShop a front end to a build system, but I certainly don't think
random user contributions should get added to the core automatically.
Including a module in the core should carry with it the understanding
that the module has undergone some reasonable evaluation by the core
maintainers and is maintained.
>
I do think the core should have more stuff than it does, so that its
functionality can be on a par with competing language distros like
J2SE and PHP. Both of those distros include database connectvity
modules and web frameworks. It could be that those other packages can
include more stuff because they have more active developer communities
and can therefore expend more resources maintaining their libraries.
But if that's the case, since the Java and PHP languages themselves
suck compared with Python, we have to ask ourselves why Python has not
been able to attract similar levels of effort and what it could be
doing differently.

[John Nagle]

Re: Python does not play well with others

Paul Boddie wrote:

I was really advocating improvements to the built-in SSL support,
anyway, which was also what the complainant was suggesting before
people started asking him mistakenly why he thought that Python was
weakened by some third party packages (PyOpenSSL, M2Crypto).

The real problems are with integration of modules that
aren't written in Python. Python-only modules don't have the
version compatibility problems which C modules do. Loading
a Python-only module from an external source is usually not
a big deal. Building a C module, especially one with
dependencies on other components, can be a big deal.
So, focus on problems with modules which have C
components.

John Nagle