python24.dll and encodings ?

[Bugs]

I believe I read in a relatively recent thread that the reason
python24.dll is so large compared to previous releases is that all the
language encodings are linked into the library?

Are there any plans for future releases to split the encodings out so
that, for example, if someone wanted to make a smaller executable using
py2exe without all the language encodings, they could do so?

I suppose one could always compile their own version of the python24.dll
but for those of us that are compiler-challanged, splitting out the
encodings would be nice.

Is that even feasible?
Thanks!

[Martin v. Löwis]

Re: python24.dll and encodings ?

Bugs wrote:[color=blue]
> I believe I read in a relatively recent thread that the reason
> python24.dll is so large compared to previous releases is that all the
> language encodings are linked into the library?[/color]

Not only that (but also). In addition, it also contains modules that
were previously implemented as separate .pyd files (_csv, _sre,
_symtable, _winreg, datetime, mmap, parser).
[color=blue]
> Are there any plans for future releases to split the encodings out so
> that, for example, if someone wanted to make a smaller executable using
> py2exe without all the language encodings, they could do so?[/color]

I previously said that I would do such a thing if somebody provided a
specification other than "split out the encodings". I.e. write a PEP
that specifies how to determine whether an extension module should be
included into pythonxy.dll, and when it should not; this specification
should allow to reason about modules that haven't yet been contributed.

The PEP should be accompanied with an implementation, i.e. a set of
patches to PCbuild.

Then, the patch should be discussed, and find support in the community.
[color=blue]
> I suppose one could always compile their own version of the python24.dll
> but for those of us that are compiler-challanged, splitting out the
> encodings would be nice.
>
> Is that even feasible?[/color]

Technically, yes. However, somebody needs to take the lead.

Regards,
Martin

[Bugs]

Re: python24.dll and encodings ?

Martin v. Löwis wrote:[color=blue]
> Not only that (but also). In addition, it also contains modules that
> were previously implemented as separate .pyd files (_csv, _sre,
> _symtable, _winreg, datetime, mmap, parser).
>[/color]
[snip][color=blue]
>
> I previously said that I would do such a thing if somebody provided a
> specification other than "split out the encodings". I.e. write a PEP
> that specifies how to determine whether an extension module should be
> included into pythonxy.dll, and when it should not; this specification
> should allow to reason about modules that haven't yet been contributed.
>[/color]

Thanks Martin! As the python24.dll now contains modules and codecs that
weren't linked into the DLL before 2.4 and as we now need a PEP to have
them split out again, I assume there was a PEP to have them included for
2.4? Can you point me to that PEP so I can evaluate writing a new PEP
to reverse it?
Thanks!

[Martin v. Löwis]

Re: python24.dll and encodings ?

Bugs wrote:[color=blue]
> Thanks Martin! As the python24.dll now contains modules and codecs that
> weren't linked into the DLL before 2.4 and as we now need a PEP to have
> them split out again, I assume there was a PEP to have them included for
> 2.4? Can you point me to that PEP so I can evaluate writing a new PEP
> to reverse it?[/color]

There was no PEP. The current policy is "the maintainer of the Windows
port does whatever he considers the best thing". The previous maintainer
(Tim Peters) said this about the state of affairs in 2003:

http://mail.python.org/pipermail/python-dev/2003-October/038639.html

I offered a policy in:

http://mail.python.org/pipermail/python-dev/2003-October/038636.html

Thomas Heller proposed an explicit list in

http://mail.python.org/pipermail/python-dev/2003-October/038869.html

When I became maintainer of the Windows port, I made this policy:
everything should be included in pythonxy.dll, unless it requires
a library not included with the Python distribution.

I can't find a message where I communicated this policy at that time,
but I commented on the rationale several times since, e.g. in

http://groups.google.com/group/comp.lang.python/msg/dd5c8f597a86e743

Regards,
Martin

[Thomas Heller]

Re: python24.dll and encodings ?

"Martin v. Löwis" <martin@v.loewi s.de> writes:
[color=blue]
> Bugs wrote:[color=green]
>> Thanks Martin! As the python24.dll now contains modules and codecs that
>> weren't linked into the DLL before 2.4 and as we now need a PEP to have
>> them split out again, I assume there was a PEP to have them included for
>> 2.4? Can you point me to that PEP so I can evaluate writing a new PEP
>> to reverse it?[/color]
>
> There was no PEP. The current policy is "the maintainer of the Windows
> port does whatever he considers the best thing". The previous maintainer
> (Tim Peters) said this about the state of affairs in 2003:
>
> http://mail.python.org/pipermail/pyt...er/038639.html
>
> I offered a policy in:
>
> http://mail.python.org/pipermail/pyt...er/038636.html
>
> Thomas Heller proposed an explicit list in
>
> http://mail.python.org/pipermail/pyt...er/038869.html
>
> When I became maintainer of the Windows port, I made this policy:
> everything should be included in pythonxy.dll, unless it requires
> a library not included with the Python distribution.
>
> I can't find a message where I communicated this policy at that time,
> but I commented on the rationale several times since, e.g. in
>
> http://groups.google.com/group/comp....5c8f597a86e743[/color]

A PEP discussing the rationales would *really* be great.

Thomas

[Bugs]

Re: python24.dll and encodings ?

Thomas Heller wrote:
[snip][color=blue]
>
> A PEP discussing the rationales would *really* be great.[/color]

My whole impetus for this thread was to minimize the size of Python
executables created with py2exe. Right now they tend to be rather
large, even for very small applications, as they include a bunch of
unutilized Python functionality and/or encodings.

It would be nice to see a Python environment structure that is more
desktop-application-friendly for applications that distribute the Python
environment WITH the application (i.e. using py2exe). Ideally, it would
be nice to be able to distribute as minimal amount of the Python runtime
environment as possible.

New PEP: "Small is beautiful!" ;-)

[Martin v. Löwis]

Re: python24.dll and encodings ?

Bugs wrote:[color=blue]
> New PEP: "Small is beautiful!" ;-)[/color]

A PEP would have to be more elaborate than that;
it best be accompanied with an implementation as well.
Somebody has to do the work.

Regards,
Martin

[Bugs]

Re: python24.dll and encodings ?

Martin v. Löwis wrote:[color=blue]
> Bugs wrote:
>[color=green]
>>New PEP: "Small is beautiful!" ;-)[/color]
>
>
> A PEP would have to be more elaborate than that;
> it best be accompanied with an implementation as well.
> Somebody has to do the work.
>
> Regards,
> Martin
>[/color]

That was my obviously ineffective attempt at humor, hence the ;-)

[Thomas Heller]

Re: python24.dll and encodings ?

Bugs <dont@spam.me > writes:
[color=blue]
> Thomas Heller wrote:
> [snip][color=green]
>> A PEP discussing the rationales would *really* be great.[/color]
>
> My whole impetus for this thread was to minimize the size of Python
> executables created with py2exe. Right now they tend to be rather
> large, even for very small applications, as they include a bunch of
> unutilized Python functionality and/or encodings.[/color]

That was what I guessed.
[color=blue]
> It would be nice to see a Python environment structure that is more
> desktop-application-friendly for applications that distribute the
> Python environment WITH the application (i.e. using py2exe). Ideally,
> it would be nice to be able to distribute as minimal amount of the
> Python runtime environment as possible.[/color]

Before py2exe had the single-file option, the most common 'complaint'
was the large number of files it generated. Now, the 'complaint' is
about the size of the executable. (Historically, the change in
Python2.4 was done by Martin on my request, that was before the
single-file option in py2exe. Plus, Python2.4 contains much more
encodings than 2.3 had.)

The PEP would probably have to find a balance between the number of
files and the size of a 'typical' py2exe built application. Personally,
I don't care too much on either the file number and the files sizes
since I use inno setup for distributing the applications.

Technically, as far as py2exe is concerned, it would be nice if the zlib
module was a builtin module because it would be easier to bootstrap the
executable - for obvious reasons the zlib module cannot be loaded from a
(compressed) zipfile. Unfortunately, this collides with Martin's
current policy to exclude extensions that need third party source code
from the python dll.

To summarize: Before someone requests a change to the builtin modules
list, a PEP would be needed to find the right balance.

Thomas

[Martin v. Löwis]

Re: python24.dll and encodings ?

Thomas Heller wrote:[color=blue]
> Technically, as far as py2exe is concerned, it would be nice if the zlib
> module was a builtin module because it would be easier to bootstrap the
> executable - for obvious reasons the zlib module cannot be loaded from a
> (compressed) zipfile. Unfortunately, this collides with Martin's
> current policy to exclude extensions that need third party source code
> from the python dll.[/color]

That could change, of course. I personally would have no problems
requiring that zlib is available on the build machine - I always
have it available.

I would expect that others using the subversion trunk on Windows
would object, though. Perhaps it might be possible to create
some sort of compile time condition, ideally automatically
detected.

In any case, a PEP would have to answer these issues.

Regards,
Martin

[Martin v. Löwis]

Re: python24.dll and encodings ?

Bugs wrote:[color=blue]
> That was my obviously ineffective attempt at humor, hence the ;-)[/color]

I wasn't in a humurous mood :-) there have been too many threads
on this subject, and I'm just tired giving the same answers over
and over again.

Regards,
Martin