Re: Native Code vs. Python code for modules

[John Nagle]

koblas wrote:

Ruby has been getting pummeled for the last year or more on the
performance subject. They've been working hard at improving it. From
my arm chair perspective Python is sitting on it's laurels and not
taking this as seriously as it probably should.

PyPy was supposed to help, but that project has been dragging on
for half a decade now. Personally, I think the Shed Skin approach
is more promising. PyPy has too many different goals, and tends
to generate lots of auxiliary tools, blogs, videos, and "sprints",
but not a usable PyPy compiler.

We'll have to see if PyPy 1.1 works.

John Nagle

[bearophileHUGS@lycos.com]

Re: Native Code vs. Python code for modules

John Nagle:

Personally, I think the Shed Skin approach
is more promising.

ShedSkin will probably have scaling problems: as the program size
grows it may need too much time to infer all the types. The author has
the strict policy of refusing any kind of type annotation, this make
it unpractical.

And, despite your interest in ShedSkin, so far very few people have
given a hand actually developing SS (I think partially because
ShedSkin Python sources aren't much hackable. This is very bad for an
OpenSource project), so I think the author now has lost part of the
will to develop this project (but probably we'll see one of two more
versions).

For me so far the most viable way to produce a faster Python system
seems a version of CPython with Cython and something Psyco-like built-
in (and a built-in compiler on Windows, like MinGW 4.2.1), maybe with
some syntax support in the Python language, allowing to mix statically
compiled Python code with dynamically compiled Python code in an easy
way (as CLisp sometimes does).

Bye,
bearophile

[srepmub]

Re: Native Code vs. Python code for modules

ShedSkinwill probably have scaling problems: as the program size
grows it may need too much time to infer all the types. The author has
the strict policy of refusing any kind of type annotation, this make
it unpractical.

well, I admit I really don't like manual type annotations (unless for
documentation purposes). it seems a much nicer (..pythonic) approach
to just get type information from a profiler. if I had four hands (and
two brains), shedskin would probably already include one.

that said, I know of several ways to improve the scalability
shedskin's type analysis itself, and I might still pursue those. but I
think, in combination with a profiler, things should scale pretty well
already.. certainly enough to compile most smallish programs/extension
modules of up to a few thousands lines.

And, despite your interest inShedSkin, so far very few people have
given a hand actually developing SS (I think partially

well, it's been quite a few people actually, about 15 that have
contributed substantial improvements. of course doing a compiler like
this is probably more than 10 person-years of work, so I could always
use more help.

becauseShedSkin Python sources aren't much hackable. This is very bad
for an

OpenSource project), so I think the author now has lost part of the

I think they are reasonably hackable for the most part, and this can
only improve. in the beginning I had little documentation, and there
was just this 7000-line Python file :-) now things are more split up,
and I even added documentation recently to each part. yes, type
inference will always be hard to hack on, but that's only one part.
the C++ side, where I can arguably use most help, and which consists
of more than half of the code, has always been easily hackable.

will to develop this project (but probably we'll see one of two more
versions).

I have my ups and downs of course, but at the moment I'm quite
enthousiastic about the whole thing, in part because people are
actually contributing. a new release is coming up, with support for
datetime and ConfigParser among many other improvements/fixes, and
there is a much faster set implementation in the pipeline. at the
moment, I have no plans to halt development at all.

For me so far the most viable way to produce a faster Python system
seems a version of CPython with Cython and something Psyco-like built-
in (and a built-in compiler on Windows, like MinGW 4.2.1), maybe with
some syntax support in the Python language, allowing to mix statically
compiled Python code with dynamically compiled Python code in an easy
way (as CLisp sometimes does).

shedskin can of course generate extension modules (shedskin -e), that
can be imported from larger Python programs. it's a bit clumsy, as
only builtins can be passed to/from shedskin, and everything (args,
return values) is copied recursively, but it can be quite useful
already. and of course it can only improve as well..


mark.