Python compilers?

[Svein Ove Aas]

Re: Python compilers?

Heiko Wundram wrote:
[color=blue]
> Am Dienstag, 18. Mai 2004 13:41 schrieb Jacek Generowicz:[color=green]
>> Native compilers for other languages just as dynamic as Python
>> exist. These compilers manage to achieve very significant speed
>> increases[*].[/color]
>
> You are again refering to LISP as an example of a dynamic language which
> when compiled gives huge speed increases. This is true in some respect,
> in others it isn't. LISP has the advantage that type-inference may be
> used throughout the program to create one version of each function,
> which can then be compiled. Of course it still has to call into runtime
> functions to do the high-level work, but there is actually only one
> representation of each finished LISP program, and only one set and one
> proper order of runtime-functions to call.
>
> In Python this isn't true. Python, instead of LISP, is "completely "
> dynamic, meaning that it's pretty impossible to do type-inference for
> each function that is called (even checking types isn't possible). E.g.
> how do you expect type-inference to work with the pickle module? string
> -> something/Error would be the best description what pickle does. For
> the function which calls pickle, do you want to create versions for each
> possible output of Pickle? Which outputs of Pickle are possible?
> (depends on the loaded modules, which can be loaded at runtime) There is
> no (sane) way to create machine-code which calls into the appropriate
> (low-level) Python-runtime functions (such as Py_List*, Py_Dict*, etc.)
> for such a method, at least not at compile-time.[/color]

I suppose that's true; pickle is an exception, and the compiler would pick
that up. (The Lisp (print) function is approximately equivalent, at least
when *print-readably* is true.)

What you're claiming, though, is that it's possible to write Python code
that can't easily be translated to equivalent Lisp code. Can you give an
example?

[Thorsten Kampe]

Re: Python compilers?

* Tor Iver Wilhelmsen (2004-05-18 17:26 +0100)[color=blue]
> Grant Edwards <grante@visi.co m> writes:
>[color=green]
>> Dude, didn't you take high-school math? 1/3 _is_ 0.33333...[/color]
>
> No, because at some point you will stop writing 3's, either out of
> boredom, exhaustion or because you need to pee. At that instant, you
> introduce a rounding error, making 3 * 1/3 = 0.99999999999.. . instead
> of 1.0[/color]

Must have been a long time since you went to school... 1/3 is
/exactly/ 0.3...: http://mathworld.wolfram.com/RepeatingDecimal.html

Thorsten

[Yermat]

Re: Python compilers?

Heiko Wundram wrote:[color=blue]
> [...]
>
> In Python this isn't true. Python, instead of LISP, is "completely " dynamic,
> meaning that it's pretty impossible to do type-inference for each function
> that is called (even checking types isn't possible). E.g. how do you expect
> type-inference to work with the pickle module? string -> something/Error
> would be the best description what pickle does. For the function which calls
> pickle, do you want to create versions for each possible output of Pickle?
> Which outputs of Pickle are possible? (depends on the loaded modules, which
> can be loaded at runtime) There is no (sane) way to create machine-code which
> calls into the appropriate (low-level) Python-runtime functions (such as
> Py_List*, Py_Dict*, etc.) for such a method, at least not at compile-time.
>
> At runtime, this is possible. See what psyco does. There's a nice presentation
> on the psyco-website which explains what it does, and I guess you'll
> understand when you see that why writing a "compiler" for Python is pretty
> impossible.[/color]

That is here where you are wrong !
It can be known at compile time if you know every modules that will be
imported. That is called "closed environment".

If it is the case, you would be able to compile a program even if you
would not be allowed to do incremental compilation... That just mean
that you will need to recompile everything each time you modified something.

The demonstration is quite easy:
In a "closed environment" there is a finite number of classes. So you
just have to create as much specialized functions as classes. Then where
you can infere type, call directly the good function. Elsewhere, just
call a runtime dispatcher. In fact, it is already used in some languages
like Eiffel with certain optimizations.

The problem is that many python programs are not "closed" at
compile-time, ie they import or eval stuff only known at run-time.

impossible n'est pas français ;-)

--
Yermat

[simo]

Re: Python compilers?

Leif K-Brooks <eurleif@ecritt ers.biz> wrote:
[color=blue][color=green]
> > Is anyone working on a python-to-native compiler?
> > I'd be interested in taking a look.[/color][/color]
[color=blue]
> If it's faster code execution, the primary slowdown with a very
> high-level language like Python is caused by high-level data structures
> (introspection, everything being an object, etc.), not the code itself.
> A native compiler would still have to use high-level data structures to
> work with all Python code, so the speed increase wouldn't be very much.[/color]

I think the main slowdown most people would see is the startup time -
the interpreter takes way too long to start (why the company where I
work won't replace Perl/PHP with Python).
[color=blue]
> If it's ease of distribution you're looking for, I think distutils can
> make standalone programs on Windows, and most Linux distros have Python
> installed by default.[/color]

Oh that one is funny - do most distro's have PyQT and wxPython (and
their supporting Qt/GKT libs) installed by default, oh and which ones
come with 2.3 by default?
[color=blue]
> If you just think a compiler would be cool (or would like to see how it
> would be done), check out Psyco, Pyrex, and probably some other
> projects. Pyrex even combats the speed issue by allowing native C types
> to be used in addition to Python high-level types.[/color]

I thought Pyrex was a hybrid of C and Python (like Jython/Java) not
actually a Python-to-C convertor? And Pysco is just a different VM
isn't it?

I could really push Python where I work if there was a native
compiler, my company uses C/C++/Java/Qt and were looking at QSA as a
way to allow the user to script things, but as all of our products
integrate with our software protection system, we can't be
distributing source or easily decompiled bytecode!

We could replace the whole lot with PyQt and use an embedded Python
interpreter for the scripting! Ah the frustration :-(

[Yermat]

Re: Python compilers?

simo a écrit :

[...][color=blue]
> I thought Pyrex was a hybrid of C and Python (like Jython/Java) not
> actually a Python-to-C convertor? And Pysco is just a different VM
> isn't it?[/color]

Pyrex is python plus access to C structures and type declarations. But
then the current implementation create intermediary C files.

Psyco is not a different VM, it's like the JIT of java. It's a Just In
Time compilers, ie it runs over the CPythonVM.
[color=blue]
> [...][/color]

--
Yermat

[Roger Binns]

Re: Python compilers?

SeeBelow@SeeBel ow.Nut wrote:[color=blue]
> A related problem is that it seems to be a big deal to call C routines
> from Python. I have not actually tried it, because when I read about
> how its done I was not able to understand it.[/color]

Use SWIG. If your C header file is well written, you may not need to
give any additional information to SWIG.

Roger

[Mitja]

Re: Python compilers?

Thorsten Kampe <thorsten@thors tenkampe.de>
(news:1mklty031 ke5a.dlg@thorst enkampe.de) wrote:[color=blue]
> * Tor Iver Wilhelmsen (2004-05-18 17:26 +0100)[color=green]
>> Grant Edwards <grante@visi.co m> writes:
>>[color=darkred]
>>> Dude, didn't you take high-school math? 1/3 _is_ 0.33333...[/color]
>>
>> No, because at some point you will stop writing 3's, either out of
>> boredom, exhaustion or because you need to pee. At that instant, you
>> introduce a rounding error, making 3 * 1/3 = 0.99999999999.. . instead
>> of 1.0[/color]
>
> Must have been a long time since you went to school... 1/3 is
> /exactly/ 0.3...: http://mathworld.wolfram.com/RepeatingDecimal.html
>[/color]

Even worse....
0.9999999999999 99999999... is exactly 1 :)

[color=blue]
> Thorsten[/color]

[Svein Ove Aas]

Re: Python compilers?

Mitja wrote:
[color=blue]
> Thorsten Kampe <thorsten@thors tenkampe.de>
> (news:1mklty031 ke5a.dlg@thorst enkampe.de) wrote:[color=green]
>> * Tor Iver Wilhelmsen (2004-05-18 17:26 +0100)[color=darkred]
>>> Grant Edwards <grante@visi.co m> writes:
>>>
>>>> Dude, didn't you take high-school math? 1/3 _is_ 0.33333...
>>>
>>> No, because at some point you will stop writing 3's, either out of
>>> boredom, exhaustion or because you need to pee. At that instant, you
>>> introduce a rounding error, making 3 * 1/3 = 0.99999999999.. . instead
>>> of 1.0[/color]
>>
>> Must have been a long time since you went to school... 1/3 is
>> /exactly/ 0.3...: http://mathworld.wolfram.com/RepeatingDecimal.html
>>[/color]
>
> Even worse....
> 0.9999999999999 99999999... is exactly 1 :)[/color]

Only for infinite counts of '9', and computers don't do infinity.

[Peter Hansen]

Re: Python compilers?

Tor Iver Wilhelmsen wrote:
[color=blue]
> Grant Edwards <grante@visi.co m> writes:
>[color=green]
>>Dude, didn't you take high-school math? 1/3 _is_ 0.33333...[/color]
>
> No, because at some point you will stop writing 3's, either out of
> boredom, exhaustion or because you need to pee. At that instant, you
> introduce a rounding error, making 3 * 1/3 = 0.99999999999.. . instead
> of 1.0[/color]

Actually, Grant probably meant the "..." (which is an ellipsis,
meaning it substitutes for something else that is left out) to
represent "3 repeating to infinity", the same as putting a dot
over the last 3, or a bar over the last three 3s, or whatever
other convention you might have seen. Of course, it's just a
convention, so perhaps someone else would think it meant "3s
repeating to the limit of the computer's precision" or something
like that...

-Peter

[Peter Hansen]

Re: Python compilers?

SeeBelow@SeeBel ow.Nut wrote:
[color=blue]
> Leif K-Brooks wrote:[color=green]
>>
>>What are you trying to achieve?[/color]
>
> Yes, fast execution. I have been using C. In my applications there is
> a population of "chromosome s" which are arrays of floats, about 2 to 5 k
> in length. Then there are subroutines which operate on a chromosome
> using pointers. For example, the "crossover" routine uses two pointers
> to swap portions of two chromosomes. My software sometimes runs for
> hours, perform many millions of operations like these. Clearly, speed
> of execution is of dramatic importance.[/color]

The bottleneck is almost certainly in evaluating the fitness
function, not in performing the mutations and cross-overs.
What does your fitness function do with all those floats?
Perhaps it can be handled much faster with one of the numeric
extensions for Python... or with Pyrex.

-Peter

[Carl Banks]

Re: Python compilers?

[[ Note for Peter Hansen: all uses of the word "compiler" below is
understood to refer to an optimizing compiler to machine language, and
I mean a real, not virtual, machine. ]]


Heiko Wundram wrote:[color=blue]
> Am Dienstag, 18. Mai 2004 13:41 schrieb Jacek Generowicz:[color=green]
>> Native compilers for other languages just as dynamic as Python
>> exist. These compilers manage to achieve very significant speed
>> increases[*].[/color]
>[/color]
[color=blue]
> You are again refering to LISP as an example of a dynamic language
> which when compiled gives huge speed increases. This is true in some
> respect, in others it isn't. LISP has the advantage that
> type-inference may be used throughout the program to create one
> version of each function, which can then be compiled.[/color]

Can you give an example of a function Lisp is able to compile that
Python manifestly couldn't. I don't buy it.


[snip][color=blue]
> In Python this isn't true. Python, instead of LISP, is "completely "
> dynamic, meaning that it's pretty impossible to do type-inference
> for each function that is called (even checking types isn't
> possible).[/color]

I don't follow you. In what way is Python dynamic that Lisp isn't?
And Python certainly can check types.

[color=blue]
> E.g. how do you expect type-inference to work with the pickle
> module? string -> something/Error would be the best description what
> pickle does. For the function which calls pickle, do you want to
> create versions for each possible output of Pickle? Which outputs
> of Pickle are possible?[/color]

You can write pickling package in Lisp. I think the Lisp compiler
would handle such a package fine, and I see no reason why a
hypothetical Python compiler wouldn't.

[color=blue]
> (depends on the loaded modules, which can be loaded at runtime)
> There is no (sane) way to create machine-code which calls into the
> appropriate (low-level) Python-runtime functions (such as Py_List*,
> Py_Dict*, etc.) for such a method, at least not at compile-time.[/color]

That might be true, but pickling is only one module. The fact that
I'm able to write a pickling package in Lisp doesn't make it
impossible to write a Lisp compiler, and the fact that pickling module
exists in Python doesn't make it impossible to write a Python
compiler.

What would a Lisp compiler do faced with a Lisp pickling package?

[color=blue]
> At runtime, this is possible. See what psyco does. There's a nice
> presentation on the psyco-website which explains what it does, and I
> guess you'll understand when you see that why writing a "compiler"
> for Python is pretty impossible.[/color]

Well, I don't buy it, and I don't see any fundamental between Python
and Lisp dynamicism. The only thing you've demonstrated is that there
is some code in Python that could make optimizing difficult--a
statement which is true of Lisp also--but it's a specific case that is
not applicable generally.


--
CARL BANKS http://www.aerojockey.com/software
"If you believe in yourself, drink your school, stay on drugs, and
don't do milk, you can get work."
-- Parody of Mr. T from a Robert Smigel Cartoon

[SeeBelow@SeeBelow.Nut]

Re: Python compilers?

Peter Hansen wrote:
<snip>[color=blue]
> The bottleneck is almost certainly in evaluating the fitness
> function, not in performing the mutations and cross-overs.
> What does your fitness function do with all those floats?
> Perhaps it can be handled much faster with one of the numeric
> extensions for Python... or with Pyrex.[/color]

You are right, the worst bottleneck is the fitness function. Every
population member is an ANN (artificial neural network) and the ANNout()
function must be called for each fitness evaluation. You can see there
is a lot of looping. This C code runs very quickly:
-----------------------------------------------------------------------
/* neursubs.c - for computing output of artificial neural network - for
EvSail-2.2
M. Timin, August, 2003

piecewise parabolic approximator replaced conventional sigmoid,
October, 2003
*/


#include <math.h>
#include <stdlib.h>

#define NEUR_MAX 80 /* maximum number of neurons in a
layer */
#define LOOP(i,N) for(i=0; i<N; i++) /* be careful using this! */

/* This 4 piece curve is a good sigmoid approximator. */
float sigAprox(regist er float x) {
register float z;

if(x <= -4.0)
return 0.0;
else if(x <= 0.0) {
z = x + 4.0;
return z*z/32;
}
else if(x < 4.0) {
z = x - 4.0;
return 1.0 - z*z/32;
}
else
return 1.0;
}

/* oneNeuron() uses a vector of inputs and a vector of weights, and the
sigmoid activity
function, to compute the output of one neuron. It is assumed that an
extra
input of 1.0 is at the beginning of the input vector, and that there
is a
corresponding value at the beginning of the weight vector. This is
actually
the bias. So upon entering this function, wptr points to the bias
and inptr
points to 1.0. The inputCount should include the bias, so it should
be one
more than the number of inputs. */
float oneNeuron(float *inptr, float *wptr, int inputCount) {
int i;
float sum = 0.0;

LOOP(i, inputCount) { /* summation loop */
sum += *inptr++ * *wptr++;
}
return sigAprox(sum); /* this is the sigmoid formula */
}

/* This is the routine which calculates the outputs of the ANN. Before
calling it the input
values must be in the array pointd to by inptr. Values of the
outputs will be placed
in the array pointed to by outValues */
void ANNout(int numIn, /* number of inputs to the ANN */
int numHid, /* number of neurons that receive the inputs */
int numOut, /* number of final output neurons */
float *inptr, /* pointer to the array of input values */
float *wptr, /* pointer to array of weights & biases in a
specific order */
float *outValues) /* pointer to where to write the output */
{
float t1[NEUR_MAX]; /* NEUR_MAX defined above */
float t2[NEUR_MAX];
int i;

/* prepare the input array: */
t1[0] = 1.0;
LOOP(i, numIn)
t1[i+1] = *inptr++;
/* compute and store intermediate outputs: */
t2[0] = 1.0;
LOOP(i, numHid)
{
t2[i+1] = oneNeuron(t1, wptr, numIn+1);
wptr += numIn+1;
}
/* do similar for final layer, writing to destination */
LOOP(i, numOut)
{
outValues[i] = oneNeuron(t2, wptr, numHid+1);
wptr += numHid+1;
}
}


-----------------------------------------------------------------------
Humans may write to me at this address: zenguy at shaw dot ca

[Greg Ewing]

Re: Python compilers?

SeeBelow@SeeBel ow.Nut wrote:[color=blue]
> Yes, fast execution. I have been using C. In my applications there is
> a population of "chromosome s" which are arrays of floats, about 2 to 5 k
> in length. Then there are subroutines which operate on a chromosome
> using pointers. For example, the "crossover" routine uses two pointers
> to swap portions of two chromosomes.[/color]

Take a look at Pyrex. It may be just what you need:

http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/

--
Greg Ewing, Computer Science Dept,
University of Canterbury,
Christchurch, New Zealand

http://www.cosc.canterbury.ac.nz/~greg

[Paul Rubin]

Re: Python compilers?

Carl Banks <imbosol@aerojo ckey.invalid> writes:[color=blue]
> I don't follow you. In what way is Python dynamic that Lisp isn't?[/color]

[color=blue][color=green][color=darkred]
>>> class foo:[/color][/color][/color]
.... def bar(self, x):
.... return x*x
....[color=blue][color=green][color=darkred]
>>> a = foo()
>>> a.bar(3)[/color][/color][/color]
9[color=blue][color=green][color=darkred]
>>> a.bar = lambda x: x*x*x
>>> a.bar(3)[/color][/color][/color]
27[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

[Michael Hudson]

Re: Python compilers?

Svein Ove Aas <svein+usenet01 @brage.info> writes:
[color=blue]
> What you're claiming, though, is that it's possible to write Python code
> that can't easily be translated to equivalent Lisp code. Can you give an
> example?[/color]

class C:
def __eq__(self, other):
return True

for example. For better or worse, Python *is* more dynamic than
Common Lisp, and this *does* contribute to making it harder to make
Python go fast.

I wrote a rant about this subject:

http://starship.python.net/crew/mwh/hacks/speeding-python.html

Cheers,
mwh

--
Ability to type on a computer terminal is no guarantee of sanity,
intelligence, or common sense.
-- Gene Spafford's Axiom #2 of Usenet