how fast is Python?

[dan]

It would be an understatement to say I love this language. What used
to take me all day now takes 3 hours, and I can spend the rest of the
time on my bike thinking about the problems from a high level instead
of wrestling with arcane compiler problems, etc.

Back in the day, when looking at an interpreted language (or even
compiled ones) the first thing I would ask is, "how fast is it?"
These days, with 1ghz processor machines selling for < $500, it seldom
comes up as an issue. And of course in Py's case you can always
'extend and embed' your core routines for fun & profit.

However, there are definitely cases where a lot of code would need to
be optimized, and so I ask the question: How fast is Python, compared
to say a typical optimizing C/C++ compiler?

I realize this is a more complex question than one might think. There
are various types of code constructs that might end up with different
efficiency issues. I guess what I'm asking is, in a general sense,
how fast is it now for typical code sequences, and -- importantly --
what could be done to optimize the interpreter? Are any parts written
in assembly? Could things like hash tables be optimized with parallel
units such as MMX? Etc.

Please advise.

[Irmen de Jong]

Re: how fast is Python?

Python is fast enough for me, especially 2.3.

Profile & code slow parts as C extensions.
Include your own assembly there if so desired.

Investigate Psyco. There was one example on this
newsgroup that showed that Python+psyco actually
outperformed the same program in compiled C.

--Irmen

[Andrew Dalke]

Re: how fast is Python?

dan:[color=blue]
> However, there are definitely cases where a lot of code would need to
> be optimized, and so I ask the question: How fast is Python, compared
> to say a typical optimizing C/C++ compiler?[/color]

Highly dependent on context. I use factor of 10-20 as a ballpark,
with factor of 100 for some things like low-level string processing.
Eg, I've got a pure Python regexp engine which clocks at about x80
slower than sre.
[color=blue]
> what could be done to optimize the interpreter? Are any parts written
> in assembly? Could things like hash tables be optimized with parallel
> units such as MMX? Etc.[/color]

Spend a few tens of millions on developing just-in-time compilers
and program analysis. That worked for Java.

Nothing is written in assembly, except that C can be considered
a portable assembly language. Otherwise ports to different platforms
would be a lot more difficult.

I would hope that the C compiler could optimize the C code
sufficiently well for the hardware, rather than tweaking the
code by hand. (Though I know of at least one person who sent
in a patch to gcc to optimize poorly written in-house code.
Rather circuitous way to fix things, but it worked.)

Andrew
dalke@dalkescie ntific.com

[Irmen de Jong]

Re: how fast is Python?

Alex Martelli wrote:[color=blue]
> Irmen de Jong wrote:[/color]
[color=blue][color=green]
>>Investigate Psyco. There was one example on this
>>newsgroup that showed that Python+psyco actually
>>outperforme d the same program in compiled C.[/color]
>
>
> I think (but will gladly stand corrected if I'm wrong!) that
> this is a misinterpretati on of some code I posted -- the
> C code (crazily) used pow(x,2.0), the Python one (sanely)
> x*x -- within a complicated calculation of erf, and that
> one malapropism in the C code was what let psyco make
> faster code than C did. With C fixed to use x*x -- as any
> performance-aware programmer will always code -- the
> two ran neck to neck, no advantage to either side.[/color]

Whoops, I missed that :) Thanks for the clarification.

Nevertheless, a Psyco-optimized piece of Python code
that runs as fast as compiled C is still very impressive
to me. I know that JIT compiler technology theoretically
could produce better optimized code than a static optimizing
compiler, but am happy already if it reaches equal level :-)

--Irmen de Jong

[Alex Martelli]

Re: how fast is Python?

Irmen de Jong wrote:
...[color=blue]
> Nevertheless, a Psyco-optimized piece of Python code
> that runs as fast as compiled C is still very impressive
> to me. I know that JIT compiler technology theoretically
> could produce better optimized code than a static optimizing
> compiler, but am happy already if it reaches equal level :-)[/color]

If anybody does have an actual example (idealy toy-sized:-)
where psyco's JIT does make repeatably faster code than a
C compiler (well-used, e.g. -O3 for gcc, NOT just -O...!-)
I'd be overjoyed to see it, by the way.


Alex

[David McNab]

Re: how fast is Python?

On Thu, 21 Aug 2003 06:40:04 +0200, Michael Peuser paused, took a deep
breath, then came out with:
[color=blue]
> A bottleneck can be Tkinter. Use something different then (Qt, wx)..[/color]

Wow!

I've found wx to be way slower than Tkinter.

On a P133 running Win98, a McMillan-compiled prog using wx took twice as
long to start up as a similar prog implemented in Tkinter.

[Mark Carter]

Re: how fast is Python?

> > However, there are definitely cases where a lot of code would need to[color=blue][color=green]
> > be optimized, and so I ask the question: How fast is Python, compared
> > to say a typical optimizing C/C++ compiler?[/color][/color]

I did a benchmark some time ago (nothing optimised):

PURPOSE:
The purpose of this technical report is to gauge the relative speed of
the languages: VB, VBA, Python 2.2, C++, and Fortran.


SUMMARY RESULTS:
It was discovered that uncompiled VB code in VB 6.0 ran at the same
speed as VBA code in Excel. It was half the speed of compiled VB code,
5 times the speed of Python, and 1/20th the speed of C++/Fortran.

METHOD:

The following algorithm was implemented in each of the target
languages:

X = 0.5
For I = 1 to 108
X = 1 – X* X
Next

Timings were made for the execution. The following results were
obtained:

Language Timing (seconds)
VB – uncompiled 74
VB – compiled 37
VBA – Excel 75
Python 401
C++ - debug version 4
C++ - release version 3
Fortran 3


The timings for Fortran are approximate. The execution time had to be
timed with a stopwatch because timing functions could not be
discovered.

[Neil Padgen]

Re: how fast is Python?

On Wednesday 20 August 2003 21:19, Irmen de Jong wrote:[color=blue]
> Investigate Psyco.[/color]

On the strength of this thread, I investigated Psyco. Results of a
very quick investigation with the following program:

-----------------------------------------
def calcPi(iteratio ns):
pi4 = 1.0
for i in xrange(1, iterations):
denominator = (4*i)-1
pi4 = pi4 - 1.0/denominator + 1.0/(denominator+2)
return pi4 * 4.0

def timethis(func, funcName):
import sys
try:
i = int(sys.argv[1])
except:
i = 1000000
import time
start = time.time()
pi = func(i)
end = time.time()
print "%s calculated pi as %s in %s seconds" % (funcName, pi, end
- start)

def main():
timethis(calcPi , 'calcPi')
timethis(speedy Pi, 'speedyPi')

import psyco
speedyPi = psyco.proxy(cal cPi)

if __name__ == '__main__':
main()
-----------------------------------------

produced the following results on a 1.7GHz P4 running FreeBSD 4.8:
[color=blue]
> python2.2 pi.py[/color]
calcPi calculated pi as 3.14159315359 in 3.87623202801 seconds
speedyPi calculated pi as 3.14159315359 in 0.790405035019 seconds

-- Neil

[Peter Hansen]

Re: how fast is Python?

dan wrote:[color=blue]
>
> However, there are definitely cases where a lot of code would need to
> be optimized, and so I ask the question: How fast is Python, compared
> to say a typical optimizing C/C++ compiler?[/color]

C is roughly 10 to 100 times faster than Python, though of course it's
easy to find cases outside of this range, on either side.

I use 30 as a general overall rule of thumb, in the exceptionally
few cases where it seems relevant how much faster C would be.

And in those very few cases, so far, I have consistently concluded
I'm happy enough with the speed of Python given that the speed of
*development* in Python is easily 5 to 10 times faster than the
speed of development in C. (And again, it's easy to find cases
outside of this range, on either side...)

-Peter

[Alex Martelli]

Re: how fast is Python?

Mark Carter wrote:
...[color=blue]
> The following algorithm was implemented in each of the target
> languages:
>
> X = 0.5
> For I = 1 to 108
> X = 1 – X* X
> Next
>
> Timings were made for the execution. The following results were
> obtained:
>
> Language Timing (seconds)
> VB – uncompiled 74
> VB – compiled 37
> VBA – Excel 75
> Python 401
> C++ - debug version 4
> C++ - release version 3
> Fortran 3[/color]

Interesting. One wonders what and where you measured, e.g:

[alex@lancelot gmpy]$ cat a.cpp
int main()
{
double X = 0.5;
for(int i = 0; i < 108; i++)
X = 1 + X * X;
return 0;
}

[alex@lancelot gmpy]$ g++ -O3 a.cpp
[alex@lancelot gmpy]$ time ./a.out
0.01user 0.00system 0:00.00elapsed 333%CPU (0avgtext+0avgd ata 0maxresident)k
0inputs+0output s (186major+21min or)pagefaults 0swaps

i.e., it's just too fast to measure. Not much better w/Python...:

[alex@lancelot gmpy]$ cat a.py

def main():
X = 0.5
for i in xrange(108):
X = 1 + X*X

main()
[alex@lancelot gmpy]$ time python -O a.py
0.03user 0.01system 0:00.15elapsed 26%CPU (0avgtext+0avgd ata 0maxresident)k
0inputs+0output s (452major+260mi nor)pagefaults 0swaps

i.e., for all we can tell, the ratio COULD be 100:1 -- or just about
anything else! Perhaps more details are warranted...


Alex

[achrist@easystreet.com]

Re: how fast is Python?

dan wrote:[color=blue]
>
> I realize this is a more complex question than one might think. >
> Please advise.[/color]

Consider the percentage of software projects for which the total
number of hours of developer time over the life of the project
exceeds the total number of hours of CPU run time during productive
use of the software produced. This percentage is abysmally high.
Python works on improving it on both ends, by both reducing the
developer time and increasing the number of hours of productive
use. What more could you want?


Al

[Andrew Dalke]

Re: how fast is Python?

Travis Whitton[color=blue]
> [the shootout] is probably the best site on the
> internet for side-by-side language comparisons:[/color]

Though there's also pleac.sf.net which isn't for timings
but does show how the different languages would be
used to do the same thing.

And I see my Python contribution still leads the
pack in % done.

Andrew
dalke@dalkescie ntific.com

[Michael Peuser]

Re: how fast is Python?

"David McNab" <postmaster@127 .0.0.1> schrieb im Newsbeitrag
news:pan.2003.0 8.21.09.07.20.1 00414@127.0.0.1 ...[color=blue]
> On Thu, 21 Aug 2003 06:40:04 +0200, Michael Peuser paused, took a deep
> breath, then came out with:
>[color=green]
> > A bottleneck can be Tkinter. Use something different then (Qt, wx)..[/color]
>
> Wow!
>
> I've found wx to be way slower than Tkinter.
>
> On a P133 running Win98, a McMillan-compiled prog using wx took twice as
> long to start up as a similar prog implemented in Tkinter.[/color]

Of course! The wx DLL is mor ethan 6 MB whilest Tcl/Tk still keeps around 1.
I am not talking about start up. When you have ever used a Canvas with a
600x800 Image oder with a thousend items or a TIX HList with a dozend
diffently styled columns you might know WHAT I am talking about.

Even with less filled widgets, most of what you perceive as "lazy" with e.g.
games is generally not the Python but the Tcl interpreter. Pygame shows that
you can dio fast visualisation with Python.

Kindly
Michael P

[Michael Peuser]

Re: how fast is Python?


"Neil Padgen" <neil.padgen@mo n.bbc.co.uk> schrieb im Newsbeitrag
news:bi2ki9$pai $1@nntp0.reith. bbc.co.uk...[color=blue]
> On Wednesday 20 August 2003 21:19, Irmen de Jong wrote:[color=green]
> > Investigate Psyco.[/color]
>[/color]

[...][color=blue]
>
> produced the following results on a 1.7GHz P4 running FreeBSD 4.8:
>[color=green]
> > python2.2 pi.py[/color]
> calcPi calculated pi as 3.14159315359 in 3.87623202801 seconds
> speedyPi calculated pi as 3.14159315359 in 0.790405035019 seconds
>
> -- Neil[/color]

This is certainly correct. My experiance with more general programs running
for a few minutes shows that you can expect a speed-up of two. This is still
impressiv when you have your results in 5 instead of 10 minutes..

Kindly
Michael P

[dan]

Re: how fast is Python?

Peter Hansen <peter@engcorp. com> wrote in message news:<3F44DFEB. 60AB6492@engcor p.com>...
....[color=blue]
> And in those very few cases, so far, I have consistently concluded
> I'm happy enough with the speed of Python given that the speed of
> *development* in Python is easily 5 to 10 times faster than the
> speed of development in C. (And again, it's easy to find cases
> outside of this range, on either side...)
>[/color]
I pretty much agree. The point of my question was not to knock Python
-- I'm simply curious how fast, _in_principle_, a language like Python
could be made to run.

I've looked at Psyco and Pyrex, I think both are interesting projects
but I doubt anything in the Py world has had nearly the kind of
man-hours devoted to optimization that Java, C++, and probably C# have
had.