Python is far from a top performer according to benchmark test...

[Carl]

"Nine Language Performance Round-up: Benchmarking Math & File I/O"

http://www.osnews.com/story.php?news_id=5602

I think this is an unfair comparison! I wouldn't dream of developing a
numerical application in Python without using prebuilt numerical libraries
and data objects such as dictionaries and lists.

I have been experimenting with numerical algorithms in Python with a heavy
use of the Numeric module. My experience is that Python is quite fast in
comparison with (and sometimes as fast as) traditional languages such as C
or C++.

The greatest advantage of Python is the great increase in productivity and
the generation of a much smaller number of bugs due to the very clean and
compact structure Python invites you to produce. Sometimes it amazes me how
fast I can produce a working algorithm in Python. The step from an
algorithmic outline on a paper to a working code is very short. The
interactive nature of the Python console invites numerical experimentation
and data exploration. This wasn't mentioned in the article, what a pity!

Carl

[Irmen de Jong]

Re: Python is far from a top performer according to benchmark test...

Carl wrote:
[color=blue]
> "Nine Language Performance Round-up: Benchmarking Math & File I/O"
> http://www.osnews.com/story.php?news_id=5602[/color]

This benchmark is beaten to pulp in the discussion about it on slashdot.
It's a real stupid benchmark (as most benchmarks are) IMNSHO.

I mean, using python's arbitrary precision long object for 'math'
and comparing it to the other languages' long /machine types/... come on.
And thats just one of the flaws.

--Irmen

[Krzysztof Stachlewski]

Re: Python is far from a top performer according to benchmark test...

"Carl" <phleum_nospam@ chello.se> wrote in message
news:ryELb.238$ tK2.228@amstwis t00...
[color=blue]
> I have been experimenting with numerical algorithms in Python with a heavy
> use of the Numeric module. My experience is that Python is quite fast in
> comparison with (and sometimes as fast as) traditional languages such as C
> or C++.[/color]

With "heavy use of Numeric module" you were calling functions
written in C. So how can you say that Python is fast,
when C code is doing all the work.

Stach

[Carl]

Re: Python is far from a top performer according to benchmark test...

Krzysztof Stachlewski wrote:
[color=blue]
> "Carl" <phleum_nospam@ chello.se> wrote in message
> news:ryELb.238$ tK2.228@amstwis t00...
>[color=green]
>> I have been experimenting with numerical algorithms in Python with a
>> heavy use of the Numeric module. My experience is that Python is quite
>> fast in comparison with (and sometimes as fast as) traditional languages
>> such as C or C++.[/color]
>
> With "heavy use of Numeric module" you were calling functions
> written in C. So how can you say that Python is fast,
> when C code is doing all the work.
>
> Stach[/color]

Well, I guess you are right!

What I meant was that when I run my Python algorithm it runs "almost as fast
as" when I run similar code in C or C++. This is of course due to the
highly efficient Numeric module. However, the point is that Python is a
viable language from a numerical perspective.

Carl

[Samuel Walters]

Re: Python is far from a top performer according to benchmark test...

|Thus Spake Krzysztof Stachlewski On the now historical date of Fri, 09
Jan 2004 22:13:58 +0100|
[color=blue]
> "Carl" <phleum_nospam@ chello.se> wrote in message
> news:ryELb.238$ tK2.228@amstwis t00...
>[color=green]
>> I have been experimenting with numerical algorithms in Python with a
>> heavy use of the Numeric module. My experience is that Python is quite
>> fast in comparison with (and sometimes as fast as) traditional languages
>> such as C or C++.[/color]
>
> With "heavy use of Numeric module" you were calling functions written in
> C. So how can you say that Python is fast, when C code is doing all the
> work.[/color]

Because python works best as a glue layer coordinating outside libraries
and functionality. I think the true strength of python comes from a
one-two punch of "Fast and pretty implementation with easy interface to
lower level tools." When python is not the right tool, we code our
solution with the right tool and then use python to glue all the right
tools together. For numerical processing, C is the right tool, but python
is not. Therefore, noone tried to use the wrong tool, they just used the
right tool and gave it python bindings so that python could act as a
coordinator.

I read the benchmark and I think it doesn't measure python in it's target
area. That's like taking a world-class marathon runner and wondering why
he doesn't compete well in a figure-skating event.

Sam Walters.


--
Never forget the halloween documents.

http://www.opensource.org/halloween/

""" Where will Microsoft try to drag you today?
Do you really want to go there?"""

[JanC]

Re: Python is far from a top performer according to benchmark test...

"Krzysztof Stachlewski" <stach@fr.USUN. pl> schreef:
[color=blue]
> With "heavy use of Numeric module" you were calling functions
> written in C. So how can you say that Python is fast,
> when C code is doing all the work.[/color]

I think all (or at least most) of the tested compilers, VMs, etc. were
written in C/C++, and thus are using libraries written in C/C++...

--
JanC

"Be strict when sending and tolerant when receiving."
RFC 1958 - Architectural Principles of the Internet - section 3.9

[Iwan van der Kleyn]

Re: Python is far from a top performer according to benchmark test...

Carl wrote:[color=blue]
> I think this is an unfair comparison! I wouldn't dream of developing a
> numerical application in Python without using prebuilt numerical libraries
> and data objects such as dictionaries and lists.[/color]

Well, that may be true. And many applications spend most of their time
in fast libraries anyway (GUI/DB/app servers) etc. Nice examples are Boa
Constructor and Eric3 which are fully implemented in Python and run
comfortably fast, thanks to the WxWindows and Qt libraries.

But I really dont' swallow the argument. In the few years that I'm
working with Python, I've encountered on several occasions performance
bottlenecks. And often a solution through partial implementation in C or
improvements in the algorithm is just not feasible. To paraphrase Kent
Beck: in real life you make your program run, then work and then your
manager says you've run out of time. You just cannot make it run as fast
as you would like it as well.

In other words: it would be nice if Python on average would run faster
so the need for optimisation would lessen. Psyco is a nice option to
have, though not as clear cut as I thought it would be:

http://www-106.ibm.com/developerworks/linux/library/l-psyco.html

[color=blue]
> The greatest advantage of Python is the great increase in productivity and
> the generation of a much smaller number of bugs due to the very clean and
> compact structure Python invites you to produce.[/color]

So dogma dictates. And I've found it to be true on many occasions, if
not all. BUT, the famed Python Productivity Gain is very difficult to
quantify. And for me that's a BIG but. I'm trying to push Python within
my company. Nicely presented "performanc e benchmarks" go down well with
management, bevause those are quantities which are supposedly
understood. And Python does not come across very possitively in that
respect.

Regards,

Iwan

[Tim Churches]

Re: Python is far from a top performer according to benchmarktest.. .

On Sat, 2004-01-10 at 08:05, Carl wrote:[color=blue]
> "Nine Language Performance Round-up: Benchmarking Math & File I/O"
> http://www.osnews.com/story.php?news_id=5602
>
> I think this is an unfair comparison! I wouldn't dream of developing a
> numerical application in Python without using prebuilt numerical libraries
> and data objects such as dictionaries and lists.[/color]

Benchmarks like those reported are nearly worthless, but nevertheless,
it would be interesting to re-run the them using Numeric Python and/or
Pyrex.

I notice that the author of those benchmarks, Christopher W.
Cowell-Shah, has a PhD in philosophy. Perhaps Python's very own
philosophy PhD, David Mertz, might like to repeat the benchmarking
exercise for one of his columns, but including manipulation of more
realistic data structures such as lists, arrays and dictionaries, as
Carl suggests. I'm sure it would be a popular article, and provide a
counterpoint to the good Dr Mertz's previous articles on Psyco.

--

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQA//yC2eJFGqer5k9AR Ag01AKD03VKBBxN ZBEmvHNljD+TJND MtzACeJhqN
5bvT3a5t1NQfLnp vPex3KF8=
=Lhb+
-----END PGP SIGNATURE-----

[Jeff Epler]

Re: Python is far from a top performer according to benchmarktest.. .

numarray is probably the perfect example of getting extremely good
performance from a few simple constructs coded in C. (numarray-0.8 is
less then 50k lines of C code, including comments and blank lines, so
it's also very simple given the amount of "bang" it provides)

All your important logic is in Python, you're in no way stuck worrying
about buffer lengths, when to call free(), and all those other things
that drive me crazy when I try to write in C.

If I had to execute my Python programs without executing any code
written in "C" behind the scenes, well, I'd be stuck. Of course, the
situation is about the same for any language you care to name, and if
not then substitute "machine code".

Jeff

[Tim Churches]

Re: Python is far from a top performer according to benchmarktest.. .

On Sat, 2004-01-10 at 08:13, Krzysztof Stachlewski wrote:[color=blue]
> "Carl" <phleum_nospam@ chello.se> wrote in message
> news:ryELb.238$ tK2.228@amstwis t00...
> [color=green]
> > I have been experimenting with numerical algorithms in Python with a heavy
> > use of the Numeric module. My experience is that Python is quite fast in
> > comparison with (and sometimes as fast as) traditional languages such as C
> > or C++.[/color]
>
> With "heavy use of Numeric module" you were calling functions
> written in C. So how can you say that Python is fast,
> when C code is doing all the work.[/color]

Well, yes, but the Python VM is also written in C, and every time you
make a call to a dictionary, or list etc, it is C code which is doing
all the work. If you like, you could say that Python is a set of
extremely clever wrappers around a bunch of optimised C code - but that
rather diminishes the achievement of Python. I dare say that the
Microsoft .NET and Visual Basic VMs are also written in C/C++, so you
could say the same things about them - but I don't think that is a
useful perspective.
--

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQA//yKZeJFGqer5k9AR ApbeAKC0SnajS1C Q/b6w9m+BfBLtLMep twCg9s+y
szgtzQnlsRc3cYB sNf0T7Wc=
=9rbb
-----END PGP SIGNATURE-----

[JanC]

Re: Python is far from a top performer according to benchmark test...

Tim Churches <tchur@optushom e.com.au> schreef:
[color=blue]
> I notice that the author of those benchmarks, Christopher W.
> Cowell-Shah, has a PhD in philosophy. Perhaps Python's very own
> philosophy PhD, David Mertz, might like to repeat the benchmarking
> exercise for one of his columns, but including manipulation of more
> realistic data structures such as lists, arrays and dictionaries, as
> Carl suggests.[/color]

And then don't forget to publish it on /. or nobody sees it... ;-)

--
JanC

"Be strict when sending and tolerant when receiving."
RFC 1958 - Architectural Principles of the Internet - section 3.9

[Peter Hansen]

Straw poll on Python performance (was Re: Python is far from a topperformer ...)

Iwan van der Kleyn wrote:[color=blue]
>
> In other words: it would be nice if Python on average would run faster
> so the need for optimisation would lessen.[/color]

I disagree with the above. My opinion has long been that Python runs
adequately fast and that few people should need to spend much time on
optimization. Maybe that sort of view should be put to the test.

This is my "straw poll" question:

Do you spend a "significan t" amount of time actually optimizing your
Python applications? (Significant is here defined as "more than five
percent of your time", which is for example two hours a week in a
40-hour work week.)

Note the distinction between "actually optimizing" and "worrying about
optimization" and such things. If you pause briefly during coding and
rewrite a line to use a more efficient idiom, I don't consider that to be
"optimizati on" for purposes of this question. Optimization would require
roughly (a) noticing that performance was inadequate or actual profiling
your code, and (b) rewriting specifically to get adequate performance.
Algorithmic improvements that you would make regardless of implementation
language do not qualify, and wasting time optimizing a script that you
run once a year so it takes ten seconds instead of fifteen also does not
qualify because you certainly didn't need to do it...

Yes or no answers suffice, but feel free to follow up with a paragraph
qualifying your answer (or quantifying it!). :-)

-Peter

[Paul Rubin]

Re: Straw poll on Python performance (was Re: Python is far from a top performer ...)

Peter Hansen <peter@engcorp. com> writes:[color=blue]
> This is my "straw poll" question:
>
> Do you spend a "significan t" amount of time actually optimizing your
> Python applications? (Significant is here defined as "more than five
> percent of your time", which is for example two hours a week in a
> 40-hour work week.)[/color]

Yes, absolutely.
[color=blue]
> Algorithmic improvements that you would make regardless of implementation
> language do not qualify, and wasting time optimizing a script that you
> run once a year so it takes ten seconds instead of fifteen also does not
> qualify because you certainly didn't need to do it...[/color]

Sometimes I'll take the time to implement a fancy algorithm in Python
where in a faster language I could use brute force and still be fast
enough. I'd count that as an optimization.

[Dave Brueck]

Re: Python is far from a top performer according to benchmarktest.. .

Iwan:[color=blue][color=green]
> > The greatest advantage of Python is the great increase in productivity and
> > the generation of a much smaller number of bugs due to the very clean and
> > compact structure Python invites you to produce.[/color]
>
> So dogma dictates. And I've found it to be true on many occasions, if
> not all. BUT, the famed Python Productivity Gain is very difficult to
> quantify. And for me that's a BIG but. I'm trying to push Python within
> my company. Nicely presented "performanc e benchmarks" go down well with
> management, bevause those are quantities which are supposedly
> understood.[/color]

Understood, perhaps, but very often irrelevent. That being the case, using
performance benchmarks to argue your case is a weak approach.

If you're talking to management, talk to them about something they care about,
like money. For most programs it's hard to translate performance improvements
into money: e.g. it's hard to assert that by doubling the speed of your
spell-checker implementation that sales will increase. There of course are
exceptions, but even then there's no guarantee that management would still
prefer performance above other factors if given a choice.

It's much more powerful to speak about reduced time to market, for example. Or
the ability to compete against companies with legions of programmers. Or the
decreased time it takes to turn ideas into implemented features (especially
when it's your competitor that came up with the idea). Or a lower cost of
changing directions technologically . Etc.

-Dave

[Dave Brueck]

Re: Straw poll on Python performance (was Re: Python is far from atop performer ...)

Peter wrote:[color=blue]
> Iwan van der Kleyn wrote:[color=green]
> >
> > In other words: it would be nice if Python on average would run faster
> > so the need for optimisation would lessen.[/color]
>
> I disagree with the above. My opinion has long been that Python runs
> adequately fast and that few people should need to spend much time on
> optimization. Maybe that sort of view should be put to the test.
>
> This is my "straw poll" question:
>
> Do you spend a "significan t" amount of time actually optimizing your
> Python applications? (Significant is here defined as "more than five
> percent of your time", which is for example two hours a week in a
> 40-hour work week.)[/color]

Yay, straw poll! ;-)

I program in Python full-time and each spend approximately zero hours
optimizing. In the past two years I can think of two instances in which I went
into heavy optimization mode: one was for a web server that needed to handle
hundreds of requests per second and the other was a log processor that needed
to parse and process several gigabytes of log data per hour.

In the server I added a tiny C extension to make use of the Linux sendfile API,
all the other optimizations were algorithmic. In the log processor all the
optimizations ended up being either algorithmic or doing fewer dumb things
(like recomputing cacheable data).