why python is slower than java?

[Maurice LING]

Re: why python is slower than java?

[color=blue]
>
> dude that "comparisio n" from twistedmatrix you refrence is ANCIENT!!![/color]

I am wondering the impact when IBM decided that the base memory to not
exceed 64kb, in the late 1960s...

I suppose more experienced people in this list can agree that certain
decisions made can be almost an edict. So, there is a re-building
process every now and then, hopefully to by-pass such edicts. Python
itself is already such an example.
[color=blue]
>
> it is comparing versions that are YEARS out of date and use![/color]

Are the codebase of Python 1.5.2 and Java 1.1 totally replaced and
deprecated?

Lisp compiler is the 1st compiler to be created (according to the
Red-Dragon book, I think) and almost all others are created by
bootstrapping to LISP compiler. What are the implications of design
decisions made in LISP compiler then affecting our compilers today? I
don't know. I repeat myself, I DO NOT KNOW.
[color=blue]
>
> you are just trolling or your don't know enough to understand the
> answer to your question which is way to vague to be answered, as there
> is no real correct answer.[/color]

Certainly I do not have 15 PhDs in computer science or computating
mathematics.... .. I suppose there are some syntax error in your
statement to allow me to parse it completely. "too vague", not "to vague".

Thanks
maurice

[Maurice LING]

Re: why python is slower than java?

[color=blue]
> Not only that, but Maurice Ling has started a long thread a few weeks
> looking for a good research topic for his thesis (involving Java &
> Python, btw). At some points he was bashed, but went ahead with the
> discussion, that ended up touching in several interesting topics.[/color]

Thank you. You can do a google search and find my details anyway. Don't
have to go around guessing my details. I have no idea of the motives of
whoever that highlight "unimelb.edu.au ". Does he have a problem with my
institution or what, I am not bothered, as I am only a student. Email to
the Vice Chancellor if he has a point to make, don't take it out on me,
I don't have the title of a Professor prefixing my name.

There is a Chinese saying "waves do not happen without wind." If my
impression of Python and Java is flawed, I am seriously wondering where
it came from? Is it all due to benchmark data from years ago?

Before I am accused of trolling etc etc again, let me say this all over
again. I am using both Python and Java actively and had contributed to
BioPython project as well.
[color=blue]
>
> As for the climate in c.l.py, it's just interesting to note that the
> climate *is* getting less friendly, and that it does coincide with
> several old-timers moving away from the list. A few years ago I
> remember that the likes of Tim Peters, effbot, Skip Montanaro, and
> several others (sorry, I would really like to remember more names from
> the top of my head) were frequent posters here. They've now moved to
> other interests, or are focusing their efforts on the python-dev list.
> It must not be a coincidence.
>[/color]
Maybe as the old-timers go off, the next batch of old-timers feel that
it is too much to handle. Nobody requires anyone to answer to all
threads.....

Thanks
Maurice

[Maurice LING]

seems that someone is anonymous

Message from yahoo.com.
Unable to deliver message to the following address(es).

<jarrodhroberso n@yahoo.com>:
64.157.4.78 failed after I sent the message.
Remote host said: 554 delivery error: dd This user doesn't have a
yahoo.com account (jarrodhroberso n@yahoo.com) [0] -
mta109.mail.sc5 .yahoo.com

[Eric S. Johansson]

Re: why python is slower than java?

Israel Raj T wrote:[color=blue]
> "Eric S. Johansson" <esj@harvee.org > writes:
>
>[color=green]
>>I will argue that we might want to start including in the
>>headers of mailing list/newsgroups a dana reference.[/color]
>
>
> I would argue that this would debase newsgroups even further.[/color]

debase in what way? I certainly don't see enough money flowing that it
would lead major python contributors down a road of debauchery and
depravity. at least, those that aren't already there.

First, there would be a bunch of things required to make this work. A
pay on the newsreader client, a connection to a payment system, a
registered identity for payment. This is not a trivial amount of work.

Second, much of the conflict around various forms of intellectual
property boil down to getting paid. By building a low overhead
mechanism through which people could get paid for all sorts of
intellectual property (questions answered, music, video, writings,
code), we would jumpstart a new economic engine.

initially there would be a big rush of everybody on the face the planet
trying to get that dollar payment. It would probably be not unlike the
swarms of beggars that surround rich westerners when they travel through
less well-off regions. We would evolve filters and rating systems which
would be used to cut down on the noise. Eventually some contributors
would fallaway and others would persist and get better.

nothing would compel anyone to pay anything. But I could see eventually
folks being rated on whether or not they do pay. If you've got someone
who's always asking time-consuming questions and not paying anything, it
would be totally appropriate to filter out that person.

in the same community of users, you would have some people operating on
a gift economy, others operating on a financial economy and I believe
somehow, it would work. I can't yet describe how but I believe it would
work.

---eric

[Kent Johnson]

Re: why python is slower than java?

Alex Martelli wrote:[color=blue]
> Kent Johnson <kent3737@yahoo .com> wrote:[color=green]
>>I rarely find myself acting as an apologist for Java...[/color]
>
> I'm glad I posted a sufficiently silly comparison to elicit some
> response, then;-)[/color]

Yes, good success on that one :-)
[color=blue]
> Which python and java versions are you using?[/color]

Old. Unfortunately the Mac is no longer my primary machine and not
up-to-date. I just used it to get timings comparable to yours. It's
Java 1.4.1_01
Python 2.3.3
Mac OSX 10.2.4

....We now return to our regularly scheduled program of trying to
convince Java programmers to try Python...

Kent

[Terry Hancock]

Re: why python is slower than java?

On Saturday 06 November 2004 05:44 pm, Maurice LING wrote:[color=blue]
> There is a Chinese saying "waves do not happen without wind." If my
> impression of Python and Java is flawed, I am seriously wondering where
> it came from? Is it all due to benchmark data from years ago?[/color]

Not really. It's an impression you could easily get from several
books about Python (e.g. O'Reilly's "Learning Python"), which
make a rather big deal about Python being slow to execute, but
fast to develop in.

The reality is that it isn't really all *that* slow to execute,
and later versions have gotten quite a bit faster.

But it remains really fast to develop in. ;-)

The resistence to the idea really stems from a sense of rivalry
with Java. Which is really interesting actually, because it
wasn't all that long ago that Python was "just a scripting
language" and Java programmers wouldn't feel threatened by it.
Now they do, I guess. ;-)

It's quite possible that Java programmers console themselves
for all the low-level programming work by thinking the result
will be faster, without really testing to find out. Certainly
Java is going to be faster than Jython (Python running on a
Java platform).

And I have certainly written some extremely poorly optimized
Python programs that positively *crawled*.

Cheers,
Terry

--
Terry Hancock ( hancock at anansispacework s.com )
Anansi Spaceworks http://www.anansispaceworks.com

[Kent Johnson]

Re: why python is slower than java?

Terry Hancock wrote:[color=blue]
> The reality is that [Python] isn't really all *that* slow to execute,
> and later versions have gotten quite a bit faster.
>
> But it remains really fast to develop in. ;-)[/color]
[color=blue]
> Certainly
> Java is going to be faster than Jython (Python running on a
> Java platform).[/color]

This is sort of like saying that C is faster than Python. To the Java
programmer, it's an excuse to avoid looking at Jython. To the Jython
programmer, it's "Yes, and your point is...?"

The reality is that Jython isn't really all *that" slow to execute.

But it remains really fast to develop in. ;-)

And you can always recode critical sections in Java for speed.

OK, you can probably guess what language I use in my day job now...

Kent

[Roy Smith]

Re: why python is slower than java?

Terry Hancock <hancock@anansi spaceworks.com> wrote:[color=blue]
> And I have certainly written some extremely poorly optimized
> Python programs that positively *crawled*.[/color]

My guess is the poor performance had nothing to do with the language you
wrote it in, and everything to do with the algorithms you used.

Local optimizations rarely gets you more than a factor of 2 improvement.
Choice of language (at least within the same general catagory such as
comparing one native compiled language to another, or one virtual
machine language to another) probably has a somewhat broader range, but
still a factor of 10 would be quite surprising.

To get really bad performance, you need to pick the wrong algorithm. A
project I worked on a while ago had a bit of quadratic behavior in it
when dealing with files in a directory. Most of our customers dealt
with file sets in the 100's. One customer had 50,000 files. Going
from, say, 500 to 50,000 is a 100-fold increase in N, and gave a
10,000-fold increase in execution time.

[Roger Binns]

Re: why python is slower than java?

Alex Martelli wrote:[color=blue]
> Roger Binns <rogerb@rogerbi nns.com> wrote:[color=green]
>> Incidentally, that Java code copies one character at a time.[/color]
>
> Yep, I noticed -- pretty silly, but if that's how the Java designers
> decided the read method should behave by default, who am I to argue?
> Just an example I grabbed off the net, first google hit for (if I
> recall correctly) java file reading that had the source of a complete
> example.[/color]

I/O is one area where Java is *very* different than other environments.
Java emphasises the ability to select at runtime (typically from
properties files) how to do things. This is usually done by
having layers of interfaces, and many implementations of factories
and sometimes factories of factories. That ultimately means
the code will work the same for almost any source and destination,
including doing buffering (an interface), byte to character
conversion (an interface), the actual source (an interface),
output (another interface).

Consequently I/O is one of the hardest things for a newbie Java
programmer to do since it appears so complex due to the flexibility
offered. It is also why bad examples exist, because people
find what works for them and post it.

Note how in Python, files do read/write whereas sockets do
send/recv.

But you are comparing Apples to Oranges. Java programs are written
in certain ways with various emphases (flexibility, interfaces and
factories). Python programs emphasise other things (generators,
typing). Exceptions are expensive in Java and not used much. They
are "cheap" in Python and used frequently.
[color=blue][color=green]
>> The Python code is reading the entire string into memory and
>> then writing it.[/color]
>
> Right, _Python_'s default.[/color]

Arguably Python's default is reading a line at a time, and it
is a bad default in some circumstances (large files), just
as the Java code was a bad way of doing anything but small
files.
[color=blue]
> The claim posted to this newsgroup, without any support nor examples
> being given, was that Python's I/O was far slower than Java's in
> _disk-intensive_ operations. I'm still waiting to see any small,
> verifiable examples of that posted on this thead.[/color]

If the language code is the same, then that claim boils down to
the Java Native Interface vs the Python C API. In the case of
Java, I can see the claim having some relevance in multi-threaded
code since Java doesn't have the GIL.
[color=blue]
> defaults are tuned, making Python much faster. Great, then let those
> who claim Java's I/O is much faster in disk intensive operation post
> suitable examples, and we'll see.[/color]

Your timing included the overhead of starting up and shutting down
both environments, making the rest of the measure less than
interesting.
[color=blue][color=green]
>> Perhaps it is also worth linking to the projects done in
>> Python on SourceForge and elsewhere?
>>
>> http://sf.net/softwaremap/trove_list.php?form_cat=178[/color]
>
> If the message you're keen to send is "Python is great for
> open-source", yes. If you're focusing on "Python is great for your
> _business_" (as the python *business* forum does, for example), then
> emphasizing open-source projects can reasonably be considered
> secondary to emphasizing projects that make or save money for the
> businesses which developed them.[/color]

The idea isn't to emphasise the open source side, but rather so
that anyone can see for themselves how it was all put together.
If I have a business critical app, and claim it is written in
Python but noone can see the insides then they can't really
know too much. The biggest thing is that they can't tell
if they could write code like that (or even how much was written)
to produce an app of similar functionality and complexity.

Roger

[Maurice LING]

Re: why python is slower than java?

[color=blue]
> Not really. It's an impression you could easily get from several
> books about Python (e.g. O'Reilly's "Learning Python"), which
> make a rather big deal about Python being slow to execute, but
> fast to develop in.[/color]

Come to think of it, yes, "Python, The Complete Reference", "Programmin g
Python" and "Learning Python" all seems to have that "speed warning" tag
in the beginning chapters.

If this seems to be the wrong impression, should the authors do something?
[color=blue]
>
> The reality is that it isn't really all *that* slow to execute,
> and later versions have gotten quite a bit faster.
>
> But it remains really fast to develop in. ;-)[/color]

I attest to that, at least for the development speed.
[color=blue]
>
> It's quite possible that Java programmers console themselves
> for all the low-level programming work by thinking the result
> will be faster, without really testing to find out. Certainly
> Java is going to be faster than Jython (Python running on a
> Java platform).[/color]

Jython, as I know, wraps each Python objects in a Java class. For
example, each variable is a class of it own. I suppose this overhead
does have speed penalities on it, simply because "every dash of cheese
is calories, even for low fat cheese."

At the same time, Jython FAQ question 1.6 (how fast is jython?) suggests
that Jython may run 10 times slower than CPython. Personally, I hadn't
seen this kind of performance in my programs yet. On the other hand,
"Jython Essentials" suggest that Jython is about 1.5 times the speed of
CPython (CPython takes about 75% of the time compared to Jython). To
this, I've also not seen this fast in Jython codes myself. I will say
that 2-5 times seems to be a reasonable range.

Cheers
Maurice

[Fábio Mendes]

Re: why python is slower than java?

Em Sáb, 2004-11-06 às 22:07 -0500, Roy Smith escreveu:[color=blue]
> Terry Hancock <hancock@anansi spaceworks.com> wrote:[color=green]
> > And I have certainly written some extremely poorly optimized
> > Python programs that positively *crawled*.[/color]
>
> My guess is the poor performance had nothing to do with the language you
> wrote it in, and everything to do with the algorithms you used.[/color]

Well, try to write a Linear Algebra algorithm in pure python... Then
you'll see that pyrex, scipy or the C API are your friends. Of course
algorithmis matters, but for some intensive CPU applications python can
be as slow as 1/100 C using the same algorith. This, of course, matters.
At least if your program is running this kind of operation most of the
time.
[color=blue]
> Local optimizations rarely gets you more than a factor of 2 improvement.
> Choice of language (at least within the same general catagory such as
> comparing one native compiled language to another, or one virtual
> machine language to another) probably has a somewhat broader range, but
> still a factor of 10 would be quite surprising.[/color]

10x or 20x difference is likely to hurt you. If you have a 20x slower
computer were you be using the same apps as you use now? For scientific
simulations (which interest me most), it's the difference between get
your results after 1day calculation of one month...
[color=blue]
> To get really bad performance, you need to pick the wrong algorithm. A
> project I worked on a while ago had a bit of quadratic behavior in it
> when dealing with files in a directory. Most of our customers dealt
> with file sets in the 100's. One customer had 50,000 files. Going
> from, say, 500 to 50,000 is a 100-fold increase in N, and gave a
> 10,000-fold increase in execution time.[/color]

This is an extreme case, but is a typical behaviour of how a good
algorithm can modify the execution times of your program: scaling up
neatly. The language matters indeed, but python is well served in a
broad range of libs. For the most CPU intensive tasks there are usually
python bindings for C/C++/Fortran libraries, so it's usually possible to
execute python code at native speeds. This is why I use python in my
(simple) physics applications: scipy gives me a very good interface for
evaluating heavy numeric stuff. When I program in C++ I'm usually
tempted to write a lot of code from the scratch, for I'm lazy to search
for the write libs and learn how they work. I'm a physicist, not a real
programer, so this end up with C++ code slower than python's (which uses
scipy magic).

The point I want to make here is that library programmers usually makes
much better (in the sense it's faster) code than an application
programmer. So, in any language, if you want to write a big project from
the scratch, you'll probably end up with a buggy, slow and cumbersome
library for the 'low level' stuff, and the high level interface will
never move on. If you pick up the existing libs and extend them to
tailor your needs, you have better chances of success. Python seems to
be better than java in this point (at least in the OSS arena), for there
is an enourmously bigger set of libraries you can start with, and is
easier to wrap C/C++/Fortran code to it. So the likely 10% or so runtime
penalty I get for running a C lib through python is more than paid in
application development time. To me, what matters is: can a __python
program__ run as fast as a __c program__? In lots of cases: yes, just
use the right lib.

All this 'java interpreter is faster than python interpreter' nonsense
doesn't appeal to me. For intensive CPU operations, neither interpreters
are good, or even decent, so there is no subistitute for a good o' low
level implementation. Can java use those implementations as easyly as
python? --No-- Does java have a comparable set of low level (fast!) libs
wrapped to it? --Not as rich as python--. So forget the language
shootout contest! Those are not real world examples, they're complete
crap. Python has a richer set of faster libs than java. Use them, so
when it matters, python usually can be faster than java. This is enough
for me for saying that python IS faster than java.

If you care much, a python program can be almost as fast as a C one,
just use a python wrapper to a C lib or use pyrex! This is not saying
that language doesn't matter. Not all so called scripting languages have
the same facilities as python. For instance perl is very optimized to
regex substitution, but lacks more broad biddings for, for example
numeric libraries. We're lucky to use python, we're lucky that it has so
many facilities. Other scripting languages have nice syntax, powerful
builtin objects and other gizmos that make them much more expressive and
productive than the low level counterparts. IMHO python is unique in the
fact that it also provides very nice set of wrapped low level libraries
to run runtime critical pieces of code, so in a lot of cases we can get
the best of both worlds. Java is too much a in between for me: execution
not so fast, but not so slow; development not so fast, but not so slow;
builtins not so expressive, but not so low level either; it is not that
good, neither that bad, etc...

Cheers,
Fabio

[kosh]

Re: why python is slower than java?

On Saturday 06 November 2004 4:10 am, Alex Martelli wrote:[color=blue]
> with Python 2.4 beta 1 for the roughly equivalent:
>
> inputFile = file("/usr/share/dict/web2", 'r')
> outputFile = file("/tmp/acopy", 'w')
>
> outputFile.writ e(inputFile.rea d())
>
> inputFile.close ()
> outputFile.clos e()
>[/color]

I think a generator version works even better. I did tests at various files
sizes and overall the generator one was better and it was vastly better at
large file sizes and overall the generator one also impacted the system less.
The regular version which reads the whole file at once get really bad with
large files especially if the system is being used.

I suspect the generator version should work with any size file that the os is
capable of working with and should be resource friendly at any size.

Just thought the generator version would be a good comparison to the java
version and strangely enough in many cases it is actually faster then the
regular version. :)

Generator Version:
inputFile = file("/home/kosh/KNOPPIX_V3.6-2004-08-16-EN.iso", 'r')
outputFile = file("/tmp/acopy", 'w')

outputFile.writ elines(inputFil e)

inputFile.close ()
outputFile.clos e()

Regular Version:
inputFile = file("/home/kosh/temp.txt", 'r')
outputFile = file("/tmp/acopy", 'w')

outputFile.writ e(inputFile.rea d())

inputFile.close ()
outputFile.clos e()

The timing was done with python 2.3.4 on debian/sid


Large File Test:
File Copied: 733499392 KNOPPIX_V3.6-2004-08-16-EN.iso

Real User Sys
Gen 0m33.478s 0m4.302s 0m3.542s
Reg 2m28.029s 0m0.010s 0m4.992s *
Reg 0m34.913s 0m0.009s 0m4.713s

* This is how long the first run took. The machine swapped
heavily. The Other time is for subsequent runs. This method
overall uses a massive ammount of ram.


Memory Usage:

Virt Res Shr
Gen 3816K 2364K 2524K
Reg 703M 700M 2524K

Small File Test

File Copied: 2754459 Zope-2.7.2-0.tgz

Real User Sys
Gen 0m0.049s 0m0.023s 0m0.014s
Reg 0m0.037s 0m0.009s 0m0.019s


Tiny File Test:

File Copied: 205 May temp.txt

Real User Sys
Gen 0m0.012s 0m0.007s 0m0.005s
Reg 0m0.028s 0m0.009s 0m0.003s

[Maurice LING]

Re: why python is slower than java?

[color=blue]
> What if we do, _AND_ carefully follow Eric Raymond's excellent
> recommendations each and every time we ask for help? Are then we
> allowed to loathe and despise the mass of clueless dweebs?-)[/color]

I do have to thank you for the story on Plato and his students (in
"reverse jython" thread).

Now that courtnesy is done, I believe you do have all rights not to
reply to all requests for help. As in the law, you have the rights to
remain silent...
[color=blue]
> I can be a newbie at a bazillion subjects, easily -- but I cannot truly
> be a newbie at such tasks as human interaction, social dynamics, general
> information retrieval and processing. I can easily guess what will
> happen if I enter any mailing list or newsgroup with both guns blazing
> out of frustration, for example, and therefore I cannot easily
> sympathize with anybody who _does_ behave so foolishly. It's not a
> matter of expertise about any specific subject, not even exactly one of
> skills, but rather one of personal maturity and character.[/color]

Bazillion subjects, my respect. Looking back at this thread...

1. I've asked a question which I may be wrong (many people do that)

2. Some had answered and pointed out my misconceptions, thank you all
for that.

3. Some had pointed to the fact that I am from University of Melbourne
(with unknown motives). It was then clarified that I was an honours
student (www.zoology.unimelb.edu.au) in the Dept of Zoology. Perhaps I
may say that I am a molecular biologist by degree.

4. Some had pointed out instances (books and all) whereby my wrong
impressions might have been formed.

5. Some had painfully taxed on the my initial misconceptions and
generalizing it to the ridicule of non-experts, and in the process of
so, suggesting controversial codes in the pre-text of eliciting responses.

6. Furthering it, using notable words from famous people, to
discriminate against a group of people, when the first instance had been
breached... you can choose not to reply.

All these happens when the discussion had been taken to other areas... I
brings one to wonder on maturity and character......
[color=blue]
>
> I know how to search mailing list archives, or google groups ones, and
> considerate enough to use this easily acquired and very useful knowledge
> to try and avoid wasting other people's time and energy, for example, by
> airing some complaint that's been made a thousand times and answered
> very comprehensively . When I can't find an answer that way, I ask with
> courtesy and consideration and appreciation for the time of the people
> who, I hope, will be answering my questions. Etc, etc -- reread Eric's
> essay on how to ask for help, it's a great piece of work.[/color]

All the knowledge in this world are in libraries and now, networked
libraries via internet. Today, almost all high school students in
developed countries are versed in internet. And by your argument, it
seems that all universities are complete waste of money as all knowledge
is out there and the tools to access the knowledge is readily available.

As mentioned, the discussion is heading else where and my misconceptions
cleared before your replies. If I've indeed forgotten, my sincere
apologies and hereby thank you for your time and efforts.
[color=blue]
>
> That doesn't mean a newbie isn't always welcome, _if_ they show any sign
> whatever of being worth it. But asking for tolerance and patience
> against _rude_ newbies which barge in with shrill, mostly unjustified,
> repetitious complaints, is, I think, a rather far-fetched request.
>
>
> Alex[/color]

You still have the right to remain solemn.

maurice

[Tim Roberts]

Re: why python is slower than java?

Maurice LING <mauriceling@ac m.org> wrote:[color=blue]
>
>I've already said the following and was not noticed:
>
>1. it is a disk intensive I/O operation.
>2. users delay is not in the equation (there is no user input)
>3. I am not interested in the amount of time needed to develop it. But
>only interested execution speed.[/color]

It is fabulous that you are able to enumerate your list of requirements so
completely. I'm quite serious; many people embark on even complicated
projects without a clear understanding of the tradeoffs they will
encounter.

However, given that set of needs, why would you mess with an "exotic"
language at all? Why wouldn't you just write straight to the metal in C++
or C?
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.

[Maurice LING]

Re: why python is slower than java?

[color=blue]
> It is fabulous that you are able to enumerate your list of requirements so
> completely. I'm quite serious; many people embark on even complicated
> projects without a clear understanding of the tradeoffs they will
> encounter.
>
> However, given that set of needs, why would you mess with an "exotic"
> language at all? Why wouldn't you just write straight to the metal in C++
> or C?[/color]

Perhaps you can attribute it to C-phobia.
maurioce