why python is slower than java?

[Alex Martelli]

Re: why python is slower than java?

Dan Perl <danperl@rogers .com> wrote:
[color=blue]
> Are you learning any new subject now and are you involved in a newsgroup as
> a newbie? If you're not doing that already, python experts, please try it
> and you'll probably see what it's like. No better way to see the other
> side's point of view than being on the other side.[/color]

What if we do, _AND_ carefully follow Eric Raymond's excellent
recommendations each and every time we ask for help? Are then we
allowed to loathe and despise the mass of clueless dweebs?-)

I can be a newbie at a bazillion subjects, easily -- but I cannot truly
be a newbie at such tasks as human interaction, social dynamics, general
information retrieval and processing. I can easily guess what will
happen if I enter any mailing list or newsgroup with both guns blazing
out of frustration, for example, and therefore I cannot easily
sympathize with anybody who _does_ behave so foolishly. It's not a
matter of expertise about any specific subject, not even exactly one of
skills, but rather one of personal maturity and character.

I know how to search mailing list archives, or google groups ones, and
considerate enough to use this easily acquired and very useful knowledge
to try and avoid wasting other people's time and energy, for example, by
airing some complaint that's been made a thousand times and answered
very comprehensively . When I can't find an answer that way, I ask with
courtesy and consideration and appreciation for the time of the people
who, I hope, will be answering my questions. Etc, etc -- reread Eric's
essay on how to ask for help, it's a great piece of work.

That doesn't mean a newbie isn't always welcome, _if_ they show any sign
whatever of being worth it. But asking for tolerance and patience
against _rude_ newbies which barge in with shrill, mostly unjustified,
repetitious complaints, is, I think, a rather far-fetched request.


Alex

[Brian van den Broek]

Newsgroup mood [was] why python is slower than java?

Alex Martelli said unto the world upon 2004-11-06 02:12:[color=blue]
> Hans Nowak <hans@zephyrfal con.org> wrote:[/color]

<SNIP>
[color=blue][color=green]
>>
>>Is it just me, or is the climate in c.l.py getting less friendly to
>>newbies? In any case,[/color]
>
>
> Interestingly enough, I sort of share your perception -- and I have
> noticed the same thing, and seen it remarked upon by others, in other,
> completely unrelated newsgroups as well, such as it.comp.macinto sh.
>
> It's not, I think, about newbies in general: people who come and post
> help requests, without giving the information that's quite obviously
> indispensabile to let us help them, keep getting treated with
> unreasonable amounts of friendliness and courtesy even after many
> requests for more info go unheeded, for example.
>
> However, newbies who are clueless enough to come blasting in with the
> usual whines we've heard a zillion times -- Macs cost too much, Python
> is too slow, there's no apps for Macs, Python must absolutely add
> feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
> do appear to get on our collective nerves worse than their essentially
> indistinguishab le precursors did last year, two years ago, &c.
>
> The accusation of being a troll often follows, breaking the old advice
> to "Never attribute to malice what can be adequately explained by
> incompetence". Maybe it's a case of the corollary to that old advice,
> "Sufficient ly advanced cluelessness is indistinguishab le from malice".
> But why that should be so this year more than last year, say, is not
> clear to me...
>
>
> Alex[/color]

Hi all,

I should say up front that I'm a relative newcomer to technical
newsgroups and mail-lists myself, so these comments aren't backed-up by
years of observation. I'd also like to make clear that I am responding
with general observations to what I took to be general observations in
the quoted msgs above; I don't think any of Alex, Hans, or I intend to
be talking about any one post or poster. Last, having just finished
writing this email, I see it got a bit long. There is a concrete
proposal at the end, so I invite those who feel its a ramble to skip
down 5 or so paragraphs.

The thread has offered donor fatigue and old-hands drifting away as
partial explanations, but I'd say any decline in newbie-friendliness
must be at least partly explained by success. Many in the Python
community seem to share the laudable goal of spreading Python, and not
just to the already technically informed. From Guido's Computing For
Everybody project, to the ever-patient folks on the tutor list, to any
number of other manifestations one could point to, the Python community
seems to have a "Welcome. You don't have to know what a dip-switch is to
enter here" sign up. That's a good thing. But, if it works (and it
does--witness this post ;-), it brings in people who don't know the
social norms.

That community attitude doesn't necessarily explain the Mac list Alex
cites. (I've not read the list, so cannot say.) But there too, there
must be an upsurge of new folk who finally got tired of doing what Bill
told them to do. (Among my circle, about 1 frustrated Windows user
every month or two switches to Macs.) So, it both cases, it is likely to
be driven by continued expansion of the membership outside the bounds of
those who know what elm and pine are.

Python might also be vulnerable to a strange, paradoxical phenomena: it
a way, it seems the more people get for less ($ or effort) the more they
expect. I'm on a list for 2 shareware products, a few freeware (beer
sense) ones, and a few open-source. People do seem to demand more of
list membership and project owners the more free the project is. That
one puzzles me, but I guess helps explain why the non-free,
non-responsive Redmond machine still is widely accepted.

Of all the lists (tech and non) I have read, the Python lists are among
the most tolerant and welcoming. It certainly took me a few tries to
get reasonably close to appropriate form. But, not once was I flamed or
more than gently nudged in the right direction. This friendliness is
important, to be valued, and I think protected. (I recall a post I made
to a LUG asking for help with an aborted attempt to get Linux and my
laptop's Winmodem to co-operate. I was flamed because my mail headers
indicated that I was using Thunderbird on *Windows* -- the horror! --
Not the way to welcome people to the world of Linux.)

But, if Python's increase in user-base continues, on these issues onward
and downward seems all too possible.

So, what to do? Well, I've just subscribed to the list on another
address to check, and there is nothing in the welcome message giving
people new to newsgroups and mail-lists a "Newbie's Introduction to the
Medium". Nor is there a recap posted in the monthly subscription reminder.

I'm not a fan of 'Official List Rules' full of "Thou shalt"s and that's
not what I am suggesting. I am also aware that many of the people whose
post exasperate are also the people least likely to read or attend to
such introductions. But, still, something along the lines might help.

What I have in mind is a wiki-page or three with links prominently
located in the welcome message and the reminder message. I could see at
least three distinct pages being of use. In rapid-sketch mode:

1) Welcome to the list, here are our norms
At minimum, a short 1-2 paragraph distillation of "Smart Questions", a
link to it, a reminder of the existence of the tutor list, and links to
the other two here mentioned.

2) How to find information about Python
Links to one or more of the comp.lang.pytho n archives with perhaps some
advice for optimal searching. This could include a link to a quick "How
to google".
Perhaps a link to Mark Hammond's wicked-cool Python Sidebar for
Firefox/Mozilla <http://starship.python .net/crew/mhammond/mozilla/> and
to any other similar aids.

3) Links to threads that answer common questions
Yes, there is an FAQ and yet people still ask why Python gets division
wrong. But I've often seen people dig up links to threads for questions
perhaps not so common to be written up in an FAQ, but perhaps common
enough to merit linking on a wiki page. I'm not under the illusion that
it would solve the problem on its own, but it might well head some
repeat questions off. For those it doesn't, it would give an easy way to
answer.

Yes, some (or even all) of this info is already on Python.org. But this
way would allow it to be audience-tailored and to be ultra-easy to
direct newbie's toward. And yes, not all newbies read Eric Raymond's
"Smart Questions" essay within the first 5 times they are pointed at it.
Clearly it won't be a silver bullet, but maybe it would help.

As a relative newbie myself, I think I might make a better person to
start off (1) and (2) than some of the old-hands who are further away
from the newbie mindset. And, on the "talk not followed by effort is
chatter" principle, I surely ought volunteer, too ;-) I'm happy to do
so, provided 1) it doesn't strike people as a silly or hopelessly naive
idea, 2) I'm not committed to providing a proof of concept within the
next week :-) , and 3) I can count on other's to do the wiki thing to my
starting effort.

So, if anyone's still reading, thanks. I'm curious to see if the
suggestion seems worthwhile. Best to all,

Brian vdB

[Y2KYZFR1]

Re: why python is slower than java?

Maurice LING <mauriceling@ac m.org> wrote in message news:<418b27a1$ 1@news.unimelb. edu.au>...[color=blue][color=green]
> > at work, we use java and python. we have projects using swing and
> > others using wxpython. we have applications that do intensive io and
> > others that do intensive cpu. we have not found that python is slower
> > than java. in fact, when it comes to gui's, our swing apps take
> > "forever" to startup and when when garbage collector starts, the whole
> > app just freezes for about 15 seconds. our wxpython apps, start right
> > up and "feel" faster and snappier. can you show an example of where
> > python's "slow" speed as compared to java's "fast" speed has negatively
> > impacted your application or has been noticable in any way? i know this
> > is a trolling question you have posted, but i'm actually very interested
> > knowing why you have said this.
> >
> > thanks,
> >
> > bryan[/color]
>
> Thanks, lets just say that I have no interest in trolling.
>
> 1st of all, I thought it is somehow common knowledge that python is
> slower than java in many cases. Though I may be wrong... When I do a
> Google search, this came up...
>
> http://twistedmatrix.com/users/glyph...n-vs-java.html
>
> although http://page.mi.fu-berlin.de/~prechel.../jccpprtTR.pdf
> showsjust the opposite.
>
> What I need to work on now is something that requires speed (and dealing
> with files), without user's intervention. So the part about users' delay
> time is not in the equation. My choices boils down to Python or Java.
>
> Cheers
> maurice[/color]

dude that "comparisio n" from twistedmatrix you refrence is ANCIENT!!!

it is comparing versions that are YEARS out of date and use!

you are just trolling or your don't know enough to understand the
answer to your question which is way to vague to be answered, as there
is no real correct answer.

[Tim Jarman]

Re: Newsgroup mood [was] why python is slower than java?

In article <mailman.6011.1 099764243.5135. python-list@python.org >,
Brian van den Broek <bvande@po-box.mcgill.ca> wrote:

<snip>
[color=blue]
>
> What I have in mind is a wiki-page or three with links prominently
> located in the welcome message and the reminder message. I could see at
> least three distinct pages being of use. In rapid-sketch mode:
>
> 1) Welcome to the list, here are our norms
> At minimum, a short 1-2 paragraph distillation of "Smart Questions", a
> link to it, a reminder of the existence of the tutor list, and links to
> the other two here mentioned.
>
> 2) How to find information about Python
> Links to one or more of the comp.lang.pytho n archives with perhaps some
> advice for optimal searching. This could include a link to a quick "How
> to google".
> Perhaps a link to Mark Hammond's wicked-cool Python Sidebar for
> Firefox/Mozilla <http://starship.python .net/crew/mhammond/mozilla/> and
> to any other similar aids.
>
> 3) Links to threads that answer common questions
> Yes, there is an FAQ and yet people still ask why Python gets division
> wrong. But I've often seen people dig up links to threads for questions
> perhaps not so common to be written up in an FAQ, but perhaps common
> enough to merit linking on a wiki page. I'm not under the illusion that
> it would solve the problem on its own, but it might well head some
> repeat questions off. For those it doesn't, it would give an easy way to
> answer.
>
> Yes, some (or even all) of this info is already on Python.org. But this
> way would allow it to be audience-tailored and to be ultra-easy to
> direct newbie's toward. And yes, not all newbies read Eric Raymond's
> "Smart Questions" essay within the first 5 times they are pointed at it.
> Clearly it won't be a silver bullet, but maybe it would help.
>
> As a relative newbie myself, I think I might make a better person to
> start off (1) and (2) than some of the old-hands who are further away
> from the newbie mindset. And, on the "talk not followed by effort is
> chatter" principle, I surely ought volunteer, too ;-) I'm happy to do
> so, provided 1) it doesn't strike people as a silly or hopelessly naive
> idea, 2) I'm not committed to providing a proof of concept within the
> next week :-) , and 3) I can count on other's to do the wiki thing to my
> starting effort.
>
> So, if anyone's still reading, thanks. I'm curious to see if the
> suggestion seems worthwhile. Best to all,
>
> Brian vdB[/color]

Sounds good to me - go for it!

[Y2KYZFR1]

Re: why python is slower than java?

Maurice LING <mauriceling@ac m.org> wrote in message news:<mailman.5 992.1099726289. 5135.python-list@python.org >...[color=blue][color=green]
> > Interestingly enough, I sort of share your perception -- and I have
> > noticed the same thing, and seen it remarked upon by others, in other,
> > completely unrelated newsgroups as well, such as it.comp.macinto sh.
> >
> > It's not, I think, about newbies in general: people who come and post
> > help requests, without giving the information that's quite obviously
> > indispensabile to let us help them, keep getting treated with
> > unreasonable amounts of friendliness and courtesy even after many
> > requests for more info go unheeded, for example.
> >
> > However, newbies who are clueless enough to come blasting in with the
> > usual whines we've heard a zillion times -- Macs cost too much, Python
> > is too slow, there's no apps for Macs, Python must absolutely add
> > feature X or it will die, Apple's gonna go broke tomorrow, etc, etc --
> > do appear to get on our collective nerves worse than their essentially
> > indistinguishab le precursors did last year, two years ago, &c.
> >
> > The accusation of being a troll often follows, breaking the old advice
> > to "Never attribute to malice what can be adequately explained by
> > incompetence". Maybe it's a case of the corollary to that old advice,
> > "Sufficient ly advanced cluelessness is indistinguishab le from malice".
> > But why that should be so this year more than last year, say, is not
> > clear to me...
> >
> >
> > Alex[/color]
>
> I've already said the following and was not noticed:
>
> 1. it is a disk intensive I/O operation.
> 2. users delay is not in the equation (there is no user input)
> 3. I am not interested in the amount of time needed to develop it. But
> only interested execution speed.
>
> Thanks maurice[/color]


this is still not enough information to give an answer to. and as for
"disk intensive" all languages handle disk i/o about the same since
they have to rely on the underlying Operating System, its drivers.

[Roger Binns]

Re: why python is slower than java?

Alex Martelli wrote:[color=blue]
> while ((c = in.read()) != -1)
> out.write(c);[/color]

Incidentally, that Java code copies one character at a time.
[color=blue]
> outputFile.writ e(inputFile.rea d())[/color]

The Python code is reading the entire string into memory and
then writing it.

The interpretter overhead vs system calls could be measured
by having the language involved in every byte transferred as
in the Java example, or negligibly as in the Python example.

Whenever anyone has an agenda, you can make all sorts of silly
claims. IMHO the best thing to do is to lead by examples.
For years many people claimed Perl was line noise, hard to
maintain etc. I never really saw much response, since the
Perl people were too busy writing real world code and helping
deliver part of the web revolution. (And selling zillions of
books for O'Reilly :-)

I couldn't find any Python success stories on python.org itself,
but if you dig you can find the stories for the Python Business
Forum as well as Pythonology.

There are however remarkably few where you can go grab the
source code and see how it is all put together. In fact I
couldn't find a single one, but didn't do an exhaustive
search from python.org.

Perhaps it is also worth linking to the projects done in
Python on SourceForge and elsewhere?

http://sf.net/softwaremap/trove_list.php?form_cat=178

Roger

[Fábio Mendes]

Re: Newsgroup mood [was] why python is slower than java?

[...][color=blue]
> As a relative newbie myself, I think I might make a better person to
> start off (1) and (2) than some of the old-hands who are further away
> from the newbie mindset. And, on the "talk not followed by effort is
> chatter" principle, I surely ought volunteer, too ;-) I'm happy to do
> so, provided 1) it doesn't strike people as a silly or hopelessly
> naive
> idea, 2) I'm not committed to providing a proof of concept within the
> next week :-) , and 3) I can count on other's to do the wiki thing to
> my
> starting effort.
>
> So, if anyone's still reading, thanks. I'm curious to see if the
> suggestion seems worthwhile. Best to all,[/color]

This would be very nice in any OOS list!
+1

Fabio

[Israel Raj T]

Re: why python is slower than java?

Kendall Clark <kendall@monkey fist.com> writes:
[color=blue]
> The main reason Java is "faster" is because[/color]

I would reject the premise entirely.

When looking at desktop apps made with Python, they positively whiz when compared to Java
swing apps.

As for non desktop apps, the entire portage system of Gentoo is written in Python.
'emerge sync' causes a python app to synchronise a local application database with database at a non-local mirror.
It is i/o intensive and appears to work very well and very fast.

[Israel Raj T]

Re: Newsgroup mood

Brian van den Broek <bvande@po-box.mcgill.ca> writes:
[color=blue]
> (I recall a post I made to a LUG asking for help with an aborted attempt to get Linux and my laptop's Winmodem to
> co-operate. I was flamed because my mail headers indicated that I was using Thunderbird on *Windows* -- the horror! --[/color]

Or perhaps because you had not read the many available resources on using winmodems under linux.

BTW, if you are looking for a decent mail client check out gnus. Of the over 20 mail clients that I have used over the
years, this is the best.

[Israel Raj T]

Re: why python is slower than java?

"Eric S. Johansson" <esj@harvee.org > writes:
[color=blue]
> I will argue that we might want to start including in the
> headers of mailing list/newsgroups a dana reference.[/color]

I would argue that this would debase newsgroups even further.

[Kent Johnson]

Re: why python is slower than java?

I rarely find myself acting as an apologist for Java, and I understand
that the point Alex is making is that Python's performance for this
operation is quite good, and that the OP should post some code, but this
is really too unfair a comparison for me not to say something.

There are two major differences between these two programs:
- The Java version is doing a character by character copy; the Python
program reads the entire file into a buffer in one operation.
- The Java program is converting the entire file to and from Unicode;
the Python program is copying the literal bytes.

Here is a much more comparable Java program (that will fail if the file
size is over 2^31-1):

import java.io.*;

public class Copy {
public static void main(String[] args) throws IOException {
File inputFile = new File("/usr/share/dict/web2");
int bufferSize = (int)inputFile. length();
File outputFile = new File("/tmp/acopy");

FileInputStream in = new FileInputStream (inputFile);
FileOutputStrea m out = new FileOutputStrea m(outputFile);

byte buffer[] = new byte[bufferSize];
int len=bufferSize;

while (true)
{
len=in.read(buf fer,0,bufferSiz e);
if (len<0 )
break;
out.write(buffe r,0,len);
}

in.close();
out.close();
}
}

Here are the results I get with this program and Alex's Python program
on my G4-400 Mac:
kent% time java Copy
0.440u 0.320s 0:00.96 79.1% 0+0k 9+3io 0pf+0w
kent% time python Copy.py
0.100u 0.120s 0:00.31 70.9% 0+0k 2+4io 0pf+0w

The Python program is still substantially faster (3x), but with nowhere
near the margin Alex saw.

Kent

Alex Martelli wrote:[color=blue]
> OK, could you provide a simple toy example that meets these conditions
> -- does lot of identical disk-intensive I/O "in batch" -- and the
> execution speed measured (and on what platform) for what Python and Java
> implementations , please?
>
> For example, taking a trivial Copy.java from somewhere on the net:
>
> import java.io.*;
>
> public class Copy {
> public static void main(String[] args) throws IOException {
> File inputFile = new File("/usr/share/dict/web2");
> File outputFile = new File("/tmp/acopy");
>
> FileReader in = new FileReader(inpu tFile);
> FileWriter out = new FileWriter(outp utFile);
> int c;
>
> while ((c = in.read()) != -1)
> out.write(c);
>
> in.close();
> out.close();
> }
> }
>
> and I observe (on an iBook 800, MacOSX 10.3.5):
>
> kallisti:~ alex$ java -version
> java version "1.4.2_05"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_05-141.3)
> Java HotSpot(TM) Client VM (build 1.4.2-38, mixed mode)
>
> -r--r--r-- 1 root wheel 2486825 12 Sep 2003 /usr/share/dict/web2
>
> kallisti:~ alex$ time java Copy
>
> real 0m7.058s
> user 0m5.820s
> sys 0m0.390s
>
> versus:
>
> kallisti:~ alex$ time python2.4 Copy.py
>
> real 0m0.296s
> user 0m0.080s
> sys 0m0.170s
>
> with Python 2.4 beta 1 for the roughly equivalent:
>
> inputFile = file("/usr/share/dict/web2", 'r')
> outputFile = file("/tmp/acopy", 'w')
>
> outputFile.writ e(inputFile.rea d())
>
> inputFile.close ()
> outputFile.clos e()
>
> which isn't all that far from highly optimized system commands:
>
> kallisti:~ alex$ time cp /usr/share/dict/web2 /tmp/acopy
>
> real 0m0.167s
> user 0m0.000s
> sys 0m0.040s
>
> kallisti:~ alex$ time cat /usr/share/dict/web2 >/tmp/acopy
>
> real 0m0.149s
> user 0m0.000s
> sys 0m0.090s
>
>
> I'm sure the Java version can be optimized easily, too -- I just grabbed
> the first thing I saw off the net. But surely this example doesn't
> point to any big performance issue with Python disk I/O wrt Java. So,
> unless you post concrete examples yourself, the smallest the better,
> it's going to be pretty difficult to understand where your doubts are
> coming from!
>
>
> Alex
>[/color]

[Alex Martelli]

Re: why python is slower than java?

Roger Binns <rogerb@rogerbi nns.com> wrote:
[color=blue]
> Alex Martelli wrote:[color=green]
> > while ((c = in.read()) != -1)
> > out.write(c);[/color]
>
> Incidentally, that Java code copies one character at a time.[/color]

Yep, I noticed -- pretty silly, but if that's how the Java designers
decided the read method should behave by default, who am I to argue?
Just an example I grabbed off the net, first google hit for (if I recall
correctly) java file reading that had the source of a complete example.
[color=blue][color=green]
> > outputFile.writ e(inputFile.rea d())[/color]
>
> The Python code is reading the entire string into memory and
> then writing it.[/color]

Right, _Python_'s default.
[color=blue]
> The interpretter overhead vs system calls could be measured
> by having the language involved in every byte transferred as
> in the Java example, or negligibly as in the Python example.[/color]

The claim posted to this newsgroup, without any support nor examples
being given, was that Python's I/O was far slower than Java's in
_disk-intensive_ operations. I'm still waiting to see any small,
verifiable examples of that posted on this thead.

Apparently, at least in default operations on both sides, based on calls
to read without parameters, that is definitely not the case - no doubt,
as you say, that's because of the different ways those defaults are
tuned, making Python much faster. Great, then let those who claim
Java's I/O is much faster in disk intensive operation post suitable
examples, and we'll see.

[color=blue]
> Whenever anyone has an agenda, you can make all sorts of silly
> claims. IMHO the best thing to do is to lead by examples.[/color]

I posted examples -- probably not "leading" ones, just one grabbed off
the net using Java's defaults, and one using Python's defaults. I'm not
the one making any claims regarding I/O performance of _properly tuned_
disk-intensive programs -- what I would like to see would be some such
examples posted by those who DO make such claims. Shouldn't the burden
of proof lay on the side making positive assertions, by ordinary rules?!
[color=blue]
> For years many people claimed Perl was line noise, hard to
> maintain etc. I never really saw much response, since the
> Perl people were too busy writing real world code and helping
> deliver part of the web revolution. (And selling zillions of
> books for O'Reilly :-)[/color]

Surely more than we have sold so far with Python, yes - Tim O'Reilly has
published data about that. Which is why I'm striving to _worsen_ the
quality of Python's free docs until they match Perl's, against heathens
who (can you imagine that!) are instead trying to *improve* them further
(thus no doubt hurting book sales).

[color=blue]
> I couldn't find any Python success stories on python.org itself,
> but if you dig you can find the stories for the Python Business
> Forum as well as Pythonology.[/color]

Lots of digging, yes, considering that if you google for python success
stories those pbf and pythonology hits are just at the top (other
similar links, such as those to O'Reilly's booklets covering the same
material and their PDF forms, make up the rest of the top-ten pages).
[color=blue]
> There are however remarkably few where you can go grab the
> source code and see how it is all put together. In fact I
> couldn't find a single one, but didn't do an exhaustive
> search from python.org.[/color]

Without straying from the top google hit, you can find that some of the
listed success stories are open-source, such as mayavi -- they're
admittedly a minority of the "success stories" listed on these sites.
[color=blue]
> Perhaps it is also worth linking to the projects done in
> Python on SourceForge and elsewhere?
>
> http://sf.net/softwaremap/trove_list.php?form_cat=178[/color]

If the message you're keen to send is "Python is great for open-source",
yes. If you're focusing on "Python is great for your _business_" (as
the python *business* forum does, for example), then emphasizing
open-source projects can reasonably be considered secondary to
emphasizing projects that make or save money for the businesses which
developed them.

If you're looking for highly visible open-source projects using Python,
I'd think that bittorrent, chandler, zope and plone, mayavi, all the
scipy.org site, twisted, schooltool, ubuntu, and the like, should keep
you in reading materials for a while. Not sure how many of these
projects are specifically on sourceforge, but I believe some minority
probably is.


Alex

[Jeremy Bowers]

Re: why python is slower than java?

On Sat, 06 Nov 2004 20:53:26 +0000, Israel Raj T wrote:[color=blue]
> As for non desktop apps, the entire portage system of Gentoo is written in
> Python. 'emerge sync' causes a python app to synchronise a local
> application database with database at a non-local mirror. It is i/o
> intensive and appears to work very well and very fast.[/color]

"emerge sync" mostly runs an rsync command (at least on standard
installs). The python contribution to that process is minimal.

The primary slowdown with "emerge" appears to be the way the database is
stored as many thousands of little files in /usr/portage. Again, not
really python; you want to accelerate emerge, speed up your hard drive.

(They have more cached metadata now at least in the cutting-edge release,
but as that is the one I use I don't know if that is now considered
stable. It has really sped things up. A couple more iterations and Python
might actually become the bottleneck.)

[Brian van den Broek]

Re: Newsgroup mood

Israel Raj T said unto the world upon 2004-11-06 16:01:[color=blue]
> Brian van den Broek <bvande@po-box.mcgill.ca> writes:
>
>[color=green]
>> (I recall a post I made to a LUG asking for help with an aborted
>> attempt to get Linux and my laptop's Winmodem to co-operate. I was
>> flamed because my mail headers indicated that I was using
>> Thunderbird on *Windows* -- the horror! --[/color]
>
>
> Or perhaps because you had not read the many available resources on
> using winmodems under linux.
>
> BTW, if you are looking for a decent mail client check out gnus. Of
> the over 20 mail clients that I have used over the years, this is the
> best.[/color]

Perhaps. But the context was that I was a Linux neophyte (not much
beyond that now), identified myself as such, made sure to manifest that
I'd tried and failed to sort it out for myself, and did my best to
appear to be asking for help learning to fish, rather than to be handed
a cooked trout. :-) (As in, I'd read and tried to follow the advice of
Raymond's essay.)

I'm certain that the sort of care I put into asking the question made it
fall outside the scope of the comments about newbie questions made
up-thread. I've yet to see a Python list so react to someone who made it
clear they were trying to follow the norms, even if they didn't meet
with complete success. I think the issue is about those who don't try to
follow courteous practise, and perhaps aren't aware they aren't.

Any project, Linux, Python, whatever, that aims to get a user base
beyond the gurus has to accept that not everyone who needs help will be
able to work everything out based on what they can find on-line. (It is
clear to me that Python lists *do* accept this.) The problem isn't with
those who get stuck after trying for themselves; it those who hit send
before search :-)

But thanks for the comment, and the client pointer. Best,

Brian vdB

[Alex Martelli]

Re: why python is slower than java?

Kent Johnson <kent3737@yahoo .com> wrote:
[color=blue]
> I rarely find myself acting as an apologist for Java, and I understand
> that the point Alex is making is that Python's performance for this
> operation is quite good, and that the OP should post some code, but this
> is really too unfair a comparison for me not to say something.[/color]

I'm glad I posted a sufficiently silly comparison to elicit some
response, then;-)

[color=blue]
> There are two major differences between these two programs:
> - The Java version is doing a character by character copy; the Python
> program reads the entire file into a buffer in one operation.
> - The Java program is converting the entire file to and from Unicode;
> the Python program is copying the literal bytes.[/color]

Right: each is using the respective language's defaults, and Python's
are apparently tuned for speed, while Java's apparenty aren't.

[color=blue]
> Here is a much more comparable Java program (that will fail if the file
> size is over 2^31-1):[/color]

I believe that the Python program, if run on a suitable 64-bit OS, on a
ridiculously-large-memory machine, could succeed for much larger files
than that. Machines with many gigabytes of physical RAM are becoming
far from absurd these days -- I can't yet afford to throw 2500 euros out
of the window to buy 8 GB of fast RAM, but if I could it would fit
snugly into my own cheap dual-G5 powermac, for example (old model: I got
it used/reconditioned many months ago; I think the current cheaper model
does top out at 4 GB). On 32-bit machines or OSs, of course, Python's
memory limits _will_ byte at fewer GB than that, too.

[color=blue]
> import java.io.*;
>
> public class Copy {
> public static void main(String[] args) throws IOException {
> File inputFile = new File("/usr/share/dict/web2");
> int bufferSize = (int)inputFile. length();
> File outputFile = new File("/tmp/acopy");
>
> FileInputStream in = new FileInputStream (inputFile);
> FileOutputStrea m out = new FileOutputStrea m(outputFile);
>
> byte buffer[] = new byte[bufferSize];
> int len=bufferSize;
>
> while (true)
> {
> len=in.read(buf fer,0,bufferSiz e);
> if (len<0 )
> break;
> out.write(buffe r,0,len);
> }
>
> in.close();
> out.close();
> }
> }
>
> Here are the results I get with this program and Alex's Python program
> on my G4-400 Mac:
> kent% time java Copy
> 0.440u 0.320s 0:00.96 79.1% 0+0k 9+3io 0pf+0w
> kent% time python Copy.py
> 0.100u 0.120s 0:00.31 70.9% 0+0k 2+4io 0pf+0w[/color]

With your program, and mine simply converted to run inside a main()
function rather than at module-level for comparison, switching to tcsh
for direct comparison with the format you're using, I see:

[kallisti:~] alex% time java Copy
0.200u 0.140s 0:00.54 62.9% 0+0k 0+0io 0pf+0w
[kallisti:~] alex% time python Copy.py
0.080u 0.020s 0:00.13 76.9% 0+0k 0+1io 0pf+0w

Which python and java versions are you using? I'm trying to use the
latest and greatest of each, 2.4b1 (I know, I know, I need to install
b2!) for Python, 1.4.2_05 for Java -- just upgraded to MacOSX 10.3.6,
and my /usr/share/dict/web2 is 2486825 bytes.
[color=blue]
> The Python program is still substantially faster (3x), but with nowhere
> near the margin Alex saw.[/color]

I still see a 4:1 ratio, but, sure, nowhere like the 20:1 my originally
silly example showed. Maybe a more realistic program would use a buffer
of some fixed length, rather than 'as long as the whole file'. Say:

int bufferSize = 64 * 1024;

in a program that otherwise is just like yours, for the Java side of
things. Switching back to bash because I can't stand long exposures to
anything in the csh family;-), I see:

kallisti:~ alex$ time java Copy

real 0m0.521s
user 0m0.200s
sys 0m0.120s

after several runs, so everything gets a chance to go to cache. Oh,
btw, I did compile with 'javac -O Copy.java'.

The closest Python equivalent I know how to write is:

def main():
inputFile = file("/usr/share/dict/web2", 'r')
bufferSize = 64 * 1024
outputFile = file("/tmp/acopy", 'w')

inf = inputFile.read
ouf = outputFile.writ e

while 1:
buf = inf(bufferSize)
if not buf: break
ouf(buf)

inputFile.close ()
outputFile.clos e()

and its performance:

kallisti:~ alex$ time python -O Copy.py

real 0m0.135s
user 0m0.050s
sys 0m0.050s

so we still see the 4:1 ratio in favour of Python. It's barely more
than 3:1 in actual CPU time, user and sys, but for some reason Python
seems to be able to get more of the CPU's % attention -- I don't claim
to understand this!-)

So I scp'd everything over to the powermac dual g5 1.8 GHz, and ssh'd
there to do more measurements -- no python 2.4 there (it's my production
machine -> no betas!) and again I measured after a few runs to let stuff
get into cache:

macarthur:~ alex$ time java Copy

real 0m0.163s
user 0m0.040s
sys 0m0.110s

macarthur:~ alex$ time python -O Copy.py

real 0m0.039s
user 0m0.020s
sys 0m0.020s

Far better real/cpu ratios of course (a dual-CPU machine does that;-).
And faster overall. But roughly the same 4:1 ratio in favour of Python.


So, same thing on the oldie but goodie Linux box. In that case I don't
have an updated Java -- Kaffe 1.0.7 will have to do! The CPU is oldish
(Athlon 1.2G), but the disk subsystem is really really good. So I got
(again, good repeatable results after a few runs to stabilize):

[alex@lancelot alex]$ time java Copy
0.07user 0.04system 0:00.11elapsed 98%CPU (0avgtext+0avgd ata
0maxresident)k
0inputs+0output s (1623major+659m inor)pagefaults 0swaps

[alex@lancelot alex]$ time python2.4 -O Copy.py
0.02user 0.01system 0:00.02elapsed 115%CPU (0avgtext+0avgd ata
0maxresident)k
0inputs+0output s (460major+241mi nor)pagefaults 0swaps

don't ask ME how python got over 100% CPU on a single-CPU machine; I
guess the task is just too small, at such tiny fractions of a second, to
avoid anomalies. Still, the roughly 4:1 performance ratio appears to be
repeatable here; and an interesting clue is that _pagefaults_ also
appear to be in roughly 4:1 ratio.

But clearly we need bigger tasks to make measurements less fraught with
anomalies. Unfortunately, not knowing on exactly which tasks the OP had
observed and measured his reported results that "Python is slower than
Java" in "a disk intensive I/O operation", it's hard to guess.
Obviously, his results are anything but _trivial_ to reproduce based on
the dearth of information that he has released about them so far; I find
it hard to think of something more disk-intensive than a simple copy of
one file to another. Assuming he does have some interest in
ascertaining the truth and getting his questions answered, perhaps he'll
deign to give us Java and Python sources, and reference data file[s],
for a small benchmark that _does_ reproduce his results. Otherwise, it
appears we may be just shooting in the dark.


Alex