Python Productivity Gain?

**William Park** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

kbass <kbass@midsouth .rr.com> wrote:[color=blue]
> In different articles that I have read, persons have constantly eluded to
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases. The
> strange thing that I have noticed is that there were no examples of this
> productivity gain (i.e., projects, programs, etc.,...). Can someone give me
> some real life examples of productivity gains using Python as opposed other
> programming languages.
>
> From my our personal experience, I have been programming with Python for
> about 6 months (but I have been programming in other languages for over 10
> years) and I have noticed that the more I had gotten use to programming in
> Python, the more my programming speed has increased. But ... this is true
> with any language that you program in as long as you are learning the
> methodologies and concepts of the programming language. Your thoughts.[/color]

It used to be that Python programs were shorter, faster, readable,
writable, and simply better. But, this was during the days when most
programmers had Unix background. Nowdays, most of the programmers are
coming from Windows background, and Python programs have become as
verbose and unreadable as Visual Basic or Perl.

Ruby has not been corrupted as such. It make complicated thing less
complicated. But, it still make simply thing not as simple as Python.

--
William Park, Open Geometry Consulting, <opengeometry@y ahoo.ca>
Linux solution for data management and processing.

**kbass** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

"William Park" <opengeometry@y ahoo.ca> wrote in message
news:c0mvpc$18u rh0$1@ID-99293.news.uni-berlin.de...[color=blue]
> kbass <kbass@midsouth .rr.com> wrote:[color=green]
> > In different articles that I have read, persons have constantly eluded[/color][/color]
to[color=blue][color=green]
> > the productivity gains of Python. One person stated that Python's
> > productivity gain was 5 to 10 times over Java in some in some cases. The
> > strange thing that I have noticed is that there were no examples of this
> > productivity gain (i.e., projects, programs, etc.,...). Can someone[/color][/color]
give me[color=blue][color=green]
> > some real life examples of productivity gains using Python as opposed[/color][/color]
other[color=blue][color=green]
> > programming languages.
> >
> > From my our personal experience, I have been programming with Python for
> > about 6 months (but I have been programming in other languages for over[/color][/color]
10[color=blue][color=green]
> > years) and I have noticed that the more I had gotten use to programming[/color][/color]
in[color=blue][color=green]
> > Python, the more my programming speed has increased. But ... this is[/color][/color]
true[color=blue][color=green]
> > with any language that you program in as long as you are learning the
> > methodologies and concepts of the programming language. Your thoughts.[/color]
>
> It used to be that Python programs were shorter, faster, readable,
> writable, and simply better. But, this was during the days when most
> programmers had Unix background. Nowdays, most of the programmers are
> coming from Windows background, and Python programs have become as
> verbose and unreadable as Visual Basic or Perl.
>
> Ruby has not been corrupted as such. It make complicated thing less
> complicated. But, it still make simply thing not as simple as Python.
>
> --
> William Park, Open Geometry Consulting, <opengeometry@y ahoo.ca>
> Linux solution for data management and processing.[/color]

IMHO, the overall productivity gain of any programming language comes from
the programmers that are programming in the language. Productivity gain
comes from individuals and is not language based.

I thought that Python didn't allow for sloppy programming techniques due to
it's lines of code being indented throughout programs. Have Windows
programmers found a way pass this? Are you saying that some programmers with
a Windows programming background have poor programming practices when
compared to their Unix programming background counterparts?

Kevin

**RobE** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

> In different articles that I have read, persons have constantly eluded to[color=blue]
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases.[/color]

I have no data on this and can't say with respect to Java. I can give
my personal feeling and this is more compared to C.

I'd look at productivity in a few ways:

* How many lines of code does it take. More code lines really does
mean longer to program and more chance of errors.
* How hard it is to write code -- to be specific java and C have all
these brackets which have to be tracked down... this does take time...
though perhaps a good IDE handles that for you.
* Complexity of errors possible -- for example in C (but maybe not so
much in Java)-- un-initialized variables and memory leaks, etc are big
problems. Often very hard to find. You often have to use complex
debugging tools to test this sort of code an look for memory corruption.

My experience when writing an application is that all of these things
are favorable for python. You generate code fast, the syntax is such
that you don't have to track down missing components like brackets, and
when you write the code there are usually very few runtime errors --
except for logic errors -- which are usually obvious.

In terms of free/open environments python is also one of the best.

I do agree with some of the other posters -- actual productivity will
depend on skill, and the power of the tools you use to build the app. C
and Java may have better commercial development tools and programs may
be better at using these.

All in all though, I'm a python guy for many reasons.

Rob

P.S. Sorry if you got more than one post--I'm having trouble with my
news client.

**simo** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

I think the main gain is from the lack of the compilation/linking
process.

I have C++/Java pogramming friends (I've dabbled too) and the biggest
hinderance to getting an app up and running is the "change code ->
compile+link (read the newspaper) -> test, doesn't work, change code
-> compile+link (make some coffee) -> test" cycle

Being able to save+run is great with scripting languages. This is why
a lot of C++/Qt programmers I know are switching to Python+PyQt or
Qt+QSA, as in these 3+GHz days, we don't really need the extra speed
from compilation, but saving a few hours of expensive programmer's
time is useful.

Other niceties are on-the-fly debugging/errors - Python and Perl have
this, PHP is useless at debugging, C/C++ just core dumps....

A prime example was the other day at work - I wrote a console utility,
then wrapped a TKinter GUI around it, which took about 10 minutes to
do. I then decided to add a feature which needed another widget that
TKinter doesn't have, so converted it to PyQt, I then found that a
colleauge wanted a copy for Windows, but we didn't have a commercial
license, so converted it to wxPython, all of which took under an hour,
I didn't even use QtDesigner/wxDesigner for my forms.

My C++ programming friend said that would have taken him most of a
morning to do, and then he'd have to build a separate copy for each of
Solaris, Windows and Linux.

**Josiah Carlson** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

> In different articles that I have read, persons have constantly eluded to[color=blue]
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases. The
> strange thing that I have noticed is that there were no examples of this
> productivity gain (i.e., projects, programs, etc.,...). Can someone give me
> some real life examples of productivity gains using Python as opposed other
> programming languages.
>
> From my our personal experience, I have been programming with Python for
> about 6 months (but I have been programming in other languages for over 10
> years) and I have noticed that the more I had gotten use to programming in
> Python, the more my programming speed has increased. But ... this is true
> with any language that you program in as long as you are learning the
> methodologies and concepts of the programming language. Your thoughts.[/color]

Before I give my opinions, I should probably give some background. The
first semester of my freshman year of college, I (and many others) were
taught Scheme. After 'mastering' Scheme, we moved on to C and C++ the
following spring. Within weeks of learning C (the semester started
~February 1, competition was March 19), I competed in a local
programming competition to decide who was going to participate in the
following fall' (of 1999) regional ACM programming competition. My
partner and I placed second in the local competition, ahead of various
sophomores and juniors, but didn't do quite so well in the regional ACM
competition.

A year later, in the early spring of 2000 (sophomore year), I found
Python. In a week, I rewrote every programming assignment I had during
the previous year and a half of undergraduate CS, from Scheme or C/C++
to Python.

Not only had I learned Python, and translated programs to Python, but I
was able to write new software that I hadn't even considered before.
Multithreading, sockets, interesting data structures, databases...the
world was my oyster. Two years later, in the spring of 2002, I was
finishing the 4th rewrite of an as-of-yet unreleased (the quality was
shit, I'm still looking for time to re-do it) parallel processing
library, similar to MPI. The 4th rewrite took a total of a week and a
half, for 4500 lines of Python. Functionally, every feature of MPI was
included, written in ~10 days. I had known Python for 2 years.

Fast forward to now, I've known Python for 4 years. The only project
I've written in C since, is a password cracker (modified l0phtcrack),
that I had contemplated making into an independent project during
undergrad. Every other piece of code that I produce on a daily basis,
for teaching Introductory Algorithm Design and Analysis (this quarter is
my 5th as a TA for the course), for database classes, algorithmic
theory, etc., or any time I need some tool, Python is what I build it with.

They say that when you have a hammer, everything looks like a nail.
When wsing Python, most everything /is/ a nail. Those things that are
not nails, usually lie in the realm of different programming paradigms
(like logic programming with Prolog, etc.), or are very processor
intensive, and are not suitable for an interpreted language. Psyco
works well for a first step to stave off "not fast enough", and I hear
that Pyrex is a great second step (I've had no need for Pyrex yet).

In any case, I don't believe that I would have been able to do or learn
nearly as much, had I not had Python.

- Josiah

**Shalabh Chaturvedi** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

"kbass" <kbass@midsouth .rr.com> wrote in message news:<LaBXb.877 4$g95.1612@fe3. columbus.rr.com >...[color=blue]
> In different articles that I have read, persons have constantly eluded to
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases. The
> strange thing that I have noticed is that there were no examples of this
> productivity gain (i.e., projects, programs, etc.,...). Can someone give me
> some real life examples of productivity gains using Python as opposed other
> programming languages.[/color]

There is lot that can be said about why Python is more productive.
I'll make one point that has not yet been mentioned in this thread.

I like to say that Python has a very good impedance match with the
mind (mine at least :). Normally when you program, you:

1. Analyze the problem
2. Come up with a 'solution in mind'
3. Translate the 'solution in mind' into 'solution in code'

It's 3 where Python shines. For a concrete example, consider looping
over a list of items. Apart from the looping object and the list, you
need:

Java: An iterator
C: A counter
Perl: A lot of '$' signs :)
Python: Nothing. (for item in mylist:)

Notice how Python keeps it explicit yet minimal, and there's nothing
more in your mind than what is absolutely necessary ("fits you
brain"). As a real life analogy, when I make a multi-egg omelette, for
each egg in bunch_of_eggs, I pan.put(egg.bre ak()). If I had to think
of an iterator, or a counter, it would be lunchtime before I'd have
breakfast.

In other words, Python *is* how I think. This is what I mean by a good
impedance match.

Note that you can still use an iterator in Python (but only if and
when you need it).

For another example, consider nested lists in Perl and Python. In Perl
one struggles with the language, in Python one struggles with only the
problem.

In other languages I've used (Java, C++, Perl), I always get
sidetracked into syntax issues ("oh I have to write it *this* way!"),
library issues ("oh I have to import this and that first!") and such,
all of which consume my brainpower leaving little for the problem at
hand. These little things add up and affect productivity tremendously.
Python has few of these and I've always found myself rushing to Python
as the first choice to implement any algorithm, solution, or idea that
I had. It is the shortest path to the program.

Some newcomers are so used to thinking in roundabout ways that they
write C or Java code in Python. I did too - using counters for loops,
creating way too many classes etc. But eventually everyone learns the
power of simplicity.

Naturally, reading a Python program and figuring out what it does is
fast too, greatly improving maintainability .

I'm writing an article on Python productivity, which I'll post here at
some point.

Cheers,
Shalabh

**Matthias** · Jul 18 '05, 08:27 AM

Re: Python Productivity Gain?

"kbass" <kbass@midsouth .rr.com> writes:
[color=blue]
> In different articles that I have read, persons have constantly eluded to
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases. The
> strange thing that I have noticed is that there were no examples of this
> productivity gain (i.e., projects, programs, etc.,...). Can someone give me
> some real life examples of productivity gains using Python as opposed other
> programming languages.[/color]

In every language community you will find people claiming that their
favorite language X is 5-10 times more productive than alternatives.
Typically this is justified by personal anecdotes.

A scientific study in this regard is

@misc{ prechelt-empirical,
author = "Lutz Prechelt",
title = "An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program",
url = "citeseer.nj.ne c.com/547865.html" }

This study reports a programmer productivity gain of about 2 for all
the scripting languages over C, C++, and Java. It is a pity that not
more attempts to gain /unbiased/ information on software productivity
are undertaken.

**Thomas Guettler** · Jul 18 '05, 08:28 AM

Re: Python Productivity Gain?

Am Sun, 15 Feb 2004 02:53:31 +0000 schrieb kbass:
[color=blue]
> In different articles that I have read, persons have constantly eluded to
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases. The
> strange thing that I have noticed is that there were no examples of this
> productivity gain (i.e., projects, programs, etc.,...). Can someone give me
> some real life examples of productivity gains using Python as opposed other
> programming languages.[/color]

I think programming python is more productive since the
code is much more easy to read. You can solve the same
problem with less characters in perl, but after one year,
you will prefere the python version.

http://pleac.sf.net has some examples on several languages.

thomas

**Peter Hansen** · Jul 18 '05, 08:28 AM

Re: Python Productivity Gain?

William Park wrote:[color=blue]
>
> It used to be that Python programs were shorter, faster, readable,
> writable, and simply better. But, this was during the days when most
> programmers had Unix background. Nowdays, most of the programmers are
> coming from Windows background, and Python programs have become as
> verbose and unreadable as Visual Basic or Perl.
>
> Ruby has not been corrupted as such. It make complicated thing less
> complicated. But, it still make simply thing not as simple as Python.[/color]

With all respect, William, this sounds to me like a big stinking pile
of rubbish. Do you have *any* factual basis for the claims you are
making?

In my opinion, the Unix background of a programmer would, if anything,
tend to increase the likelihood of the code being unreadable (Perl, after
all, comes from that neck of the woods), but I really think it much
more likely that the OS background has little to do with it. Python
itself is what is different, and what produces more readable code,
and the Python community encourages and directs that trend.

As for "most programmers are coming from Windows background" I suspect
that too is very debatable.

-Peter

**Peter Hansen** · Jul 18 '05, 08:28 AM

Re: Python Productivity Gain?

Matthias wrote:[color=blue]
>
> This study reports a programmer productivity gain of about 2 for all
> the scripting languages over C, C++, and Java. It is a pity that not
> more attempts to gain /unbiased/ information on software productivity
> are undertaken.[/color]

I agree. Unfortunately, it appears, the cost of doing that is just too
high. Several times I've wanted to do a study in my company to measure
the claimed productivity improvements of Python (and of other things) but
in the end without funding it's not likely to happen, or to be credible.

-Peter

**Harry George** · Jul 18 '05, 08:28 AM

Re: Python Productivity Gain?

"kbass" <kbass@midsouth .rr.com> writes:
[color=blue]
> In different articles that I have read, persons have constantly eluded to
> the productivity gains of Python. One person stated that Python's
> productivity gain was 5 to 10 times over Java in some in some cases. The
> strange thing that I have noticed is that there were no examples of this
> productivity gain (i.e., projects, programs, etc.,...). Can someone give me
> some real life examples of productivity gains using Python as opposed other
> programming languages.
>
> From my our personal experience, I have been programming with Python for
> about 6 months (but I have been programming in other languages for over 10
> years) and I have noticed that the more I had gotten use to programming in
> Python, the more my programming speed has increased. But ... this is true
> with any language that you program in as long as you are learning the
> methodologies and concepts of the programming language. Your thoughts.
>
> Kevin
>
>[/color]

My experience is with in-hoiuse code, so I can't show examples, but I
can give impressions. We had Java code which did XML, matrix
manipulation, and lisp s-expression reader/writer. We converted that
to Python. It took roughly 1/3 less code (measured with wc -l), the
code was easier to read (consensus in code reviews), and it has proven
more maintainable (roughly 1/2 flowtime for similar enhancements).

The algorithms didn't change, so this wasn't just a matter of learning
the problem space better ("write one to throw away"). My impression
of the productivity improvements:

a) No compilation step. If you program in a "change one feature and
then run unittests" style, the edit-run cycle is critical. Python was
several seconds (> 15, < 60) faster per cycle. We may have had an
exceptionally slow java setup, but I've seen similar effects on other
platforms.

b) Clarity of thought. We could see the forest and not just the
trees. This led to refactoring, which simplified maintenance.

As for learning the language vs knowing others: We've now had a dozen
compsci people learn python. Each had considerable experience with
other languages. After a couple of weeks, each person says things
like "Wow, this is amazing." In code reviews, they learn better
idioms, but the basic impression of productivity is there from about 2
weeks after they start.

My biggest problem is with people who are not willing to refactor to
clean up working code. Without that step, some of the potential
improvement from python is lost. I've looked at their non-python code
and see this is apparently a personal programming trait. People who
write clean code in python also do so in VB, java, COBOL, etc. I
think python makes refactoring easy enough that people who care about
clean code are very impressed with the language.

--
harry.g.george@ boeing.com
6-6M21 BCA CompArch Design Engineering
Phone: (425) 342-0007

**Matthias** · Jul 18 '05, 08:29 AM

Re: Python Productivity Gain?

Peter Hansen <peter@engcorp. com> writes:
[color=blue]
> Matthias wrote:[color=green]
> >
> > This study reports a programmer productivity gain of about 2 for all
> > the scripting languages over C, C++, and Java. It is a pity that not
> > more attempts to gain /unbiased/ information on software productivity
> > are undertaken.[/color]
>
> I agree. Unfortunately, it appears, the cost of doing that is just too
> high. Several times I've wanted to do a study in my company to measure
> the claimed productivity improvements of Python (and of other things) but
> in the end without funding it's not likely to happen, or to be credible.[/color]

I can see that it's hard to get company funding for such research.
But then: Software is a multi-billion dollar business, languages are a
really fundamental tool for it (if you get a 5% increase in
productivity by adding/removing a feature X from language Y this can
be huge savings overall). Yet nobody seems to be bothered that the
evolution of computer languages goes like: Somebody has a cute idea,
builds a language around it, tries to hype it, maybe attracts
followers, maybe creates a market which then attracts more followers.
It's all trial and error, like medicine in the middle ages.

A more scientific approach would be: Take a language X, build variants
X-with-OOP, X-with-static-typing, X-with-funny-syntax and let
developers use it under controlled settings. Watch them. Generate
bug statistics. Look for differences. Try to explain them. This
would be hard work, difficult to do and expensive. But I expect this
approach would find better [1] languages faster. The benefits might
be substantial.

Matthias

---
[1] At least better w.r.t. certain application domains and certain
types of developers.

**Harry George** · Jul 18 '05, 08:29 AM

Re: Python Productivity Gain?

Matthias <no@spam.pls> writes:

[snip][color=blue]
> I can see that it's hard to get company funding for such research.
> But then: Software is a multi-billion dollar business, languages are a
> really fundamental tool for it (if you get a 5% increase in
> productivity by adding/removing a feature X from language Y this can
> be huge savings overall). Yet nobody seems to be bothered that the
> evolution of computer languages goes like: Somebody has a cute idea,
> builds a language around it, tries to hype it, maybe attracts
> followers, maybe creates a market which then attracts more followers.
> It's all trial and error, like medicine in the middle ages.
>
> A more scientific approach would be: Take a language X, build variants
> X-with-OOP, X-with-static-typing, X-with-funny-syntax and let
> developers use it under controlled settings. Watch them. Generate
> bug statistics. Look for differences. Try to explain them. This
> would be hard work, difficult to do and expensive. But I expect this
> approach would find better [1] languages faster. The benefits might
> be substantial.
>
> Matthias
>
> ---
> [1] At least better w.r.t. certain application domains and certain
> types of developers.[/color]

The scientific approach requires usefully discriminatory hypotheses.
Language design is so art-ful that arbitrarily building languages as
you describe does not meet that criterion.

Here is another way to look at it.

Normally a science passes through phases:

a) Natural History. Wander around, get the lay of the land,
collect specimens, and try to organize what you find into mnemonically
effective schemes.

b) Field Research. Pose a hypothesis, isolate a piece of the field as
best you can, and apply your experimental factors and controls.
Observe results and interpret with a large grain of salt.

c) Lab Research. Set up isolated envieonments with significant
attention to eliminating non-experimental reasons for variation. Pose
the hypotheses. Observe results, and interpret with recognition that
a lab may be a poor model for reality.

You are asking that we jump to lab research when the field barely
sustains field research. Mostly we are still in natural history and
anecdotes.

Of course, even in the natural history phase pioneers and advance
scouts are capable of detecting an easier pass through the mountains
of comlexity. If 20 people from varied background, each of whom has
worked in several languages, tell me that Python is a really great
language, then I'll take that as a significant data point. Especially
if they are dumping their previously favorite languages (as varied as
COBOL, Perl, Java, C++, VB, Modula-3, Lisp, Prolog) to focus on
Python.

--
harry.g.george@ boeing.com
6-6M21 BCA CompArch Design Engineering
Phone: (425) 342-0007

**Peter Hansen** · Jul 18 '05, 08:29 AM

Re: Python Productivity Gain?

Harry George wrote in a thought-provoking post:[color=blue]
>
> Of course, even in the natural history phase pioneers and advance
> scouts are capable of detecting an easier pass through the mountains
> of comlexity. If 20 people from varied background, each of whom has
> worked in several languages, tell me that Python is a really great
> language, then I'll take that as a significant data point. Especially
> if they are dumping their previously favorite languages (as varied as
> COBOL, Perl, Java, C++, VB, Modula-3, Lisp, Prolog) to focus on
> Python.[/color]

My background is (roughly in order) APL, FORTRAN, BASIC, Assembly, C,
university :-), Pascal, C++, Object Pascal, Java, LabVIEW, and Python
(with a dozen others I forget) and I'm telling you Python is a really
great language. I've also dumped my previously favourite languages
(to wit, BASIC, C, C++, Delphi, and Java) to focus on Python.

Now all you need are 19 others and we'll have a significant data point.
(Signifying what? That's what I want to know. ;-)

-Peter

Python Productivity Gain?

Python Productivity Gain?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment