fairly large webapp: from Java to Python. experiences?

[john_sips_tea@yahoo.com]

I've got a fairly substantial webapp written in Java (plus Tomcat,
Hibernate, Struts, JSP, MySQL) that is a bit of a bear to work with. I
didn't write it. Much of it is only very sparsely documented (if at
all). No design docs anywhere. It's a large webapp with many classes
and fairly deep inheritance hierarchies. There seems to be JUnit test
cases in place.

The possibility of re-writing it has come up, and I must say that,
although it would likely be a ton of work, it could also be a real
pythonic adventure. ;) Reasons for a rewrite are to get better
flexibility, easier maintenance (easier to make changes), and the
possibility of really slimming down the size of the thing. And of
course, it would be nice to work with Python -- possibly significantly
more productive.

Does anyone here have any experience with large(ish) webapps in Python?
I was sniffing around the following A-team of toolkits: CherryPy,
Cheetah, SQLObject, all behind Apache2. Has anyone seen performance
issues with Python webapps?

Could one reasonably expect to have an easier time with maintainence
and future modifications with using Python over Java?

Any tips, stories, recommendations , and/or experiences are most
welcome.

Thanks.

[brianhray@gmail.com]

Re: fairly large webapp: from Java to Python. experiences?

To replace a large framework you will probably need a framework. Take a
look at http://www.djangoproject.com or http://www.turbogears.org. They
both use some of the tools you mention but operate on a higher level.

I find Python fairly easy to maintain. Unfortunatly, I do not find it
easy to take a bunch of Java code and move it to Python. But once it is
there, Python is a good choice for web apps. Java is slow and has a big
footprint. Python is reasonably nimble and good design choices pay off.
Plus, you must write many more lines of Java to do what you can in
Python. I do not know the code you are not working with, but a lot of
large Java apps can be poorly written.

I guess you could also look at http://www.jython.org/ and see if you
can somehow make the transition but not all at once.

Good luck,
<http://brianray.chipy. org>

[John M. Gabriele]

Re: fairly large webapp: from Java to Python. experiences?

brianhray@gmail .com wrote:[color=blue]
> To replace a large framework you will probably need a framework.[/color]

Well, I'm sure not all web frameworks are created equal, however,
CherryPy does bill itself as a "web framework".
[color=blue]
> Take a
> look at http://www.djangoproject.com or http://www.turbogears.org. They
> both use some of the tools you mention but operate on a higher level.[/color]

Django looks interesting to me. Would like to try out mod_python.
Taking a look...
[color=blue]
> I find Python fairly easy to maintain. Unfortunatly, I do not find it
> easy to take a bunch of Java code and move it to Python.[/color]

Well, I figured it might be better to use the previous webapp
as a *model* to learn from, rather than to directly translate
code from Java to Python.
[color=blue]
> But once it is
> there, Python is a good choice for web apps. Java is slow[/color]

Slow? They're both dynamic languages, but Java is statically
typed (with less work to do at runtime). For long-running processes,
I'd guess that Java bytecode executes faster than Python bytecode.
[color=blue]
> and has a big
> footprint.[/color]

Yes.
[color=blue]
> Python is reasonably nimble and good design choices pay off.
> Plus, you must write many more lines of Java to do what you can in
> Python.[/color]

Yup. The main issue with the current Java webapp seems to be
that it takes a long time to make changes/additions.
[color=blue]
> I do not know the code you are not working with, but a lot of
> large Java apps can be poorly written.[/color]

I've found that good documentation is *most* critical. At least
if the docs are good, but the code needs work, you have some
direction as to how to fix the code. If there's no docs, you
spend hours wondering why things are done the way they're done.

Currently having a look at the Django docs. :)

Thanks Brian!
---J
--
(remove zeez if demunging email address)

[Scott David Daniels]

Re: fairly large webapp: from Java to Python. experiences?

John M. Gabriele wrote:[color=blue]
> brianhray@gmail .com wrote:[color=green]
>> But once it is
>> there, Python is a good choice for web apps. Java is slow[/color]
>
> Slow? They're both dynamic languages, but Java is statically
> typed (with less work to do at runtime). For long-running processes,
> I'd guess that Java bytecode executes faster than Python bytecode.[/color]

Well, perhaps yes, but not so if you count the fact that recoding
is so easy that you can race several implementations of the same
thing to see which trade-offs work. Java may run the one algorithm
you decide to write faster than Python will run the same thing,
but in Python you may race five algorithms and keep the fastest.
Using programmer resources effectively is Python's secret sauce.

--Scott David Daniels
scott.daniels@a cm.org

[Giovanni Bajo]

Re: fairly large webapp: from Java to Python. experiences?

John M. Gabriele wrote:
[color=blue][color=green]
>> But once it is
>> there, Python is a good choice for web apps. Java is slow[/color]
>
> Slow? They're both dynamic languages, but Java is statically
> typed (with less work to do at runtime). For long-running processes,
> I'd guess that Java bytecode executes faster than Python bytecode.[/color]


It's not the raw computing performance that counts in this case. I got this
joke in my mail today:

Python:
print "%10.2f" % x

Java:
java.text.Numbe rFormat formatter = java.text.Numbe rFormat.getNumb erInstance();
formatter.setMi nimumFractionDi gits(2);
formatter.setMa ximumFractionDi gits(2);
String s = formatter.forma t(x);
for (int i = s.length(); i < 10; i++) System.out.prin t(' ');
System.out.prin t(s);

Let alone the time it takes to write this routine, I'm hundered percent sure
that the Python's version is also faster at runtime. Python lets you write
pretty expressive code which is easy to maintain and where the computation cost
is easily at the C level. Also Python code is pretty bare-metal, so that
file.write or socket.write go to the syscall immediately. Try that in Java and
you'll find 30 layers of complex abstractions for doubtful benefits and obvious
slowness.
--
Giovanni Bajo

[Shalabh Chaturvedi]

Re: fairly large webapp: from Java to Python. experiences?

john_sips_tea@y ahoo.com wrote:[color=blue]
> I've got a fairly substantial webapp written in Java (plus Tomcat,
> Hibernate, Struts, JSP, MySQL) that is a bit of a bear to work with. I
> didn't write it. Much of it is only very sparsely documented (if at
> all). No design docs anywhere. It's a large webapp with many classes
> and fairly deep inheritance hierarchies. There seems to be JUnit test
> cases in place.
>
> The possibility of re-writing it has come up, and I must say that,
> although it would likely be a ton of work, it could also be a real
> pythonic adventure. ;)[/color]
<snip>[color=blue]
> Any tips, stories, recommendations , and/or experiences are most
> welcome.
>[/color]

A class-to-class and method-to-method rewrite will give some but likely
not the full benefit of moving to Python. A redesign might be necessary
- making it more 'Pythonic' in the process. In my experience, many cruft
classes that exist in a Java design simply disappear when ported to
Python. See also http://dirtsimple.org/2004/12/python-is-not-java.html

Cheers,
Shalabh

[John M. Gabriele]

Re: fairly large webapp: from Java to Python. experiences?

Shalabh Chaturvedi wrote:[color=blue]
> john_sips_teaz@ yahooz.com wrote:
>
> A class-to-class and method-to-method rewrite will give some but likely
> not the full benefit of moving to Python. A redesign might be necessary
> - making it more 'Pythonic' in the process. In my experience, many cruft
> classes that exist in a Java design simply disappear when ported to
> Python. See also http://dirtsimple.org/2004/12/python-is-not-java.html
>
> Cheers,
> Shalabh
>[/color]

Great link Shalabh. Thanks. :) Googling around, I'd also come across
that one.

---J

--
(remove zeez if demunging email address)

[AdSR]

Re: fairly large webapp: from Java to Python. experiences?

Giovanni Bajo wrote:[color=blue]
> Also Python code is pretty bare-metal, so that
> file.write or socket.write go to the syscall immediately. Try that in Java and
> you'll find 30 layers of complex abstractions for doubtful benefits and obvious
> slowness.[/color]

+1 QOTW
(I'd recommend the whole post but it might be too long.)

Cheers,

AdSR

[Fabio Zadrozny]

Re: fairly large webapp: from Java to Python. experiences?

I agree that python code is usually smaller... but what you did is too
unfair (the code below would be more suitable for the comparrison).

python:
print "%10.2f" % 10

java:
System.out.prin tln(String.form at("%10.2f", 10.0));

-- altough for me it would be the same, as I have defined a print
method... so, it is always something as:
print ("%10.2f", 10) in java for me :-)

What I'm trying to say here is that if you already have a big java app,
sometimes spending some time trying to refactor it for better
understanding is more useful than scrapping what already exists
(especially if you have good test cases)... altough I would reccomend
jython for new development on that same application.

I guess that the main difference is that python usually makes you do the
right thing, whereas in java you need to know a lot more about it to
manage to do the right thing...

Cheers,

Fabio

--
Fabio Zadrozny
------------------------------------------------------
Software Developer

ESSS - Engineering Simulation and Scientific Software

https://www.esss.com.br

PyDev - Python Development Enviroment for Eclipse

http://pydev.sf.net

http://pydev.blogspot.com

Giovanni Bajo wrote:
[color=blue]
>John M. Gabriele wrote:
>
>
>[color=green][color=darkred]
>>>But once it is
>>>there, Python is a good choice for web apps. Java is slow
>>>
>>>[/color]
>>Slow? They're both dynamic languages, but Java is statically
>>typed (with less work to do at runtime). For long-running processes,
>>I'd guess that Java bytecode executes faster than Python bytecode.
>>
>>[/color]
>
>
>It's not the raw computing performance that counts in this case. I got this
>joke in my mail today:
>
>Python:
>print "%10.2f" % x
>
>Java:
>java.text.Numb erFormat formatter = java.text.Numbe rFormat.getNumb erInstance();
>formatter.setM inimumFractionD igits(2);
>formatter.setM aximumFractionD igits(2);
>String s = formatter.forma t(x);
>for (int i = s.length(); i < 10; i++) System.out.prin t(' ');
>System.out.pri nt(s);
>
>Let alone the time it takes to write this routine, I'm hundered percent sure
>that the Python's version is also faster at runtime. Python lets you write
>pretty expressive code which is easy to maintain and where the computation cost
>is easily at the C level. Also Python code is pretty bare-metal, so that
>file.write or socket.write go to the syscall immediately. Try that in Java and
>you'll find 30 layers of complex abstractions for doubtful benefits and obvious
>slowness.
>
>[/color]

[Kent Johnson]

Re: fairly large webapp: from Java to Python. experiences?

Fabio Zadrozny wrote:[color=blue]
> I agree that python code is usually smaller... but what you did is too
> unfair (the code below would be more suitable for the comparrison).
>
> python:
> print "%10.2f" % 10
>
> java:
> System.out.prin tln(String.form at("%10.2f", 10.0));[/color]

Though String.format() is new in Java 1.5 so in older code or for
backward compatibility the longer code may be found. OTOH Python has
supported % formatting since at least version 1.4 released in 1996.

http://www.python.org/doc/1.4/lib/node11.html#SECTION00315100000000000000

In my experience the original point is valid - Python is usually
(always?) more concise than equivalent Java code, and often dramatically
so. Python makes common operations easy; Java sometimes seems to go out
of its way to make them awkward.

Kent

[Fabio Zadrozny]

Re: fairly large webapp: from Java to Python. experiences?

Kent Johnson wrote:
[color=blue]
>Fabio Zadrozny wrote:
>
>[color=green]
>>I agree that python code is usually smaller... but what you did is too
>>unfair (the code below would be more suitable for the comparrison).
>>
>>python:
>>print "%10.2f" % 10
>>
>>java:
>>System.out.pr intln(String.fo rmat("%10.2f", 10.0));
>>
>>[/color]
>
>Though String.format() is new in Java 1.5 so in older code or for
>backward compatibility the longer code may be found. OTOH Python has
>supported % formatting since at least version 1.4 released in 1996.
>http://www.python.org/doc/1.4/lib/no...00000000000000
>
>In my experience the original point is valid - Python is usually
>(always?) more concise than equivalent Java code, and often dramatically
>so. Python makes common operations easy; Java sometimes seems to go out
>of its way to make them awkward.
>
>Kent
>
>[/color]

Yeap, I agree with the original point too (as I explained in the rest of
my comment)... just didn't think it was a 'fair' comparisson. ;-)
-- me? When I use java I write 1.5 code and 'retroweave' it to be
compatible with java 1.4 (Pydev is done this way) -- and I use Python at
work -- and I'm very happy with it... I just think that a language is as
a tool, and each may be suited better for a different occasion.

The 'usually smaller' I mentioned is because sometimes java code gets
smaller because of the already-existing codebase... E.g.: for developing
an IDE, Eclipse already has a huge codebase, so, if I wanted to do all
that was already done in it in python just because Eclipse was in java,
I think I would be doing the wrong decision, and besides, I could use
jython for it if I wanted to...

And from what I understood, the original poster had a large codebase in
java already, so, I'm not sure that changing everything to python would
be the way to go (altough that DOES depend a lot on the original
codebase quality).

Cheers,

Fabio

--
Fabio Zadrozny
------------------------------------------------------
Software Developer

ESSS - Engineering Simulation and Scientific Software

https://www.esss.com.br

PyDev - Python Development Enviroment for Eclipse

http://pydev.sf.net

http://pydev.blogspot.com

[George Sakkis]

Re: fairly large webapp: from Java to Python. experiences?

> Any tips, stories, recommendations , and/or experiences are most[color=blue]
> welcome.[/color]

Just one suggestion, read the article "Things You Should Never Do, Part
I" first (
http://www.joelonsoftware.com/articl...00000069.html). Quoting
from the article:

(Netscape made) "the *single worst strategic mistake* that any software
company can make: They decided to rewrite the code from scratch."

As much as I find python more maintenable and fun to work with in the
long run, I would be *very hesitant* to start a complete rewrite,
especially given the insufficient documentation of your webapp and your
lack of experience with a pythonic web framework.

George

[bruno at modulix]

Re: fairly large webapp: from Java to Python. experiences?

john_sips_tea@y ahoo.com wrote:

(snip)
[color=blue]
> Does anyone here have any experience with large(ish) webapps in Python?[/color]

Depends on the definition of "large(ish) ". If you use KLOC as a metric,
you can expect a Python solution to be 5 to 10 time smaller, so what's a
'large' app in Java may end up as a small/medium app with Python !-)
[color=blue]
> I was sniffing around the following A-team of toolkits: CherryPy,
> Cheetah, SQLObject, all behind Apache2.[/color]

If you plan to run behind Apache2, you may want to have a look at
Myghty. It's somewhere between a framework and a templating system, and
the architecture is based on Perl's HTML::Mason, which has proven to be
successful. It adds to Mason the possibility to take a clean MVC
approach (as well as the more ServerPage oriented approach of Mason),
and can be deployed as either a mod_python handler, fastcgi, plain old
cgi, WSGI, and even as a standalone app (with a Python HTTPServer) for
developpement.

You may also want to take a look at Turbogears (CherryPy + SQLObject +
Kid + Mochikit) or Subway (CherryPy + SQLObject + Cheetah).

[color=blue]
> Has anyone seen performance
> issues with Python webapps?[/color]

Actually, only with Plone or CPS (or other ZopeCMF based app). Zope
itself is pretty usable, but the whole CMF part is a monstruosity IMHO.
[color=blue]
> Could one reasonably expect to have an easier time with maintainence
> and future modifications with using Python over Java?[/color]

Err... It of course depends on first having well written code and a
sensible design, but when it comes to agility - and as long as you stay
away from Zope's CMF !-) -, I bet that Python beats Java hands down.
[color=blue]
> Any tips, stories, recommendations , and/or experiences are most
> welcome.[/color]

It took me about one month to write my first Zope application - an
ad-hoc mix of an online catalog and basic CMS (pages, news,
downloadables files, ...), with LDAP and PostgresSQL in the mix,
fulltext and advanced search, newsletter subscription, a 'members' zone
etc. I had to learn Zope during the process, and I guess I could
actually rewrite the whole thing in less than 2 weeks with a much better
design - but what, the app is in production for 2 years with near zero
corrective maintenance (the biggest problem we had was with system
updates messing with the psycopg adapter...), gives full satisfaction to
the customer, and has for now proven to be very easy to modify and
evolve (so easy that we didn't even bothered to bill the customer for
much of the modification requests)... And Zope is certainly not known
as the most agile or lightweight Python web framework/application server !-)



--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'onurb@xiludom. gro'.split('@')])"