how to become a really good Python programmer?

[Randall Smith]

I've been programming in Python for about 2 years. I think it offers
the best combination of simplicity and power of any language I have
explored. As I write more and larger and complex programs, I need to
code better. By better I mean clearer, cleaner, more efficient and
maintainable. As the subject states, I want to become a really good
Python programmer. I learn most everything from random examples and
experience and that's good, but now I would like some sound, directed
guidance. Following Guido around would be ideal, but I will settle for
a good book. Any recommendations for learning resources?

Randall

[Troy Melhase]

Re: how to become a really good Python programmer?

On Tuesday 15 June 2004 10:06 pm, Randall Smith wrote:[color=blue]
> I've been programming in Python for about 2 years. I think it offers
> the best combination of simplicity and power of any language I have
> explored. As I write more and larger and complex programs, I need to
> code better.[/color]

[snip]
[color=blue]
> Following Guido around would be ideal, but I will settle for
> a good book. Any recommendations for learning resources?[/color]

Yes.

1. import this

2. Write more code. The more you write, the more opportunity you have to
learn.

3. Study the code written by folks more experienced than you. There's a ton
of good code in the standard library, and there's even more available on the
net. Pick a package and read it until you understand it. Evaluate the
problems it solves and how well it solves them.

4. Refactor your own code, and be merciless when doing it. There's no
teacher like breaking 10,000 lines of code.

5. Just as you are doing now, don't stop learning.

--
Troy Melhase, troy@gci.net
--
Tolerance is the virtue of the man without convictions. - G. K. Chesterton

[Roger Binns]

Re: how to become a really good Python programmer?

Randall Smith wrote:[color=blue]
> Following Guido around would be ideal, but I will settle for
> a good book. Any recommendations for learning resources?[/color]

Becoming really good generally means also making your own mistakes
and/or learning from the mistakes of others.

The Python cookbook is full of excellent ways of doing things
you may not have stumbled upon by just reading the doc or in your
day to day work. I prefer the book, but the content is all on
the website as well.

I would recommend programming in the "open". Find an open source
project and do some code for it. That will give the chance for
others to critique what you have coded, as well as you seeing
what others code all out in the open. You also get an appreciation
for other issues if you want such as release engineering, documentation,
configuration management which are all part of programming.

Here is a list of over 3,300 Python projects at SourceForge:

http://sof.net/softwaremap/trove_list.php?form_cat=178

The first page alone has a wide variety of areas so you should be
able to find some areas that interest you. It also makes good
resume filler :-)

Roger

[Miki Tebeka]

Re: how to become a really good Python programmer?

Hello Randall,
[color=blue]
> I've been programming in Python for about 2 years. I think it offers
> the best combination of simplicity and power of any language I have
> explored. As I write more and larger and complex programs, I need to
> code better. By better I mean clearer, cleaner, more efficient and
> maintainable. As the subject states, I want to become a really good
> Python programmer. I learn most everything from random examples and
> experience and that's good, but now I would like some sound, directed
> guidance. Following Guido around would be ideal, but I will settle for
> a good book. Any recommendations for learning resources?[/color]
Apart from other recommendations you'll get I suggest you learn other
programming languages. This will enable you to view problems in a
different light.

I recommend at least one lisp dialect (scheme is perfect for learning)
and one prolog dialects. Both have free interpreters available (I
recommend mzscheme and SWI prolog).

HTH.

Bye.
--
-------------------------------------------------------------------------
Miki Tebeka <miki.tebeka@zo ran.com>
The only difference between children and adults is the price of the toys.

[Peter Hansen]

Re: how to become a really good Python programmer?

Randall Smith wrote:
[color=blue]
> I've been programming in Python for about 2 years. I think it offers
> the best combination of simplicity and power of any language I have
> explored. As I write more and larger and complex programs, I need to
> code better. By better I mean clearer, cleaner, more efficient and
> maintainable. As the subject states, I want to become a really good
> Python programmer.[/color]

I think one of the best ways to do that (in addition to the excellent
suggestions you've already received) might be to learn test-driven
development (TDD). The things it will tell you about the clarity,
cleanliness, efficiency, and maintainability of your own code will
surprise you, and you'll likely improve rapidly in these areas as
a result.

Not to mention find yourself beginning to write code with fewer bugs!

-Peter

[Cameron Laird]

Re: how to become a really good Python programmer?

In article <bsWdneSnANDktk 3dRVn-gw@powergate.ca >,
Peter Hansen <peter@engcorp. com> wrote:[color=blue]
>Randall Smith wrote:
>[color=green]
>> I've been programming in Python for about 2 years. I think it offers
>> the best combination of simplicity and power of any language I have
>> explored. As I write more and larger and complex programs, I need to
>> code better. By better I mean clearer, cleaner, more efficient and
>> maintainable. As the subject states, I want to become a really good
>> Python programmer.[/color]
>
>I think one of the best ways to do that (in addition to the excellent
>suggestions you've already received) might be to learn test-driven
>development (TDD). The things it will tell you about the clarity,
>cleanliness, efficiency, and maintainability of your own code will
>surprise you, and you'll likely improve rapidly in these areas as
>a result.
>
>Not to mention find yourself beginning to write code with fewer bugs!
>
>-Peter[/color]

Already mentioned in this thread was *The Python Cookbook*,
edited by leading practitioners of TDD. For more on this
book, as well as another written by the one of the authors,
see <URL: http://www.unixreview.com/documents/s=7822/ur0303j/ >
and the references there.
--

Cameron Laird <claird@phaseit .net>
Business: http://www.Phaseit.net

[Roy Smith]

Re: how to become a really good Python programmer?

Randall Smith <randall@tnr.cc > wrote:[color=blue]
> I've been programming in Python for about 2 years. I think it offers
> the best combination of simplicity and power of any language I have
> explored. As I write more and larger and complex programs, I need to
> code better. By better I mean clearer, cleaner, more efficient and
> maintainable. As the subject states, I want to become a really good
> Python programmer.[/color]

What you really want to do is become a really good programmer, who
happens to write programs in Python.

The virtues you mention; clarity, efficiency, and maintainability , are
pretty much universal, regardless of what language you write in,
although I'm not sure I would put them in that order. For the vast
majority of code that you'll write, efficiency should not be that high
on the list.

First off, you want your programs to be correct. The first step in
correctness is understanding what they are supposed to do. If you don't
know what it's supposed to do, how do you know if it's doing it or not?
Whether you use some kind of formal specification process, or something
more dynamic such as the XP folks espouse, don't start writing code
until you know what you're trying to do.

The second step in making sure your programs are correct is testing.
You can spend a lifetime learning (and arguing) about the best way to
test, but any testing is better than none. Once you start making
testing an essential part of your process, you'll start to discover that
some programs are easier to test than others. Strive to make what you
write easy to test.

Clarity comes from breaking your program up into logical units
(functions, classes, modules, etc) which can be understood by
themselves. Huge monolithic programs with complex interactions and
global data are hard to understand. Good choice of variable and
function names helps too, as does good use of comments (although,
exactly what constitutes good use of comments is another whole topic of
debate).

None of the above is Python specific.

[Grant Edwards]

Re: how to become a really good Python programmer?

On 2004-06-16, Miki Tebeka <miki.tebeka@zo ran.com> wrote:
[color=blue]
> Apart from other recommendations you'll get I suggest you
> learn other programming languages. This will enable you to
> view problems in a different light.
>
> I recommend at least one lisp dialect (scheme is perfect for
> learning) and one prolog dialects. Both have free interpreters
> available (I recommend mzscheme and SWI prolog).[/color]

Have a look at Smalltalk as well if you've got the time. It's
close enough to Python to be comfortable, but different enough
to provide some insights.

--
Grant Edwards grante Yow! Now KEN and BARBIE
at are PERMANENTLY ADDICTED to
visi.com MIND-ALTERING DRUGS...

[David MacQuigg]

Re: how to become a really good Python programmer?

On Wed, 16 Jun 2004 01:06:41 -0500, Randall Smith <randall@tnr.cc >
wrote:
[color=blue]
>I've been programming in Python for about 2 years. I think it offers
>the best combination of simplicity and power of any language I have
>explored. As I write more and larger and complex programs, I need to
>code better. By better I mean clearer, cleaner, more efficient and
>maintainable . As the subject states, I want to become a really good
>Python programmer. I learn most everything from random examples and
>experience and that's good, but now I would like some sound, directed
>guidance. Following Guido around would be ideal, but I will settle for
>a good book. Any recommendations for learning resources?[/color]

My favorite book on good programming, not Python specifically, but an
accumulation of decades of experience, is The Art of Unx Programming,
by Eric Raymond. If you are doing anything with user interfaces, read
The Inmates are Running the Asylum, by Alan Cooper.

-- Dave

[Richie Hindle]

Re: how to become a really good Python programmer?


[Randall][color=blue]
> As I write more and larger and complex programs, I need to
> code better. By better I mean clearer, cleaner, more efficient and
> maintainable.[/color]

If you're not doing so already, start using Test-Driven Development. Google
will give you any number of resources about that, but in a nutshell it means
writing unit tests *before* writing code, then writing the code to make the
tests pass. This forces you to think about the design of your modules up
front, and usually leads to better-designed software. The modules you write
will have cleaner interfaces and will be less dependent upon each other,
more clearly written, more reusable, and more amenable to changing without
breaking things.

The other important feature of test-driven development is that it's *hard*,
or at least it's hard for some application areas (like networking and GUIs).
There's nothing like working on a hard problem to sharpen your skills -
knocking up a messy script is easy, but you don't learn from it.

A concrete example of why it's hard and why it leads to better code: I have
a lightweight RPC system (as yet unpublished) that sends "safe pickles" over
the network to invoke functions on another machine. The unit tests are a
mess, because they run the client and server socket code in different
threads in the same process, and synchronising them is painful. Had I
thought more about it up front, I'd have realised that the RPC system should
be transport-independent, and the unit tests should provide a trivial
transport system for the RPC code to use during the tests, rather than
mucking about with sockets. I'd have ended up with a nice
transport-independent RPC module with simple and complete unit tests, plus a
tiny transport module to implement a socket transport for it. Done right,
the transport module would be so trivial that either it didn't need unit
tests, or the tests would be very simple.

A question for experienced TDD-ers: is TDD really more intellectually
demanding than code-first, or am I either a) doing it wrong, or b) stupid?

--
Richie Hindle
richie@entrian. com

[Ray Cote]

Re: how to become a really good Python programmer?

At 1:06 AM -0500 6/16/04, Randall Smith wrote:[color=blue]
>I've been programming in Python for about 2 years. I think it
>offers the best combination of simplicity and power of any language
>I have explored. As I write more and larger and complex programs, I
>need to code better. By better I mean clearer, cleaner, more
>efficient and maintainable. As the subject states, I want to become
>a really good Python programmer. I learn most everything from
>random examples and experience and that's good, but now I would like
>some sound, directed guidance. Following Guido around would be
>ideal, but I will settle for a good book. Any recommendations for
>learning resources?
>
>Randall
>--
>http://mail.python.org/mailman/listinfo/python-list[/color]

Hi Randall:
Some random thoughts in random order.

1: Write lots of code.
2: Show it to other Python programmer's for comment/criticism.
3: Show it to other non-Python programmer's who will question why you
did it a certain way.
4: Write lots of code.
5: Read about Python, product design (not just software), aesthetics, style.
Python idiom.
6: Spend time in an art museum.
7: Go back and review, re-document, and redesign what you wrote last year.
8: Use other languages (Lisp, Java, Smalltalk, C++, C#, Perl, ...) to
understand their approach to problem solving.
9: Test lots of code and try different approaches to solving the same problem.
10: Read lots of other people's code.
11: Walk through your code in the debugger.
12: Become familiar with one assembler language and walking through
it in a debugger.
13 Become comfortable with Python idioms so your code 'looks like'
the Python way to do things.
14: Oh yes, write lots of code.
--Ray

--

[Peter Hansen]

on TDD (was Re: how to become a really good Python programmer?)

(Yes, yes, I can see you smiling, you who was just waiting for my
response... or John's. :-)

Richie Hindle wrote:
[color=blue]
> If you're not doing so already, start using Test-Driven Development. Google
> will give you any number of resources about that, but in a nutshell it means
> writing unit tests *before* writing code, then writing the code to make the
> tests pass. This forces you to think about the design of your modules up
> front, and usually leads to better-designed software. The modules you write
> will have cleaner interfaces and will be less dependent upon each other,
> more clearly written, more reusable, and more amenable to changing without
> breaking things.[/color]

As Richie is (self-admittedly) somewhat new to TDD, I'd like to tweak
the above description about how it works, though the final sentence is
exactly correct about the results.

Even though it used to be called "test-first", saying the tests are
written *before* the code is actually misleading. A single test is
written before any code, then run (and it should obviously fail!),
and then some code is written, just enough to pass the test. Once
that test passes, the code is cleaned up if necessary, then another
test is written, and the cycle begins again. It will generally
take a few minutes for an entire cycle, sometimes more, sometimes less.

And although Richie is right that this will make you think more about
the design, I'd argue the "up front" claim and say that it makes you
think about the design all the time, rather than *merely* up front
as with some other approaches. Doing it up front only is like
pointing your car in a particular direction, then hitting the gas
and not turning the wheel until you get there. It can work, but
a lot of people get hurt in the process. Doing it all the time
is like driving really works, where you actually pay attention and
turn the wheel from time to time. (Note that you can still hurt
people this way, if you want, so all the fun is not removed.)
[color=blue]
> The other important feature of test-driven development is that it's *hard*,
> or at least it's hard for some application areas (like networking and GUIs).[/color]
[...][color=blue]
> A question for experienced TDD-ers: is TDD really more intellectually
> demanding than code-first, or am I either a) doing it wrong, or b) stupid?[/color]

TDD is both harder and easier, I think. Once you're practiced with it,
most of the small stuff -- "typical" code -- becomes fairly trivial
and thus less demanding than trying to do it the typical way (code
and fix, resorting to debuggers to step through code, lots of print
statements and head-scratching, manually tests). On the other hand,
for non-trivial stuff, such as GUIs, threading, networking, etc, the
fact that it requires you to write automated tests definitely forces
you to think harder about difficult things, and to face challenges
other programmers never have to face.

As Richie's client/server example showed, sometimes tests can get out
of hand. You can live with them, or you can refactor and clean them
up. Once you've been down that path once or twice, you will start to
see the warning signals well in advance, and eventually you will just
do the thing he concluded should have been done, automatically and
without thinking much, right at the start. So Richie, you're definitely
not stupid, but you're probably still new enough at it that you're
learning some valuable lessons. (That's my more positive way of saying
you were doing it wrong. ;-)

-Peter

[Jeff Sandys]

Re: how to become a really good Python programmer?

Randall Smith wrote:[color=blue]
>
> ... I want to become a really good
> Python programmer. ...[/color]

Find someone to pair program with.

It would be cool if they were an expert,
but it is ok if they aren't. You will
learn more, program faster and see lots
of possibilities when you pair program.

Thanks,
Jeff Sandys

[Richie Hindle]

Re: on TDD (was Re: how to become a really good Python programmer?)

[Peter][color=blue]
> (Yes, yes, I can see you smiling, you who was just waiting for my
> response... or John's. :-)[/color]

<innocent-expression>Oh, so you're into TDD, Peter? I had no
idea.</innocent-expression>

Thanks for clarifying my summary of TDD - I can see how it might have been
misunderstood.
[color=blue]
> Once you've been down that path once or twice, you will start to
> see the warning signals well in advance, and eventually you will just
> do the thing he concluded should have been done, automatically and
> without thinking much, right at the start.[/color]

I'm looking forward to it. 8-)

For others looking to learn more about TDD, I've found this paper:

http://www.edwardh.com/ieee_article/goingfaster.pdf

useful as an introduction to the principles of TDD - not the nuts and
bolts of how to practise it, but ideas like splitting your core code
(eg. my RPC protocol) away from hard-to-test external libraries
(eg. sockets).

I think one of the problems I'm having is that I'm thinking like a
module coder rather than a test coder. A question for any passing
TDD-er (not with anyone specific in mind, of course, smart questions
and all that 8-) - how much does the design of the module and its API
get influenced by the requirement of writing tests for it? A coder
has two tasks: write a module (and the code that needs to use it), and
write tests for it. Both will have some bearing on the interface to
the module, but in what proportion? My suspicion is that 75% of the
design of the module's API will fall out of the way the tests are
written, and that that's a good thing...?

--
Richie Hindle
richie@entrian. com

[Peter Hansen]

Re: on TDD (was Re: how to become a really good Python programmer?)

Richie Hindle wrote:
[color=blue]
> I think one of the problems I'm having is that I'm thinking like a
> module coder rather than a test coder. A question for any passing
> TDD-er (not with anyone specific in mind, of course, smart questions
> and all that 8-) - how much does the design of the module and its API
> get influenced by the requirement of writing tests for it? A coder
> has two tasks: write a module (and the code that needs to use it), and
> write tests for it. Both will have some bearing on the interface to
> the module, but in what proportion? My suspicion is that 75% of the
> design of the module's API will fall out of the way the tests are
> written, and that that's a good thing...?[/color]

Since I was already here, thought I'd answer anyway... ;-)

I think it depends on the coder and the context (and probably the
phase of the moon) a lot, but sometimes I've found nearly 100%
of the API and design falls out of the tests... That is, the API
comes from the tests largely, or from when you're writing the tests,
because that's when you are first thinking about "okay, I need
to get an X here, so how would I like to go about getting it?".

The design comes partly from the fact that you have now specified
the external interface, but also from the code you write as you
try to pass the test. If the tests are small enough (i.e. you
are switching very often between test and code) then the code
will inevitably be fairly modular, loosely coupled, easily
testable, and all or most of those other good things.

And don't forget to refactor... sometimes the design is not
quite so good, even though you are passing the tests, until you
do this critical step.

In other cases, I think I already have a good idea what the API
should look like, probably because "I've written one of these
before". In that case, the tests may reflect my preconceptions
more. I wouldn't be surprised if the resulting API and design
are worse in such cases than if I had never written one before...

-Peter