Books for learning how to write "big" programs

[duli]

Hi:
I would like recommendations for books (in any language, not
necessarily C++, C, python) which have walkthroughs for developing
a big software project ? So starting from inception, problem
definition, design, coding and final delivery on a single theme
or application.

Most of the code I have written and books that I have read deal with
toy programs and I am looking for something a bit more
comprehensive. For example, maybe a complete compiler written in C++
for some language, or a complete web server or implementing
..net libraries in some language (just a few examples of the scale of
things I am interested in learning).

Thanks!
Duli.

[Victor Bazarov]

Re: Books for learning how to write "big&qu ot; programs

duli wrote:

I would like recommendations for books (in any language, not
necessarily C++, C, python) which have walkthroughs for developing
a big software project ? So starting from inception, problem
definition, design, coding and final delivery on a single theme
or application.
[..]

I think you're asking in wrong newsgroups. We deal with languages, not
processes of developing large systems. Try 'comp.software-eng'. I am
not saying that members of this newsgroup can't recommend books on that
subject or don't know about the processes involved. I am just saying
that a discussion on issues and solutions are *not* language-specific
and thus belong elsewhere.

V
--
Please remove capital 'A's when replying by e-mail
I do not respond to top-posted replies, please don't ask

[Martin P. Hellwig]

Re: Books for learning how to write "big&qu ot; programs

duli wrote:

Hi:
I would like recommendations for books (in any language, not
necessarily C++, C, python) which have walkthroughs for developing
a big software project ? So starting from inception, problem
definition, design, coding and final delivery on a single theme
or application.
>
Most of the code I have written and books that I have read deal with
toy programs and I am looking for something a bit more
comprehensive. For example, maybe a complete compiler written in C++
for some language, or a complete web server or implementing
.net libraries in some language (just a few examples of the scale of
things I am interested in learning).
>
Thanks!
Duli.
>
>

Hmm I think it has not so much to do with the programming language or
the complexity itself. Might be a good thing to look for stuff like
project management, Release Management and Software Life Cycle.

A good starting point might be this:

http://en.wikipedia.org/wiki/Software_development_process

In my opinion, bigger projects are like smaller projects excepts you
have about 90% overhead on documentation before you even write a single
line of code, of course you can try without but that usually ends in
failure of the project.

--
mph

[Mike Driscoll]

Re: Books for learning how to write "big&qu ot; programs

On May 22, 10:55 am, duli <dulipi...@gmai l.comwrote:

Hi:
I would like recommendations for books (in any language, not
necessarily C++, C, python) which have walkthroughs for developing
a big software project ? So starting from inception, problem
definition, design, coding and final delivery on a single theme
or application.
>
Most of the code I have written and books that I have read deal with
toy programs and I am looking for something a bit more
comprehensive.  For example, maybe a complete compiler written in C++
for some language, or a complete web server or implementing
.net libraries in some language (just a few examples of the scale of
things I am interested in learning).
>
Thanks!
Duli.

The "Programmin g Python 3rd Ed." by Lutz has a fairly extensive email
program both as a CLI and a GUI (Tkinter) program. He has various
other real programs too, but they're much smaller.

Other than that, I'd recommend joining a project programmed in Python.
Search SoureForge for them...

Mike

[Ian Collins]

Re: Books for learning how to write &quot;big&qu ot; programs

Martin P. Hellwig wrote:

>
In my opinion, bigger projects are like smaller projects excepts you
have about 90% overhead on documentation before you even write a single
line of code, of course you can try without but that usually ends in
failure of the project.
>

I though it was the over emphasis on paper and lack of working code that
has caused the failure of too many projects.

--
Ian Collins.

[James Kanze]

Re: Books for learning how to write &quot;big&qu ot; programs

On May 23, 3:37 am, Ian Collins <ian-n...@hotmail.co mwrote:

Martin P. Hellwig wrote:

In my opinion, bigger projects are like smaller projects
excepts you have about 90% overhead on documentation before
you even write a single line of code, of course you can try
without but that usually ends in failure of the project.

I though it was the over emphasis on paper and lack of working
code that has caused the failure of too many projects.

If all you produce is paper, I don't think that the project can
be considered a success. On the other hand, anything that
hasn't been written down and reviewed isn't there. I certainly
wouldn't go with "90% overhead on documentation"; I would argue
that any "overhead" on documentation is a sign of a poor
process. Documentation, in various formats, does appear as a
side effect of other aspects of a good process, however: you
can't do a design review unless there is a design, written down
somewhere; you can't do code review unless there is code,
written down somewhere (and the code has to be readable, which
could also be considered documentation); you can't review the
completeness of the unit tests unless there is something which
says what the unit tests test. And you're really wasting your
time if you write a single line of code without knowing why, or
what it should do---and what isn't written down, isn't known.
(That doesn't mean you need a classical external document. Just
that whatever you were thinking when you wrote the code is
written down somewhere, in some form or another.)

From experience, I find that about 60% of the time in a project
will be spent in design, 30% in coding (including coding the
tests), and about 10% in debugging. But of course, it varies
greatly. And what you count in which phase may vary---if you're
using a library you're not familiar with and which isn't well
documented, you'll almost certainly end up writing some
"experiment s" to see how it behaves: is that "coding", "design",
or yet something else?

I don't think that there's that much difference between big
projects and small projects in this respect. The difference
between the two is how you manage larger numbers of people on
the bigger project. In other words, not the "documentation" ,
but how you communicate it between developers, and how you
ensure that everyone is on the same wave length.

--
James Kanze (GABI Software) email:james.kan ze@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[Matthias Buelow]

Re: Books for learning how to write &quot;big&qu ot; programs

James Kanze wrote:

And you're really wasting your
time if you write a single line of code without knowing why, or
what it should do---and what isn't written down, isn't known.

This might be true for some projects but certainly not in the general
case -- sometimes, the goals to achieve are somewhat blurred or might
not yet be known in entirety and have yet to be discovered. In such
situations, explorative programming is (imho) more effective than trying
to pen it down on paper beforehand.

[Ian Collins]

Re: Books for learning how to write &quot;big&qu ot; programs

Matthias Buelow wrote:

James Kanze wrote:
>

>And you're really wasting your
>time if you write a single line of code without knowing why, or
>what it should do---and what isn't written down, isn't known.

>
This might be true for some projects but certainly not in the general
case -- sometimes, the goals to achieve are somewhat blurred or might
not yet be known in entirety and have yet to be discovered. In such
situations, explorative programming is (imho) more effective than trying
to pen it down on paper beforehand.

Indeed. I'm just embarking on that voyage of discovery with my current
client. With eight stakeholders, all with slightly different and
somewhat vague ideas what they want, it should be an interesting journey.

--
Ian Collins.

[Pete Becker]

Re: Books for learning how to write &quot;big&qu ot; programs

On 2008-05-23 03:49:06 -0400, James Kanze <james.kanze@gm ail.comsaid:

>
I don't think that there's that much difference between big
projects and small projects in this respect. The difference
between the two is how you manage larger numbers of people on
the bigger project. In other words, not the "documentation" ,
but how you communicate it between developers, and how you
ensure that everyone is on the same wave length.

I think there's an important difference, but it depends on just what a
"small project" is. For relatively new programmers, a small project is
something they've worked on by themselves, something simple enough that
they can hold all of the documentation in their mind without writing it
down. Getting back into such a project after a long layoff can reveal
the fundamental flaw here: memory fails, and, as with your example of
an unfamiliar library, it may require some experiments to "remember"
why certain things were done. Of course, what this reveals is that a
"small project" in this sense is one that hasn't really been properly
managed, but was small enough that the resulting losses were not
noticed or, typically, were accepted as inevitable. In that sense,
moving from a small project to a large project means learning and
applying project management techniques that previously weren't well
understood or rigorously applied.

After getting through that developmental phase, there is, indeed, not
much difference between big projecgts and small projects. But that's
because the approach to small projects has changed.

--
Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of "The
Standard C++ Library Extensions: a Tutorial and Reference
(www.petebecker.com/tr1book)

[Michele Simionato]

Re: Books for learning how to write &quot;big&qu ot; programs

On May 22, 5:55 pm, duli <dulipi...@gmai l.comwrote:

Hi:
I would like recommendations for books (in any language, not
necessarily C++, C, python) which have walkthroughs for developing
a big software project ? So starting from inception, problem
definition, design, coding and final delivery on a single theme
or application.

Take any Python Web framework, study the source and ask questions.
Also, keep reading this newsgroup, there are plenty of discussions
about (good) design practices.

Michele Simionato

[James Kanze]

Re: Books for learning how to write &quot;big&qu ot; programs

On 23 mai, 10:36, Matthias Buelow <m...@incubus.d ewrote:

James Kanze wrote:

And you're really wasting your time if you write a single
line of code without knowing why, or what it should do---and
what isn't written down, isn't known.

This might be true for some projects but certainly not in the
general case -- sometimes, the goals to achieve are somewhat
blurred or might not yet be known in entirety and have yet to
be discovered. In such situations, explorative programming is
(imho) more effective than trying to pen it down on paper
beforehand.

Yes, but you still have to decide what you're exploring. You
don't just write random code; you write code to fulfill some
specific specification. Whether you're sure that it's the right
specification or not is separate issue---in practice, I'd say
that it's even more important to document what the code is doing
if you're not sure about the specification.

--
James Kanze (GABI Software) email:james.kan ze@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[James Kanze]

Re: Books for learning how to write &quot;big&qu ot; programs

On 23 mai, 13:56, Pete Becker <p...@versatile coding.comwrote :

On 2008-05-23 03:49:06 -0400, James Kanze <james.ka...@gm ail.comsaid:

I don't think that there's that much difference between big
projects and small projects in this respect. The difference
between the two is how you manage larger numbers of people on
the bigger project. In other words, not the "documentation" ,
but how you communicate it between developers, and how you
ensure that everyone is on the same wave length.

I think there's an important difference, but it depends on
just what a "small project" is. For relatively new
programmers, a small project is something they've worked on by
themselves, something simple enough that they can hold all of
the documentation in their mind without writing it down.

Part of my point was that you don't in fact know anything that
isn't written down. You just think you do.

Normally, no professional project should be done by just one
person, but if you don't have the choice, I find that writing
something down forces me to be clearer about it, and to actually
think it through. In my own case, at least, I don't seem to be
able to get away with as much "hand waving" when I'm writing
things down.

--
James Kanze (GABI Software) email:james.kan ze@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

[Ian Collins]

Re: Books for learning how to write &quot;big&qu ot; programs

James Kanze wrote:

On 23 mai, 10:36, Matthias Buelow <m...@incubus.d ewrote:

>James Kanze wrote:

>>And you're really wasting your time if you write a single
>>line of code without knowing why, or what it should do---and
>>what isn't written down, isn't known.

>

>This might be true for some projects but certainly not in the
>general case -- sometimes, the goals to achieve are somewhat
>blurred or might not yet be known in entirety and have yet to
>be discovered. In such situations, explorative programming is
>(imho) more effective than trying to pen it down on paper
>beforehand.

>
Yes, but you still have to decide what you're exploring. You
don't just write random code; you write code to fulfill some
specific specification.

As specified by your unit tests. The requirements from the customer are
often very vague, especially in the web application world. In some
domains, the best way to get detailed requirements from a customer is to
start building the application. Once they see something, they then
realise exactly what they want it to do.

The quality of requirements varies from the exact (say a safety critical
device driver) to the virtually nonexistent (I started what turned out
to be a one year web project with a single page sketch). We have to be
able to adapt our development process to match.

Whether you're sure that it's the right
specification or not is separate issue---in practice, I'd say
that it's even more important to document what the code is doing
if you're not sure about the specification.
>

Indeed, half the battle is getting the customer to agree (or if they are
game, write) acceptance tests. In the vague and woolly world of web
based applications, tools like FitNesse have evolved to help users write
tests.

--
Ian Collins.

[Ian Collins]

Re: Books for learning how to write &quot;big&qu ot; programs

James Kanze wrote:

On 23 mai, 13:56, Pete Becker <p...@versatile coding.comwrote :

>On 2008-05-23 03:49:06 -0400, James Kanze <james.ka...@gm ail.comsaid:

>>I don't think that there's that much difference between big
>>projects and small projects in this respect. The difference
>>between the two is how you manage larger numbers of people on
>>the bigger project. In other words, not the "documentation" ,
>>but how you communicate it between developers, and how you
>>ensure that everyone is on the same wave length.

>

>I think there's an important difference, but it depends on
>just what a "small project" is. For relatively new
>programmers, a small project is something they've worked on by
>themselves, something simple enough that they can hold all of
>the documentation in their mind without writing it down.

>
Part of my point was that you don't in fact know anything that
isn't written down. You just think you do.
>

That's a good point.

Normally, no professional project should be done by just one
person, but if you don't have the choice, I find that writing
something down forces me to be clearer about it, and to actually
think it through. In my own case, at least, I don't seem to be
able to get away with as much "hand waving" when I'm writing
things down.
>

As we are all individuals, the form of the writing down that works best
varies from person to person. I prefer to express my thoughts as code,
in the form of tests. Some prefer UML diagrams, others prefer use cases
or even good old functional requirements.

The most important thing, as you say, is that you do write something
down before you write any application code.

--
Ian Collins.

[Walter Roberson]

Re: Books for learning how to write &quot;big&qu ot; programs

In article <549b03cc-4394-4f66-9175-ea65971d10d7@26 g2000hsk.google groups.com>,
James Kanze <james.kanze@gm ail.comwrote:

>Part of my point was that you don't in fact know anything that
>isn't written down. You just think you do.

And of course as soon as more than one programmer gets involved,
what one thinks they know might not be the same as what the other
person thinks is known about the same topic.

Just this afternoon I asked a co-worker to put an interface in writing,
specifically so as to "lock it in", making it clearer that if the
interface changes after this point that I have to be notified of
that.

The interface itself was relatively simply described... or at least
it was until I replied back and asked about a bunch of corner-cases
that have shown up in our work from time to time.

The responses to several of my questions might be "No, we don't do that".
But those "No's" will form part of the interface description, parts
that were left undifferentiate d may become at least partly defined
by our now saying that we will NOT use particular fields in
particular ways.

It is exactly as you said: until something is written down (and, ideally,
reviewed), we just *think* we know it.
--
"Eightly percent of the people in the world are fools and the
rest of us are in danger of contamination." -- Walter Matthau