A simple unit test framework

[Phlip]

Re: A simple unit test framework

Pete Becker wrote:

Nope. Test driven design cannot account for the possibility that a
function will use an internal buffer that holds N bytes, and has to
handle the edges of that buffer correctly. The specification says
nothing about N, just what output has to result from what input.

You slipped from "write tests before code" to "write tests based only on the
specifications before writing code".

Even under high amnesia situations, that N-byte buffer must be one function
call away from its test driver. This improves the odds you notice it and
question if production code can overflow it. And if you expressly treat the
test cases as documenting the code, then you should write tests which
document that buffer.

All methodologies still require review...

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

I never underestimate the ingenuity of good testers.

Ian, did someone play "fallacy of excluded middle" on you?

;-)

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

Bo Persson wrote:

So Pete will pass your first test with "return 1;".
>
How many more tests do you expect to write, before you are sure that
Pete's code is always no more than one unit off in the last decimal?

That technique is "fake it til you make it". It's not the core principle of
TDD, but sometimes it's surprisingly powerful. You pass a test with obtusely
simple code, then write the next test to force that code to upgrade in the
correct direction. And you frequently refactor to simplify the code, even if
it's fake.

The surprising part comes when you stop faking. You pass your new test cases
either by trivially extending the existing design, or even by doing nothing.
This system can rapidly produce the Open Closed Principle for a given set of
inputs.

If, for a given situation, it's _not_ surprisingly powerful, you _will_
learn that very early!

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

Pete Becker wrote:

When I was a QA manager I'd bristle whenever any developer said they'd
"let testers" do something. That's simply wrong. Testing is not an adjunct
to development. Testing is a profession, with technical challenges that
differ from, and often exceed in complexity, those posed by development.
The skills required to do it well are vastly different, but no less
sophisticated, than those needed to write the product itself. Most
developers think they can write good tests, but in reality, they simply
don't have the right skills, and test suites written by developers are
usually naive. A development manager who gives testers "complete freedom"
is missing the point: that's not something the manager can give or take,
it's an essential part of effective testing.

Nobody said to fire all the testers. You made that up, then replied to your
presumption.

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

Gianni Mariani wrote:

The most successful teams I worked with were teams that wrote their own
tests.

That's why the testers are not far away, in time, space, or timezone. They
should be in the same lab.

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

Pete Becker wrote:

Yup. Typical developer-written test: I don't understand testing well
enough to do it right, so I'll do something random and hope to hit a
problem. <g>

Nobody said the developers would write soak tests! What part of "write a
thing incorrectly called a 'test' that fails because the next line of code
is not there" are you stuck on?

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

James Kanze wrote:

I find it makes the design and code process easier and way more fun. No
more debugging sessions!

Until the code starts actually being used.

That's _why_ people who use TDD tend to practice Daily Deployment. They
actually use the code immediately.

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

Ian Collins wrote:

If you code test first (practice Test Driven Design/Development), you
don't have to do coverage analysis because your code has been written to
pass the tests.

Each line has coverage. That's still not the ultimate coverage, where tests
cover every path through the entire program. But the effect on code makes
such coverage less important. The tests force the code to be simpler,
because it must pass simple tests.

--
Phlip

http://www.oreilly.com/catalog/9780596510657/

"Test Driven Ajax (on Rails)"
assert_xpath, assert_javascri pt, & assert_ajax

[Phlip]

Re: A simple unit test framework

James Kanze wrote:

Note that the two steps above work together. The people doing the code
review also review the tests, and verify that your choice of test cases
was appropriate for the code you wrote.

Oh, you must have read Rule #42 in the Official TDD Handbook:

"After writing simple code to pass the test, you must then immediately
release your code, without letting anyone review it or add more tests to
it."

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

Ian Collins wrote:

No, The first embedded product my team developed using TDD has had about
half a dozen defects reported in 5 years and several thousand units of
field use. They were all in bits of the code with poor tests.

Ain't it sad when former Thought Leaders reach the next bend in the road of
progress, and we discover how ossified their geriatric brains have become?
;-)

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Phlip]

Re: A simple unit test framework

>The problem with Google:

They use TDD up the wazoo.

It sure shows in their pitiful quality, long time to market, customer
disatisfaction, and snowed executives, huh?

;-)

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Ian Collins]

Re: A simple unit test framework

Phlip wrote:

Ian Collins wrote:
>

>No, The first embedded product my team developed using TDD has had about
>half a dozen defects reported in 5 years and several thousand units of
>field use. They were all in bits of the code with poor tests.

>
Ain't it sad when former Thought Leaders reach the next bend in the road of
progress, and we discover how ossified their geriatric brains have become?
;-)
>

I had wondered where you were hiding!

--
Ian Collins.

[Phlip]

Re: A simple unit test framework

I had wondered where you were hiding!

RoR, holmes.

--
Phlip

http://flea.sourceforge.net/PiglegToo_1.html

[Ian Collins]

Re: A simple unit test framework

James Kanze wrote:

On May 6, 3:31 am, Ian Collins <ian-n...@hotmail.co mwrote:

>Gianni Mariani wrote:

>

>>I have met very few customers that know what a spec is even if it
>>smacked them up the side of the head.

>

>Welcome to the club!

>

>>Sad. Inevitably it leads to pissed off customer.

>

>Any agile process (XP, Scrum or whatever) is ideal for this situation.

>
If your goal is to rip off the customer, yes.
>

So by helping them to get what they really wanted, rather than forcing
them to commit to what they thought the wanted, I'm ripping them off?
The person I'm ripping off is me, I'm doing my self out of all the bug
fixing and rework jobs.

Man you have a strange view of customer focused development.

>This situation is one of the main reasons these processes have evolved.
> The sort release cycles keep the customer in the loop and give them the
>flexibility to change their mind without incurring significant rework
>costs.

>
Prototyping and interacting with the customer are nothing new.
They've been part of every software development process for the
last 30 or 40 years.
>

There's a huge difference between demonstrating a prototype and
delivering production quality code every week or so.

--
Ian Collins.

[Gianni Mariani]

Re: A simple unit test framework

James Kanze wrote:
....

>
There's nothing to disagree with. It's a basic definition. If
a MC test finds the error, and a hand crafted test doesn't, the
hand crafted test isn't well designed or carefully done.

We disagree on the practicality of your claim.

>

>I have one anecdotal evidence which suggests that no-one is capable of
>truly foreseeing the full gamut of issues that can be found in a well
>designed MC test.

>
I have more than anecdotal evidence that there are significant
errors which will slip through MC testing. Admittedly, the
most significant ones also slip through the most carefully
crafted tests as well.

.... ok, so we don't disagree.

... It is, in fact, impossible to write a
test for them which will reliably fail.
>
This is why no shop serious about quality will rely solely on
testing.

Can we limit the scope of discussion to unit testing. If you want to go
to complete product life-cycle we will be here forever.

>

>A pass on an MC test raises the level of confidence which is always a
>good thing.

>
Certainly. It's just the often, generating enough random data
is more effort than doing things correctly, and beyond a very
low level, doesn't raise the confidence level very much.

OK -- we'll have to disagree again. I've been able to find bugs using
MC tests that were never found by "well crafted tests".

... If we
take Pete's example of a log() function, testing with a million
random values doesn't really give me much more confidence than
testing with a hundred, and both give significantly less
confidence than a good code review, accompanied by testing with
a few critical values. (Which values are critical, of course,
being determined by the code review.)

Right. Very good. So this is your proof that MC tests are all bad ?