Isolation testing, need suggestion

[Andrew]

Hi,
I am thinking about ways for efficient techniques for isolation testing.
Here is the problem as I see it:
Complex OO systems are designed with massive number of dependencies between
modules and classes. In most cases these dependencies end-up in external
systems such as databases and OS. Overhead in designing systems with "could
be stubbed" in mind is great, so in many cases dropped.

I would appreciate if anyone interested in this problem could share his/her
opinion on possible approaches for isolation testing. I also attached some
code with my approach to it. It is based on introducing template parameter
with explicit template instantiation and specification.

Some comments to example and code:
Existing system:
- ImageDB class responsible for reading / writing images to database.
- Image class responsible for retrieval of required image from database,
processing it and storing back to database and depends on ImageDB class.
- Both classes are in middle of developed, many function are not yet fully
implemented.

Task:
- Create test unit for processing functionality (Image::clear() method)
without overhead related to preparing and populating DB with test data.
- Calls to Image and ImageDB classes should keep the same syntax.

Files (sorry for bulky example):
Before (code without stubbing facilities):
=============== =======
// file: app/main.cpp
#include "image.h"
int main( )
{
Image img;
img.load( "John Doe" );
img.clear( );
return 0;
}

----------------------
// file: app/image.cpp

#ifndef __IMAGE_H__
#define __IMAGE_H__

#include <string>
#include "../services/imagedb.h"

class Image
{
public:
Image( );
~Image( );
void load( const std::string& name );
void clear( );

private:
void deallocate( );
ImageDB imageDB_;
size_t width_;
size_t height_;
unsigned int* buffer_;
bool isLoaded_;
std::string name_;
};

#endif // __IMAGE_H__
----------------------
// file: app/image.h
#ifndef __IMAGE_H__
#define __IMAGE_H__

#include <string>
#include "../services/imagedb.h"

class Image
{
public:
Image( );
~Image( );
void load( const std::string& name );
void clear( );

private:
void deallocate( );
ImageDB imageDB_;
size_t width_;
size_t height_;
unsigned int* buffer_;
bool isLoaded_;
std::string name_;
};

#endif // __IMAGE_H__
------------------------------
// services/imagedb.h

#ifndef __IMAGEDB_H__
#define __IMAGEDB_H__

#include <string>

class ImageDB
{
public:
ImageDB( );
bool getImageByName( const std::string& name, unsigned int*& buffer,
size_t& width, size_t& height );
bool setImageByName( const std::string& name, unsigned int* buffer,
size_t width, size_t height );
};

#endif // __IMAGEDB_H__

------------------------------
// services/imagedb.cpp

#include "imagedb.h"

ImageDB::ImageD B( ) {
throw 1; // cannot connect to database
}

bool ImageDB::getIma geByName( const std::string& name,
unsigned int*& buffer, size_t& width,
size_t& height ) {
return false; // not implemented yet
}

bool ImageDB::setIma geByName( const std::string& name,
unsigned int* buffer, size_t width,
size_t height ) {
return false; // not implemented yet
}

=============== ============
After (with some stubbing facilities and implemented unit test):
=============== ============
// file: utest/utest.cpp
#include <string>

struct TestImageProc { };
#define __DIAGNOSTICS__ TestImageProc

#define private public
#include "../app/image.h"
#undef private

// Stubbing ImageDB ctor, so it doesn't throws exception
template< > ImageDBT< TestImageProc >::ImageDBT( ) {}

// Stubbing ImageDB functionality, so it now returns 'true'
template< > bool ImageDBT< TestImageProc >::setImageByNa me(
const std::string& s, unsigned int* b,
size_t w, size_t h ) {
return true;
};

// Stubbing Image constructor, so we do not 'delete [ ] buffer_'
template< > ImageT< TestImageProc >::~ImageT( ) { };

int main( ) {
// test inputs, test setup
Image img;
img.width_ = img.height_ = 2;
unsigned int buf[ ] = { 1, 1, 1, 1 };
img.buffer_ = buf;
// test
img.clear( );
if( buf[ 0 ] != 0 )
{
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
---------------
// file: services/imagedb.h

#ifndef __IMAGEDB_H__
#define __IMAGEDB_H__

#include <string>

template< typename Diagnostics >
class ImageDBT {
public:
ImageDBT( );
bool getImageByName( const std::string& name, unsigned int*& buffer,
size_t& width, size_t& height );
bool setImageByName( const std::string& name, unsigned int* buffer,
size_t width, size_t height );
};

#ifndef __DIAGNOSTICS__
typedef ImageDBT< void > ImageDB;
#else
typedef ImageDBT< __DIAGNOSTICS__ > ImageDB;
#include "imagedb.cp p"
#endif

#endif // __IMAGEDB_H__
---------------
// file: imagedb.cpp
#include "imagedb.h"

#ifndef __DIAGNOSTICS__
template class ImageDBT< void >;
#endif

template< typename D >
ImageDBT< D >::ImageDBT( ) {
throw 1; // not yet defined
}

template< typename D >
bool ImageDBT< D >::getImageByNa me( const std::string& name, unsigned int*&
buffer,
size_t& width, size_t& height ) {
return false; // not yet implemented, fails
}

template< typename D >
bool ImageDBT< D >::setImageByNa me( const std::string& name, unsigned int*
buffer,
size_t width, size_t height ) {
return false; // not yet implemented, fails
}
---------
// file: app/main.cpp
#include "image.h"
int main( ) {
Image img;
img.load( "John Doe" );
img.clear( );
return 0;
}
-----------------------
// file: app/image.h
#ifndef __IMAGE_H__
#define __IMAGE_H__

#include <string>
#include "../services/imagedb.h"

template< typename Diagnostics >
class ImageT {
public:
ImageT( );
~ImageT( );
void load( const std::string& name );
void clear( );
private:
void deallocate( );
ImageDB imageDB_;
size_t width_;
size_t height_;
unsigned int* buffer_;
bool isLoaded_;
std::string name_;
};

#ifndef __DIAGNOSTICS__
typedef ImageT< void > Image;
#else
typedef ImageT< __DIAGNOSTICS__ > Image;
#include "image.cpp"
#endif
#endif //__IMAGE_H__

-----------------

// file: app/image.cpp

#include "image.h"
#include <stdexcept>

#ifndef __DIAGNOSTICS__
template class ImageT< void >; // explicite instanciation of version without
stubbing
#endif

template< typename D > ImageT< D >::ImageT( ):
width_( 0 ), height_( 0 ), buffer_( NULL )
{ }

template< typename D > ImageT< D >::~ImageT( ) {
deallocate( );
}

template< typename D > void ImageT< D >::load( const std::string& name ) {
deallocate( );
isLoaded_ = imageDB_.getIma geByName( name, buffer_, width_, height_ );
name_ = name;
}

template< typename D > void ImageT< D >::clear( ) {
if( isLoaded_ ) {
unsigned int* end = buffer_ + width_ * height_;
for( unsigned int* p = buffer_; p != end; ++p ) {
*p = 0;
}
imageDB_.setIma geByName( name_, buffer_, width_, height_ );
}
}

template< typename D > void ImageT< D >::deallocate ( ){
delete [ ] buffer_;
buffer_ = NULL;
width_ = height_ = 0;
isLoaded_ = false;
name_.empty( );
}

[lilburne]

Re: Isolation testing, need suggestion

Andrew wrote:
[color=blue]
> Hi,
> I am thinking about ways for efficient techniques for isolation testing.
> Here is the problem as I see it:
> Complex OO systems are designed with massive number of dependencies between
> modules and classes. In most cases these dependencies end-up in external
> systems such as databases and OS. Overhead in designing systems with "could
> be stubbed" in mind is great, so in many cases dropped.
>
> I would appreciate if anyone interested in this problem could share his/her
> opinion on possible approaches for isolation testing. I also attached some
> code with my approach to it. It is based on introducing template parameter
> with explicit template instantiation and specification.
>[/color]

Briefly.

We have a system which we call unit testing. For each class
created a subsidiary test class is created. So if we have a
class NurbSurface then there is a class NurbSurfaceTest
which is a friend of NurbSurface, this is so that the test
class can examine the private data of the class under test.

Typically the test class will perform white box testing of
all the methods of the class it is examining. Auxiliary
functions are created for providing complex classes with
test data, in your case populating a test database.

We have scripts for creating boilerplate test classes and
test harnesses.

Each test is self contained requiring no tester input other
than to run the test program. Actually the complete suite of
tests are run unattended, and automatically overnight.

Similar systems are in place for testing class integration
and the main application.

Developers are not allowed to submit code changes unless all
the unit, integration, and application tests have been run
without any failures.

Any new functionality must be amenable to automatic
testing, which may require the development of additional
test/debug commands being added to the application.

All our tests are designed to crash the application or test
program on detecting an error.

[Frank Schmitt]

Re: Isolation testing, need suggestion

syrov@msn.com (Andrew) writes:
[color=blue]
> Hi,
> I am thinking about ways for efficient techniques for isolation testing.
> Here is the problem as I see it:
> Complex OO systems are designed with massive number of dependencies between
> modules and classes. In most cases these dependencies end-up in external
> systems such as databases and OS. Overhead in designing systems with "could
> be stubbed" in mind is great, so in many cases dropped.
>
> I would appreciate if anyone interested in this problem could share his/her
> opinion on possible approaches for isolation testing. I also attached some
> code with my approach to it. It is based on introducing template parameter
> with explicit template instantiation and specification.[/color]

One possible approach are Mock Objects - Objects that have the same interface
as the real Objects, but are much easier to set up, e.g. for a database:

struct DBInterface {
virtual DataList getDataForQuery (const std::string& query) = 0;
virtual ~DBInterface() {}
};

struct MockDBInterface : public DBInterface {
DataList getDataForQuery (const std::string&) {
// return predefined DataList
}
};

struct OracleDBInterfa ce: public DBInterface {
DataList getDataForQuery (const std::string&) {
// connect to Database, execute query, return results
}
};

of course, in your client code you'll have to use the DBInterface class
rather than the real OracleDBInterfa ce class - but that seems like a
good idea, anyway.
For further reading on Mock Objects, go to

http://www.connextra.com/aboutUs/mockobjects.pdf

<SNIP>
[color=blue]
> ----------------------
> // file: app/image.cpp
>
> #ifndef __IMAGE_H__
> #define __IMAGE_H__[/color]

That's a very bad idea (TM) - any name containing two consecutive
underscores is reserved to the implementation, as well as any name
starting with an underscore followed by an uppercase letter - or
any name starting with an underscore visible in the global
namespace.
In short - don't use double underscores or leading underscores
for your identifiers / macros.

HTH & kind regards
frank

--
Frank Schmitt
4SC AG phone: +49 89 700763-0
e-mail: frankNO DOT SPAMschmitt AT 4sc DOT com

[Goran Sliskovic]

Re: Isolation testing, need suggestion


"Andrew" <syrov@msn.co m> wrote in message
news:a1c22c8e.0 310071502.2f470 0d0@posting.goo gle.com...[color=blue]
> Hi,
> I am thinking about ways for efficient techniques for isolation testing.
> Here is the problem as I see it:
> Complex OO systems are designed with massive number of dependencies[/color]
between[color=blue]
> modules and classes. In most cases these dependencies end-up in external
> systems such as databases and OS. Overhead in designing systems with[/color]
"could[color=blue]
> be stubbed" in mind is great, so in many cases dropped.
>
> I would appreciate if anyone interested in this problem could share[/color]
his/her[color=blue]
> opinion on possible approaches for isolation testing. I also attached some
> code with my approach to it. It is based on introducing template parameter
> with explicit template instantiation and specification.
>
> Some comments to example and code:
> Existing system:
> - ImageDB class responsible for reading / writing images to database.
> - Image class responsible for retrieval of required image from database,
> processing it and storing back to database and depends on ImageDB class.
> - Both classes are in middle of developed, many function are not yet fully
> implemented.
>
> Task:
> - Create test unit for processing functionality (Image::clear() method)
> without overhead related to preparing and populating DB with test data.
> - Calls to Image and ImageDB classes should keep the same syntax.
>[/color]
....

I think you are bit off-topic here. You can try in group
comp.software.e xtreme-programming. This group is related to XP method which
is tightly linked to unit testing. The thing is: if you can't simply test
the class, class probably is not designed well. You have strong dependancy
on implementation of ImageDB class and because of that it is not possible to
simply test it. Image class, besides providing storage for the image data,
provides for loading from database. If you separate this tasks, your
testing will be much easier. You have increased coupling and lowered class
cohesion by not separating this tasks, which is usually bad. But then
againg, it depends on the context of problem you are trying to solve.

Regards,
Goran