validation testing -- use TPC-C? (or what?)

[z]

How can validation testing on DBMSs such as DB2 and Oracle be performed?

Is there a quick commonly used publicly available DBMS test that can be
used?

Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)

Thanks.

[Larry Menard]

Re: validation testing -- use TPC-C? (or what?)

My TPC knowledge is a bit dated, so take it with a grain of salt.

First of all, make sure you are looking for the right benchmark for the
type of processing you are interested in. For example, TPC-C is a benchmark
that is indicative primarily of rapid transaction processing. There are
other benchmarks for workloads that are more DSS-intensive (TPC-H or TPC-D)
or web-intensive (TPC-W).

Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications are
closely-guarded secrets, since the application code itself might contain
code that accounts for significant performance improvements.
--
--------------------
Larry Menard
"Defender of Geese and of All Things Natural"


"z" <z@y.x.invalidw rote in message
news:6OBnf.3705 4$tV6.22163@new ssvr27.news.pro digy.net...

How can validation testing on DBMSs such as DB2 and Oracle be performed?
>
Is there a quick commonly used publicly available DBMS test that can be
used?
>
Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)
>
Thanks.
>
>

[Mark A]

Re: validation testing -- use TPC-C? (or what?)

"Larry Menard" <root@GoSpamYou rself.comwrote in message
news:ToqdnbsTUv 7FYgPenZ2dnUVZ_ t-dnZ2d@rogers.co m...

My TPC knowledge is a bit dated, so take it with a grain of salt.
>
First of all, make sure you are looking for the right benchmark for the
type of processing you are interested in. For example, TPC-C is a
benchmark that is indicative primarily of rapid transaction processing.
There are other benchmarks for workloads that are more DSS-intensive
(TPC-H or TPC-D) or web-intensive (TPC-W).
>
Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications
are closely-guarded secrets, since the application code itself might
contain code that accounts for significant performance improvements.
--
--------------------
Larry Menard
"Defender of Geese and of All Things Natural"
>

The TPC benchmarks are published at www.tpc.org

I have reproduced the TPC-H (decision support) benchmark and the tpc.org
site provides you with everything you will need, including the test data
generator program. You may find it a bit difficult to compile the test data
generator program successfully, but I was able to get it to work after
changing a few compile parms. They also supply the schema and queries. You
will want to look at the vendor docs for each benchmark to pick up any
customization they have done to queries to allow for slight variations in
SQL syntax between the databases.

The TPC-C OLTP benchmark may be harder to reproduce as mentioned above, but
details of the benchmarks are published by each vendor on the website so you
could look there.

[Serge Rielau]

Re: validation testing -- use TPC-C? (or what?)

Larry Menard wrote:

My TPC knowledge is a bit dated, so take it with a grain of salt.
>
First of all, make sure you are looking for the right benchmark for the
type of processing you are interested in. For example, TPC-C is a benchmark
that is indicative primarily of rapid transaction processing. There are
other benchmarks for workloads that are more DSS-intensive (TPC-H or TPC-D)
or web-intensive (TPC-W).
>
Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications are
closely-guarded secrets, since the application code itself might contain
code that accounts for significant performance improvements.

Larry,

Actually the rules demand full disclosure. And everything should be
available rom the TPC website. Starting from the algorithms to generate
the data to the exact configuration and sourcecodee for any run.
In theory a competitor can validate a TPC result inhouse
(I doubt that this has ever been done though).
Due to this rule to expose these benchmarks can be challenged.
E.g. Oracle successfully challenged the first SQL Server 2000 TPC-C
benchmark using clustering (partitioned views) because SS2000 couldn't
update the partitioning key.

Cheers
Serge
--
Serge Rielau
DB2 SQL Compiler Development
IBM Toronto Lab

[Larry Menard]

Re: validation testing -- use TPC-C? (or what?)

I stand (or actually, sit) corrected.

--
--------------------
Larry Menard
"Defender of Geese and of All Things Natural"


"Serge Rielau" <srielau@ca.ibm .comwrote in message
news:40936rF19d 7ckU1@individua l.net...

Larry Menard wrote:

> My TPC knowledge is a bit dated, so take it with a grain of salt.
>>
> First of all, make sure you are looking for the right benchmark for
>the type of processing you are interested in. For example, TPC-C is a
>benchmark that is indicative primarily of rapid transaction processing.
>There are other benchmarks for workloads that are more DSS-intensive
>(TPC-H or TPC-D) or web-intensive (TPC-W).
>>
> Second, I don't know of any publicly available TPC-C kits. It is my
>understandin g that the source code used for TPC-C benchmark applications
>are closely-guarded secrets, since the application code itself might
>contain code that accounts for significant performance improvements.

Larry,
>
Actually the rules demand full disclosure. And everything should be
available rom the TPC website. Starting from the algorithms to generate
the data to the exact configuration and sourcecodee for any run.
In theory a competitor can validate a TPC result inhouse
(I doubt that this has ever been done though).
Due to this rule to expose these benchmarks can be challenged.
E.g. Oracle successfully challenged the first SQL Server 2000 TPC-C
benchmark using clustering (partitioned views) because SS2000 couldn't
update the partitioning key.
>
Cheers
Serge
--
Serge Rielau
DB2 SQL Compiler Development
IBM Toronto Lab

[Matt Emmerton]

Re: validation testing -- use TPC-C? (or what?)


"z" <z@y.x.invalidw rote in message
news:6OBnf.3705 4$tV6.22163@new ssvr27.news.pro digy.net...

How can validation testing on DBMSs such as DB2 and Oracle be performed?
>
Is there a quick commonly used publicly available DBMS test that can be
used?
>
Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)

If you want to validate a database for use in your business environment, the
best thing to do is build a simple prototype of your business app and test
it on the databases you're considering. Both DB2 and Oracle come in
"express" editions, and should be full-featured enough to put a prototype
through it's paces. This way you can play with performance scaling, various
DDL options, backup/restore and other things that you're bound to do when in
production, and really get a feel for what works best in the intended
environment.

To echo someone else's comments -- using an off-the-shelf benchmark (such as
any of the TPC benchmarks) may not be anywhere near close to what your
business needs are, and thus you'd end up with something that isn't a good
fit. It's much like deciding to buy a sports car because it was the fastest
and flashiest, and then finding out that your wife is pregnant with triplets
and you should have bought a mini-van :)

--
Matt Emmerton

[DA Morgan]

Re: validation testing -- use TPC-C? (or what?)

Matt Emmerton wrote:

"z" <z@y.x.invalidw rote in message
news:6OBnf.3705 4$tV6.22163@new ssvr27.news.pro digy.net...
>

>>How can validation testing on DBMSs such as DB2 and Oracle be performed?
>>
>>Is there a quick commonly used publicly available DBMS test that can be
>>used?
>>
>>Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
>>available?)

>
>
If you want to validate a database for use in your business environment, the
best thing to do is build a simple prototype of your business app and test
it on the databases you're considering.

Absolutely. TCP-? has absolutely nothing to do with the real world and
will provide you zero information related to your specific hardware,
your specific operating system, your specific configuration, your
specific application or anything else.
--
Daniel A. Morgan

http://www.psoug.org

damorgan@x.wash ington.edu
(replace x with u to respond)