Machine precision

[Philipp]

Hello (not sure this is the right forum for that question so please redirect
me if necessary)

How can I know how many double values are available between 0 and 1?
On my machine (pentium 3) I get a sizeof(double) = 8

Is the distribution of the double values in a fixed range (eg here between 0
and 1) uniform? ie same number of values in the range [0.0 ; 0.1[ than in
the range [0.9 ; 1.0[

How can I interpret the DBL_EPSILON value (which is 2.22045e-16 on my
machine).

Any good website on the subject to recommend?

Thanks Phil

[Thomas Matthews]

Re: Machine precision

Philipp wrote:[color=blue]
> Hello (not sure this is the right forum for that question so please redirect
> me if necessary)
>
> How can I know how many double values are available between 0 and 1?
> On my machine (pentium 3) I get a sizeof(double) = 8[/color]

The precision of a type double is left up to the compiler. The C++
specification states a minimum precision, but your compiler is allowed
to exceed that precision, regardless of whether the processor has
the capability or not. The compiler is allowed to use software to
for floating point calculations. Summary: See your compiler
documentation or ask in a newsgroup about your compiler.

[color=blue]
> Is the distribution of the double values in a fixed range (eg here between 0
> and 1) uniform? ie same number of values in the range [0.0 ; 0.1[ than in
> the range [0.9 ; 1.0[[/color]

My guess is that the distribution is uniform, and depends on the
limits set by the compiler.


[color=blue]
> How can I interpret the DBL_EPSILON value (which is 2.22045e-16 on my
> machine).[/color]

My understanding is the DBL_EPSILON is the finest resolution for a
double. Although you may want to check the C++ specification on that.


[color=blue]
> Any good website on the subject to recommend?
>
> Thanks Phil[/color]

Probably the site for the ANSI electronic documents.

--
Thomas Matthews

C++ newsgroup welcome message:

http://www.slack.net/~shiva/welcome.txt

C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.l earn.c-c++ faq:

http://www.raos.demon.uk/acllc-c++/faq.html

Other sites:
http://www.josuttis.com -- C++ STL Library book

[Patrick Frankenberger]

Re: Machine precision


"Philipp" wrote:[color=blue]
> How can I know how many double values are available between 0 and 1?
> On my machine (pentium 3) I get a sizeof(double) = 8
>
> Is the distribution of the double values in a fixed range (eg here between[/color]
0[color=blue]
> and 1) uniform? ie same number of values in the range [0.0 ; 0.1[ than in
> the range [0.9 ; 1.0[
>
> How can I interpret the DBL_EPSILON value (which is 2.22045e-16 on my
> machine).
>
> Any good website on the subject to recommend?[/color]

C++ doubles are based on the IEEE754-standard, which most CPUs implement.

There is a nice paper titled "What Every Computer Scientist Should Know
About Floating-Point Arithmetic" by David Goldberg. It answers all of your
questions, except the DBL_EPSILON one.

DBL_EPSILON should be the smallest double d so that (1+d)!=1 IIRC.

HTH,
Patrick

[P.J. Plauger]

Re: Machine precision

"Thomas Matthews" <Thomas_Matthew sHatesSpam@sbcg lobal.net> wrote in message
news:2RUkb.1002 5$eY6.8992@news svr16.news.prod igy.com...
[color=blue]
> Philipp wrote:[color=green]
> > Hello (not sure this is the right forum for that question so please redirect
> > me if necessary)
> >
> > How can I know how many double values are available between 0 and 1?
> > On my machine (pentium 3) I get a sizeof(double) = 8[/color]
>
> The precision of a type double is left up to the compiler. The C++
> specification states a minimum precision, but your compiler is allowed
> to exceed that precision, regardless of whether the processor has
> the capability or not. The compiler is allowed to use software to
> for floating point calculations. Summary: See your compiler
> documentation or ask in a newsgroup about your compiler.[/color]

All true, but that doesn't answer the OP's question. There's only a loose
relation between the number of bytes occupied by a floating-point value
and the number of values it can represent between 0 and 1. To first order,
the representation typically uses one bit to represent the sign of the
value and one bit to represent the sign of the exponent. That's a slight
simplification, and the range of exponents is often asymmetric around 1.0.
But this is enough to tell you that, for eight-bit bytes, you can expect
about 2^62 values between 0.0 and 1.0. FWIW.
[color=blue][color=green]
> > Is the distribution of the double values in a fixed range (eg here between 0
> > and 1) uniform? ie same number of values in the range [0.0 ; 0.1[ than in
> > the range [0.9 ; 1.0[[/color]
>
> My guess is that the distribution is uniform, and depends on the
> limits set by the compiler.[/color]

No, the distribution is extremely *non* uniform, with values much more densely
packed close to zero.
[color=blue][color=green]
> > How can I interpret the DBL_EPSILON value (which is 2.22045e-16 on my
> > machine).[/color]
>
> My understanding is the DBL_EPSILON is the finest resolution for a
> double. Although you may want to check the C++ specification on that.[/color]

DBL_EPSILON is the smallest value you can add to 1.0 and get a representable
answer greater than 1.0. It's a measure of the granularity of values in the
uniform range just above 1.0. (If the floating-point base is 2, the values are
twice as dense in the uniform range just below 1.0.)
[color=blue][color=green]
> > Any good website on the subject to recommend?[/color][/color]

The most readable intro to this stuff I've ever read is an ancient book
by Pat Sterbenz, called Floating Point Computation. Wish I could think of
a modern version that's as good. You can try reading the preambles to the
various modern floating-point formats, particularly those based on IEEE 754,
but they seldom discuss the implications of the representation.

HTH,

P.J. Plauger
Dinkumware, Ltd.

http://www.dinkumware.com

[P.J. Plauger]

Re: Machine precision

"Patrick Frankenberger" <p.frankenberge r@gmx.net> wrote in message
news:bn15u2$3us $03$1@news.t-online.com...
[color=blue]
> C++ doubles are based on the IEEE754-standard, which most CPUs implement.[/color]

Sadly, there is no such requirement. It is often the case, however, because
most modern processors do indeed implement IEEE 754 floating-point arithmetic.
[color=blue]
> There is a nice paper titled "What Every Computer Scientist Should Know
> About Floating-Point Arithmetic" by David Goldberg. It answers all of your
> questions, except the DBL_EPSILON one.[/color]

Generally good reading, if a bit alarmist.
[color=blue]
> DBL_EPSILON should be the smallest double d so that (1+d)!=1 IIRC.[/color]

You RC.

P.J. Plauger
Dinkumware, Ltd.

http://www.dinkumware.com

[Pete Becker]

Re: Machine precision

Patrick Frankenberger wrote:[color=blue]
>
> C++ doubles are based on the IEEE754-standard, which most CPUs implement.
>[/color]

It's the other way around: most CPUs implement IEEE 754, so that's what
most C++ implementations do. The C++ standard does not require IEEE 754.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

[Keith S.]

Re: Machine precision

P.J. Plauger wrote:
[color=blue]
> No, the distribution is extremely *non* uniform, with values much more densely
> packed close to zero.[/color]

This has me interested, since I would have assumed the same as the
previous poster, i.e. that values would be evenly spaced according
to the smallest value (DBL_EPSILON).

Anyone have a simple explanation of why?

- Keith

[grejdanospam@pacbell.net]

Re: Machine precision

On Mon, 20 Oct 2003 19:08:32 +0100, Keith S. <false@ntlworld .com> wrote:
[color=blue]
> P.J. Plauger wrote:
>[color=green]
>> No, the distribution is extremely *non* uniform, with values much more
>> densely
>> packed close to zero.[/color]
>
> This has me interested, since I would have assumed the same as the
> previous poster, i.e. that values would be evenly spaced according
> to the smallest value (DBL_EPSILON).
>
> Anyone have a simple explanation of why?
>
> - Keith
>
>[/color]

Up to the time when most significant bit becomes one.
Then the spacing is becomes two times bigger.
You forgot about the exponent.


--
grzegorz

[Keith S.]

Re: Machine precision

grejdanospam@pa cbell.net wrote:
[color=blue]
> Up to the time when most significant bit becomes one.
> Then the spacing is becomes two times bigger.
> You forgot about the exponent.[/color]

Ah. All is clear :)

- Keith

[Patrick Frankenberger]

Re: Machine precision


"Keith S." wrote:[color=blue]
> P.J. Plauger wrote:
>[color=green]
> > No, the distribution is extremely *non* uniform, with values much more[/color][/color]
densely[color=blue][color=green]
> > packed close to zero.[/color]
>
> This has me interested, since I would have assumed the same as the
> previous poster, i.e. that values would be evenly spaced according
> to the smallest value (DBL_EPSILON).[/color]

A floating-point number is: a*2^(b-offset)
a is a signed integer and b is an unsigned integer.

HTH,
Patrick

[Ron Natalie]

Re: Machine precision


"Keith S." <false@ntlworld .com> wrote in message news:bn18b1$sb4 ge$1@ID-169434.news.uni-berlin.de...[color=blue]
> P.J. Plauger wrote:
>[color=green]
> > No, the distribution is extremely *non* uniform, with values much more densely
> > packed close to zero.[/color]
>
> This has me interested, since I would have assumed the same as the
> previous poster, i.e. that values would be evenly spaced according
> to the smallest value (DBL_EPSILON).
>
> Anyone have a simple explanation of why?
>[/color]
FLOATING POINT. Do you understand mantissa and exponent?

[Keith S.]

Re: Machine precision

Ron Natalie wrote:
[color=blue]
> FLOATING POINT. Do you understand mantissa and exponent?[/color]

I understand politeness, shame that you do not.

- Keith

[Ron Natalie]

Re: Machine precision


"Keith S." <false@ntlworld .com> wrote in message news:bn1g82$sbl v6$1@ID-169434.news.uni-berlin.de...[color=blue]
> Ron Natalie wrote:
>[color=green]
> > FLOATING POINT. Do you understand mantissa and exponent?[/color]
>
> I understand politeness, shame that you do not.
>[/color]
I wasn't trying to be impolite, just a bit terser then usual. doubles aren't
just fractions, they shift. I thought the above would be enough of a hint
if you thought about it.

[osmium]

Re: Machine precision

Ron Natalie wrote:
[color=blue][color=green][color=darkred]
> > > No, the distribution is extremely *non* uniform, with values much more[/color][/color][/color]
densely[color=blue][color=green][color=darkred]
> > > packed close to zero.[/color]
> >
> > This has me interested, since I would have assumed the same as the
> > previous poster, i.e. that values would be evenly spaced according
> > to the smallest value (DBL_EPSILON).
> >
> > Anyone have a simple explanation of why?
> >[/color]
> FLOATING POINT. Do you understand mantissa and exponent?[/color]

In the dark ages that thing was called, mistakenly, mantissa. It has
nothing to do with the mantissa as in logarithms. Many (most?) people are
now using a much less tortured term "significan d". AFAIK that word was
coined specifically for the use at hand. Much better to invent a word than
to use one wrongly, which is what has been done in this field. So if the
poster knew about mantissas (which he probably did) he would be doubly
confused.

[E. Robert Tisdale]

Re: Machine precision

Philipp wrote:
[color=blue]
> How can I know how many double values are available between 0 and 1?
> On my machine (pentium 3) I get a sizeof(double) = 8
>
> Is the distribution of the double values in a fixed range
> (eg here between 0 and 1) uniform?
> i.e. same number of values in the range [0.0 ; 0.1[
> than in the range [0.9 ; 1.0[
>
> How can I interpret the DBL_EPSILON value
> (which is 2.22045e-16 on my machine).
>
> Any good website on the subject to recommend?[/color]

Read
What Every Scientist Should Know About Floating-Point Arithmetic

http://docs.sun.com/db?p=/doc/800-7895

On your machine, a floating-point number

double x = (1 - 2*s)*m*2^e

where s in {0, 1} is the sign bit,
1/2 <= m < 1 is the *normalized* mantissa and
e is the exponent.
There are DBL_MANT_DIG = 53 binary digits
in the mantissa m but, since the most significant bit
is always 1, it isn't represented and is known as the hidden bit
so there are just 2^52 possible values for m.
For normalized double precision floating-point,
DBL_MIN_EXP = -1021 <= e <= 1024 = DBL_MAX_EXP.
When e = -1022, a *denormalized* double precision
floating-point number x = (1 - 2*s)*m*2^(-1021)
where 0 <= m < 1/2.
When e = +1025, x is Not a Number (NaN)
or a positive or negative infinity.
The IEEE representation is

SEM

where S is the sign bit,
E is an eleven bit [excess 1023] exponent
and 1 <= M < 2 is the 52 bit mantissa with a hidden 1 bit.

s = S
m = M/2
e = (E - 1023) + 1

Note that E = 0 when e = -1022 so that
the representation of +0 is all zeros.