how long is double

[Martijn Lievaart]

Re: how long is double

On Mon, 05 Jan 2004 18:00:17 -0500, Ron Natalie wrote:

[color=blue]
> "Francis Glassborow" <francis@robint on.demon.co.uk> wrote in message
> news:EjxAe+HYje +$Ew3s@robinton .demon.co.uk...
>[color=green]
>> The Standard requires that additions are evaluated on a left to right
>> basis. There are no alternatives. In the case of integer operations the
>> compiler can usually get away with the 'as if' rule and reorder them
>> but in the case of floating point calculations where re-ordering
>> produces a different result it manifestly cannot do that and conform.\[/color]
>
> The standard says NO SUCH THING. With little exception the order of
> operations in C++ is unspecified. Do not confuse parsing
> association/precedence with evaluation.[/color]

I couldn't find it either. Both you and Francis normally know what you are
talking about, so I'll wait for Francis answer before deciding for sure.

M4

[Balog Pal]

Re: how long is double

"Francis Glassborow" <francis@robint on.demon.co.uk> wrote in message
news:qPVLV3BKvT +$Ewh7@robinton .demon.co.uk...
[color=blue][color=green]
> >IMHO a more likely explanation can be that the 3 numbers in a + b + c are
> >added in a different order. And that is a possible source of difference[/color][/color]
at[color=blue][color=green]
> >the last bit of the precision.[/color]
>
> I am not sure that the compiler has that much freedom when the order
> produces different results. This is not the same as the requirements re
> order of evaluation of sub-expressions (i.e. that there is no
> requirement)[/color]

Suppose I have expression, like

double d = 1.1 + x + 2.2 + 3.3; // we have double x in scope

here the compiler must generate code that will do 3 separate additions an
that order? And emitting code equivalent to expression

double d = x + 6.6;

is not really allowed, just in practice we allow the compiler doing it
anyway with some switch?

IIRC the question for this thread used MSVC which has the /Op [Improve Float
Consistency] option. Which the original poster likely dind't use
consistently.

Paul




[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[James Curran]

Re: how long is double

According to the Note in the Standard (1.9.15, pg 7, PDF Pg 33):
"operators can be regrouped according to the usual mathematical
rules....[caveat about machines in which overflows produce an
exception]...the above expression statement can be rewritten by the
implementation in any of the above ways because the same result will occur."

I guess the key point is the meaning of "same result". I was told
(pre-C89) that a C compile could assume infinite precision when reordering
floating point expressions, something not ruled by that statement, and not
addressed (as far as I could see) elsewhere in the Standard.

--
Truth,
James Curran
Home: www.noveltheory.com Work: www.njtheater.com
Blog: www.honestillusion.com Day Job: www.partsearch.com
(note new day job!)

"Francis Glassborow" <francis@robint on.demon.co.uk> wrote in message
news:qPVLV3BKvT +$Ewh7@robinton .demon.co.uk...[color=blue]
>
> I am not sure that the compiler has that much freedom when the order
> produces different results. This is not the same as the requirements re
> order of evaluation of sub-expressions (i.e. that there is no
> requirement)[/color]



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Ed Avis]

Re: how long is double

"P.J. Plauger" <pjp@dinkumware .com> writes:
[color=blue]
>If you want good floating-point results, you have to:
>
>a) test the quality of your environment,[/color]

Is there a good test suite that will work for this? (Both for
detecting hardware failures/bugs and C++ compiler stupidities.)

Not that a test program would eliminate the need to know what you're
doing, of course.

--
Ed Avis <ed@membled.com >

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Ron Natalie]

Re: how long is double


"[color=blue]
>
> I couldn't find it either. Both you and Francis normally know what you are
> talking about, so I'll wait for Francis answer before deciding for sure.
>[/color]
The fourth paragraph of section 5 of the C++ standard:
Except where noted, the order of evaluation of operands of individual operators and subexpressions of individual

expressions, and the order in which side effects take place, is unspecified.

[Martijn Lievaart]

Re: how long is double

On Mon, 05 Jan 2004 23:15:15 -0500, Ron Natalie wrote:

[color=blue]
> "[color=green]
>>
>> I couldn't find it either. Both you and Francis normally know what you
>> are talking about, so I'll wait for Francis answer before deciding for
>> sure.
>>[/color]
> The fourth paragraph of section 5 of the C++ standard: Except where
> noted, the order of evaluation of operands of individual operators and
> subexpressions of individual
>
> expressions, and the order in which side effects take place, is
> unspecified.[/color]

Got it. Thx.

M4

[Ron Natalie]

Re: how long is double


=>[color=blue]
> Suppose I have expression, like
>
> double d = 1.1 + x + 2.2 + 3.3; // we have double x in scope
>
> here the compiler must generate code that will do 3 separate additions an
> that order? And emitting code equivalent to expression
>
> double d = x + 6.6;[/color]

The compiler is free to reorder the expression. If you want to enforce ordering
you have to introduce sequence points in the calculation.


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Ron Natalie]

Re: how long is double


"James Curran" <JamesCurran@mv ps.org> wrote in message news:btd0mb$7g5 @netlab.cs.rpi. edu...[color=blue]
> According to the Note in the Standard (1.9.15, pg 7, PDF Pg 33):
> "operators can be regrouped according to the usual mathematical
> rules....[caveat about machines in which overflows produce an
> exception]...the above expression statement can be rewritten by the
> implementation in any of the above ways because the same result will occur."
>[/color]
First, Note's are non-normative.
Second, the regrouping it's talking about isn't just the reordering of the
order of evaluation. The operative description is in the beginning of
Section 5 (4th paragraph).


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Michal Kowalski]

Re: how long is double

> IMHO a more likely explanation can be that the 3 numbers in a + b + c are[color=blue]
> added in a different order. And that is a possible source of difference at
> the last bit of the precision.[/color]


Good catch. This is exactly what happens in case of a test app I compiled
using VC 6.0 in Debug & Release.

Cumulative results (of summation) in both cases are as follows:

debug:
ST0 = -1.1435963883996 3047e+0001
ST0 = -1.1511880545132 3815e+0001
ST0 = -1.2259384719383 5845e+0001 <= end result

release:
ST0 = -7.4750417425120 2524e-0001
ST0 = -8.2342083538727 8838e-0001
ST0 = -1.2259384719383 5827e+0001 <= end result


MK


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Eric J. Kostelich]

Re: how long is double

In article <l1n092m4pe.fsf @budvar.future-i.net>,
Ed Avis <ed@membled.com > wrote:[color=blue]
>"P.J. Plauger" <pjp@dinkumware .com> writes:
>[color=green]
>>If you want good floating-point results, you have to:
>>
>>a) test the quality of your environment,[/color]
>
>Is there a good test suite that will work for this? (Both for
>detecting hardware failures/bugs and C++ compiler stupidities.)
>[/color]

William Kahan's well-known Paranoia program attempts to deduce,
in a portable way, the basic properties of floating-point arithmetic
on a given implementation. Versions are available for K&R C, Fortran 66,
and others at www.netlib.org/paranoia/. Though archaic, the Fortran
version should be compatible with modern Fortran compilers; the C
version may require modification to be compatible with ANSI C or C++.

--Eric

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[kanze@gabi-soft.fr]

Re: how long is double

Martijn Lievaart <m@remove.this. part.rtij.nl> wrote in message
news:<pan.2004. 01.05.17.20.34. 600665@remove.t his.part.rtij.n l>...[color=blue]
> On Mon, 05 Jan 2004 11:18:40 -0500, Francis Glassborow wrote:[/color]
[color=blue]
> [ f'up comp.lang.c++ ][color=green]
> > In message <3ff8bddd@andro meda.datanet.hu >, Balog Pal <pasa@lib.hu>
> > writes[/color][/color]
[color=blue][color=green][color=darkred]
> >>IMHO a more likely explanation can be that the 3 numbers in a + b +
> >>c are added in a different order. And that is a possible source of
> >>difference at the last bit of the precision.[/color][/color][/color]
[color=blue][color=green]
> > I am not sure that the compiler has that much freedom when the order
> > produces different results. This is not the same as the requirements
> > re order of evaluation of sub-expressions (i.e. that there is no
> > requirement)[/color][/color]
[color=blue]
> I think so. if b is almost -c and a is much smaller, more precision is
> lost is a is added to b first than when b is added to c first. There
> is nothing the compiler can do about this. If this order changes due
> to optimization switches may be seen as a QOI issue, but even that is
> a bridge to far for me.[/color]

I think that this was Francis' point. According to the standard, a+b+c
is (a+b)+c. The compiler is free to rearrange this any way it pleases,
as long as the results are the same as if it had done (a+b)+c. On most
machines, with integer arithmetic, there is no problem. On no machine
that I know of, however, can the compiler legally rearrange floating
point, unless it absolutely knows the values involved.

There was quite a lot of discussion about this when the C standard was
first being written. K&R explicitly allowed rearrangement, even when it
would result in different results. In the end, the C standard decided
not to allow this.

--
James Kanze GABI Software mailto:kanze@ga bi-soft.fr
Conseils en informatique orientée objet/ http://www.gabi-soft.fr
Beratung in objektorientier ter Datenverarbeitu ng
11 rue de Rambouillet, 78460 Chevreuse, France, +33 (0)1 30 23 45 16

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[kanze@gabi-soft.fr]

Re: how long is double

"James Curran" <JamesCurran@mv ps.org> wrote in message
news:<btd0mb$7g 5@netlab.cs.rpi .edu>...[color=blue]
> According to the Note in the Standard (1.9.15, pg 7, PDF Pg 33):
> "operators can be regrouped according to the usual mathematical
> rules....[caveat about machines in which overflows produce an
> exception]...the above expression statement can be rewritten by the
> implementation in any of the above ways because the same result will
> occur."[/color]
[color=blue]
> I guess the key point is the meaning of "same result". I was told
> (pre-C89) that a C compile could assume infinite precision when
> reordering floating point expressions, something not ruled by that
> statement, and not addressed (as far as I could see) elsewhere in the
> Standard.[/color]

In K&R 1, there was an explicit license for the compiler to rearrange
expressions according to the usual laws of algebra. Thus, without
considering possible overflow or rounding errors. The authors of the C
standard removed this liberty, intentionally.

I suppose that a compiler writer could wriggle out on the grounds that
the standard doesn't require a minimum precision for floating point
arithmetic. On the other hand, the considerations of overflow would
still probably hold -- while most hardware will give the correct results
for integer arithmetic, provided they are representable, even if there
was an intermediate overflow, this is not generally the case for
floating point.

--
James Kanze GABI Software mailto:kanze@ga bi-soft.fr
Conseils en informatique orientée objet/ http://www.gabi-soft.fr
Beratung in objektorientier ter Datenverarbeitu ng
11 rue de Rambouillet, 78460 Chevreuse, France, +33 (0)1 30 23 45 16

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Peter van Merkerk]

Re: how long is double


"Francis Glassborow" <francis@robint on.demon.co.uk> wrote in message
news:5DlPW7CEP$ 9$Ew6A@robinton .demon.co.uk...[color=blue]
> In message <8f4ce98a.04010 31338.4208ac8@p osting.google.c om>, f
> <ffunus@yahoo.c om> writes[color=green]
> >I have this
> >
> >double sum, a, b, c;
> >sum = a + b + c;
> >printf("%.20 f = %.20f, %.20f, %.20f", sum, a, b, c);
> >
> >I found that the debug version and release version of the same code
> >give me different result. I am using VC++ 6.0.
> >
> >In debug version, the print out is:
> >-12.259384719383 58500000 = -11.435963883996 30500000,
> >-0.0759166611360 7631300, -0.7475041742512 0252000,
> >
> >In the release version, the print out is:
> >-12.259384719383 58300000 = -11.435963883996 30500000,
> >-0.0759166611360 7631300, -0.7475041742512 0252000,
> >
> >The above sum = a + b + c is just a part of my computation. I found
> >that my whole computation crushed in the debug version because some
> >number became zero and another number divide this number. But this did
> >not happened in the release version.
> >
> >Why?[/color]
>
> FP arithmetic is very sensitive to such things as rounding mode and
> order of evaluation. On x86 architectures there are considerable
> differences between calculations done entirely in register and ones
> where the intermediate results are written back to memory. My guess is
> that in debug mode more intermediate results are being written back and
> thereby are being stripped of guard digits.[/color]

That is correct.
[color=blue]
> For example your problem with '0' can be the consequence of subtracting
> two values that are almost equal and are actually 'equal' within the
> limits of the precision supported by memory values (which often have
> lower precision than register values). This is an interesting case
> because it means that the heavily optimised (minimum of writing back)
> release version works as naively expected while the debug version that
> adheres strictly to the semantics of the abstract C++ machine fails.[/color]

I ran into this problem in project I did several years; the release build
produced slightly different results than the debug build. The Microsoft
compiler has a 'Improve Float Consistency' option (/Op) which fixes this
problem. Unfortunately enabling this option slows down floating point
intensive code quite a bit.

--
Peter van Merkerk
peter.van.merke rk(at)dse.nl



[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[Martijn Lievaart]

Re: how long is double

On Tue, 06 Jan 2004 06:35:26 -0500, kanze wrote:
[color=blue]
> I think that this was Francis' point. According to the standard, a+b+c
> is (a+b)+c. The compiler is free to rearrange this any way it pleases,
> as long as the results are the same as if it had done (a+b)+c. On most
> machines, with integer arithmetic, there is no problem. On no machine
> that I know of, however, can the compiler legally rearrange floating
> point, unless it absolutely knows the values involved.
>
> There was quite a lot of discussion about this when the C standard was
> first being written. K&R explicitly allowed rearrangement, even when it
> would result in different results. In the end, the C standard decided
> not to allow this.[/color]

Then this seems a place where C and C++ differ, see the answer and quote
from the C++ standard from Ron Natalie.

Anyone who can confirm this?

M4


[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]

[P.J. Plauger]

Re: how long is double

"Ed Avis" <ed@membled.com > wrote in message
news:l1n092m4pe .fsf@budvar.fut ure-i.net...
[color=blue]
> "P.J. Plauger" <pjp@dinkumware .com> writes:
>[color=green]
> >If you want good floating-point results, you have to:
> >
> >a) test the quality of your environment,[/color]
>
> Is there a good test suite that will work for this? (Both for
> detecting hardware failures/bugs and C++ compiler stupidities.)
>
> Not that a test program would eliminate the need to know what you're
> doing, of course.[/color]

Fred Tydeman has an incredibly thorough test suite for floating-point
support. That's what we've used to hunt down the most subtle problems,
both in our own libraries and in the environments we build it on.
We have a product called a Quick Proofer which is way less thorough,
but still does a remarkably good job of highlighting FPP lapses.
And we're developing a very powerful set of math function tests in
house that we're not in a hurry to sell.

As for free stuff, there's bugger all out there that's worth the
bother.

P.J. Plauger
Dinkumware, Ltd.

http://www.dinkumware.com

[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.m oderated. First time posters: Do this! ]