FAQ Topic - Why does simple decimal arithmetic give strange results? (2008-10-11)

[FAQ server]

-----------------------------------------------------------------------
FAQ Topic - Why does simple decimal arithmetic give strange
results?
-----------------------------------------------------------------------

For example, ` 5 * 1.015 ` does not give exactly
` 5.075 ` and ` 0.06+0.01 ` does
not give exactly ` 0.07 ` in javascript.

ECMAScript numbers are represented in binary as IEEE-754 (IEC 559)
Doubles, with a resolution of 53 bits, giving an accuracy of
15-16 decimal digits; integers up to about ` 9e15 ` are precise, but
few decimal fractions are. Given this, arithmetic is as exact
as possible, but no more. Operations on integers are exact if
the true result and all intermediates are integers within that
range.

In particular, non-integer results should not normally be
compared for equality; and non-integer computed results
commonly need rounding; see 4.6.

http://msdn2.microsoft.com/en-us/library/7wkd9z69.aspx

http://www.merlyn.demon.co.uk/js-misc0.htm#DW4

Otherwise, use ` Math.round ` on the results of expressions which
should be of integer value.


--
Postings such as this are automatically sent once a day. Their
goal is to answer repeated questions, and to offer the content to
the community for continuous evaluation/improvement. The complete
comp.lang.javas cript FAQ is at http://jibbering.com/faq/index.html.
The FAQ workers are a group of volunteers. The sendings of these
daily posts are proficiently hosted by http://www.pair.com.

[Dr J R Stockton]

Re: FAQ Topic - Why does simple decimal arithmetic give strange results? (2008-10-11)

In comp.lang.javas cript message <48efde78$0$902 75$14726298@new s.sunsite.
dk>, Fri, 10 Oct 2008 23:00:02, FAQ server <javascript@dot internet.be>
posted:

>ECMAScript numbers are represented in binary as IEEE-754 (IEC 559)
>Doubles, with a resolution of 53 bits, giving an accuracy of
>15-16 decimal digits; integers up to about ` 9e15 ` are precise, but
>few decimal fractions are. Given this, arithmetic is as exact
>as possible, but no more. Operations on integers are exact if
>the true result and all intermediates are integers within that
>range.

The last sentence is correct. However, the semi-literate may not
recognise the difference between the concepts expressed by "if" and
"only if".

One can say that "Operations on Numbers are exact if the true result
and all intermediates can be expressed exactly as binary fractions with
a 53-bit mantissa and an exponent in the range -X to +Y" which is also
(I think) exact and covers more, but not all, cases. One can say that
"Operations on Numbers are exact if the true result and all
intermediates can be expressed exactly as IEEE Doubles", but that won't
mean anything to the intended audience.

Can the wording be briefly improved so that naive readers will realise
that operations on numbers which are multiples of 0.5, 0,25, 0,125,
.... about 1.1102230246251 565404236316680 908e-16 are safe? Or is "few
decimal fractions" good enough?

--
(c) John Stockton, Surrey, UK. ?@merlyn.demon. co.uk Turnpike v6.05 MIME.
Web <URL:http://www.merlyn.demo n.co.uk/- FAQish topics, acronyms, & links.
Proper <= 4-line sig. separator as above, a line exactly "-- " (SonOfRFC1036)
Do not Mail News to me. Before a reply, quote with ">" or "" (SonOfRFC1036)

[dhtml]

Re: FAQ Topic - Why does simple decimal arithmetic give strange results?(2008-10-11)

Dr J R Stockton wrote:

In comp.lang.javas cript message <48efde78$0$902 75$14726298@new s.sunsite.
dk>, Fri, 10 Oct 2008 23:00:02, FAQ server <javascript@dot internet.be>
posted:
>

>ECMAScript numbers are represented in binary as IEEE-754 (IEC 559)
>Doubles, with a resolution of 53 bits, giving an accuracy of
>15-16 decimal digits; integers up to about ` 9e15 ` are precise, but
>few decimal fractions are. Given this, arithmetic is as exact
>as possible, but no more. Operations on integers are exact if
>the true result and all intermediates are integers within that
>range.

>
The last sentence is correct. However, the semi-literate may not
recognise the difference between the concepts expressed by "if" and
"only if".
>
One can say that "Operations on Numbers are exact if the true result
and all intermediates can be expressed exactly as binary fractions with
a 53-bit mantissa and an exponent in the range -X to +Y" which is also
(I think) exact and covers more, but not all, cases. One can say that
"Operations on Numbers are exact if the true result and all
intermediates can be expressed exactly as IEEE Doubles", but that won't
mean anything to the intended audience.
>
Can the wording be briefly improved so that naive readers will realise
that operations on numbers which are multiples of 0.5, 0,25, 0,125,
... about 1.1102230246251 565404236316680 908e-16 are safe? Or is "few
decimal fractions" good enough?
>

There could be an example, but then that might warrant more explanation
that cannot be quickly explained.

Garrett

--
comp.lang.javas cript FAQ <URL: http://jibbering.com/faq/ >