float representation and precision

[Sebastian]

Hi,

yesterday I discovered something that won't come as a surprise to more experienced C++ - programmers - but I was genuinely amazed. I had this line of code:

float x = 0.9f;

and when I had a look at it in the VS.NET 2003 debugger, it said x had avalue of 0.89999998 ! When I printf'ed x to a logfile, it was 0.8999999761581 420900000000000 00. Now, why on earth is that? I understand that the results of floatingpoint operations will always be somewhat imprecise due to machine limitations, but 0.9? Can anybody explain this to me?

[JH Trauntvein]

Re: float representation and precision


Sebastian wrote:[color=blue]
> Hi,
>
> yesterday I discovered something that won't come as a
> surprise to more experienced C++ - programmers - but I
> was genuinely amazed. I had this line of code:
>
> float x = 0.9f;
>
> and when I had a look at it in the VS.NET 2003 debugger,
> it said x had a value of 0.89999998 ! When I printf'ed x
> to a logfile, it was 0.8999999761581 420900000000000 00. Now,
> why on earth is that? I understand that the results of
> floatingpoint operations will always be somewhat imprecise
> due to machine limitations, but 0.9? Can anybody explain
> this to me?[/color]

IEEE754, which is the floating point representation used on most
platforms that I am aware of, is a base two encoding for floating point
values. Just as base 10 has irrational fractions (somewhat like 1/3)
so does base two. The adventure here is that the fractions that are
irrational in base two are different from those that are irrational in
base 10. Apparently 9/10 is one of those irrational fractions in base
2.

Regards,

Jon Trauntvein

[Rolf Magnus]

Re: float representation and precision

Sebastian wrote:
[color=blue]
> Hi,
>
> yesterday I discovered something that won't come as a surprise to more
> experienced C++ - programmers - but I was genuinely amazed. I had this
> line of code:
>
> float x = 0.9f;
>
> and when I had a look at it in the VS.NET 2003 debugger, it said x had a
> value of 0.89999998 ! When I printf'ed x to a logfile, it was
> 0.8999999761581 420900000000000 00. Now, why on earth is that? I understand
> that the results of floatingpoint operations will always be somewhat
> imprecise due to machine limitations, but 0.9? Can anybody explain this to
> me?[/color]

Floating point numbers approximate a continuous range of numbers. Since that
range has an infinite count of different numbers, it isn't possible to
store every number exactly in a computer.
Remember that computers are not calculating in base 10 as humans usually
are, but in base 2. So a number that might look simple to us (like 0.9) can
in base 2 have an infinite number of digits, and thus be impossible to
store exactly in a base 2 floating point number.

[Sebastian]

Re: float representation and precision

Wow. Thanks for the speedy answer. I like this - the idea that there would be irrational numbers in the binary system never crossed my mind, butit sounds absolutely logical. :-)


On 7 May 2005 05:21:19 -0700, JH Trauntvein <j.trauntvein@c omcast.net> wrote:
[color=blue]
>
> Sebastian wrote:[color=green]
>> Hi,
>>
>> yesterday I discovered something that won't come as a
>> surprise to more experienced C++ - programmers - but I
>> was genuinely amazed. I had this line of code:
>>
>> float x = 0.9f;
>>
>> and when I had a look at it in the VS.NET 2003 debugger,
>> it said x had a value of 0.89999998 ! When I printf'ed x
>> to a logfile, it was 0.8999999761581 420900000000000 00. Now,
>> why on earth is that? I understand that the results of
>> floatingpoint operations will always be somewhat imprecise
>> due to machine limitations, but 0.9? Can anybody explain
>> this to me?[/color]
>
> IEEE754, which is the floating point representation used on most
> platforms that I am aware of, is a base two encoding for floating point
> values. Just as base 10 has irrational fractions (somewhat like 1/3)
> so does base two. The adventure here is that the fractions that are
> irrational in base two are different from those that are irrational in
> base 10. Apparently 9/10 is one of those irrational fractions in base
> 2.
>
> Regards,
>
> Jon Trauntvein
>
>[/color]



--

Regards,

Sebastian

[Frederick Bruckman]

Re: float representation and precision

In article <1115468479.276 263.21340@z14g2 000cwz.googlegr oups.com>,
"JH Trauntvein" <j.trauntvein@c omcast.net> writes:[color=blue]
> Sebastian wrote:[color=green]
>>
>> yesterday I discovered something that won't come as a
>> surprise to more experienced C++ - programmers - but I
>> was genuinely amazed. I had this line of code:
>>
>> float x = 0.9f;
>>
>> and when I had a look at it in the VS.NET 2003 debugger,
>> it said x had a value of 0.89999998 ![/color][/color]
[color=blue]
> IEEE754, which is the floating point representation used on most
> platforms that I am aware of, is a base two encoding for floating point
> values. Just as base 10 has irrational fractions (somewhat like 1/3)
> so does base two. The adventure here is that the fractions that are
> irrational in base two are different from those that are irrational in
> base 10. Apparently 9/10 is one of those irrational fractions in base
> 2.[/color]

An "irrational fraction" is a contradiction in terms, "irrational "
meaning "cannot be expressed as a ratio", i.e. "cannot be expressed
as a fraction". You mean "repeating" , as in "repeating decimal".
Now, what makes a decimal number repeat, is that it has factors
other than "2" and "5" in the denominator. A non-repeating binary
ratio can only have powers of "2" in the denominator. So, evidently,
every repeating decimal number also repeats in binary, but there
are repeating binary nubers that won't repeat in decimal (e.g. 2/5).

In any case, that's not the reason for the loss of precision in
converting to floating point and back. What's "stored in binary"
is not the number itself, but the natural logarithm of the number.
Since _e_, the base of the natural logarithm is irrational, any
rational number is going to have a non-rational representation in
floating point. The only numbers that come out even in floating
point are irrational themselves. If you ever don't get a rounding
error when converting to floating point and back, it's essentially
coincidence (although it will be reproducible, of course).

--
Frederick

[cadull]

Re: float representation and precision

> In any case, that's not the reason for the loss of precision in[color=blue]
> converting to floating point and back. What's "stored in binary"
> is not the number itself, but the natural logarithm of the number.
> Since _e_, the base of the natural logarithm is irrational, any
> rational number is going to have a non-rational representation in
> floating point. The only numbers that come out even in floating
> point are irrational themselves. If you ever don't get a rounding
> error when converting to floating point and back, it's essentially
> coincidence (although it will be reproducible, of course).[/color]

I don't know where you get the natural log information from.

http://www.nuvisionmiami.com/books/asm/workbook/floating_tut.htm

Regards,
cadull

[Frederick Bruckman]

Re: float representation and precision

In article <d5ihjg$adj$1@n nrp.waia.asn.au >,
"cadull" <clinton.no@spa m.cadull.com> writes:[color=blue][color=green]
>> In any case, that's not the reason for the loss of precision in
>> converting to floating point and back. What's "stored in binary"
>> is not the number itself, but the natural logarithm of the number.
>> Since _e_, the base of the natural logarithm is irrational, any
>> rational number is going to have a non-rational representation in
>> floating point. The only numbers that come out even in floating
>> point are irrational themselves. If you ever don't get a rounding
>> error when converting to floating point and back, it's essentially
>> coincidence (although it will be reproducible, of course).[/color]
>
> I don't know where you get the natural log information from.[/color]

Oops. It's log-2. Fractions involving powers of two will thus
convert back and forth without loss.

Frederick

[Rolf Magnus]

Re: float representation and precision

Frederick Bruckman wrote:
[color=blue]
> In any case, that's not the reason for the loss of precision in
> converting to floating point and back. What's "stored in binary"
> is not the number itself, but the natural logarithm of the number.[/color]

Where did you get that idea from? Floating point numbers are usually stored
in base 2, not base e. I have seen a format where they were stored in base
16, but never e.

[Pete Becker]

Re: float representation and precision

Frederick Bruckman wrote:
[color=blue]
>
> Oops. It's log-2.
>[/color]

No, it's not. It's just like 1.3 x 10^3, but in base 2 instead of base
10. The exponent is the logarithm of the value, but the fraction is just
the representation of the scaled value in the appropriate base. So 1/2
is represented in binary floating point as .1 base 2, 1 is represented
as .1 x 2^1, and 2 is represented as .1 x 2^2.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

[Pete Becker]

Re: float representation and precision

Rolf Magnus wrote:
[color=blue]
>
> Floating point numbers approximate a continuous range of numbers. Since that
> range has an infinite count of different numbers, it isn't possible to
> store every number exactly in a computer.[/color]

It doesn't even have to be infinite, just more than the floating point
type can hold.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

[Rolf Magnus]

Re: float representation and precision

Pete Becker wrote:
[color=blue]
> Frederick Bruckman wrote:
>[color=green]
>>
>> Oops. It's log-2.
>>[/color]
>
> No, it's not. It's just like 1.3 x 10^3, but in base 2 instead of base
> 10. The exponent is the logarithm of the value, but the fraction is just
> the representation of the scaled value in the appropriate base. So 1/2
> is represented in binary floating point as .1 base 2,[/color]

No. It's represented as 1 * 2^-1. The mantissa always is >=1 and <2, similar
to what you would do in base 10. You don't write 0.13 x 10^4, but 1.3 *
10^3.
[color=blue]
> 1 is represented as .1 x 2^1, and 2 is represented as .1 x 2^2.[/color]

1 is represented as 1 * 2^0, 2 as 1 * 2^1.

Btw: How would you calculate the exponent from a given number?

[Pete Becker]

Re: float representation and precision

Rolf Magnus wrote:[color=blue]
> Pete Becker wrote:
>
>[color=green]
>>Frederick Bruckman wrote:
>>
>>[color=darkred]
>>>Oops. It's log-2.
>>>[/color]
>>
>>No, it's not. It's just like 1.3 x 10^3, but in base 2 instead of base
>>10. The exponent is the logarithm of the value, but the fraction is just
>>the representation of the scaled value in the appropriate base. So 1/2
>>is represented in binary floating point as .1 base 2,[/color]
>
>
> No. It's represented as 1 * 2^-1. The mantissa always is >=1 and <2,[/color]

Not under IEEE 754 and its successors. Normalized non-zero fractions are
always less than 1 and greater than or equal to 1/base. This gets
confusing, though, because some architectures (in particular, Intel)
suppress the leading bit, since it's always 1. But when you stick it
back in, the fraction (as the name implies <g>) is always less than 1.

--

Pete Becker
Dinkumware, Ltd. (http://www.dinkumware.com)

[Frederick Bruckman]

Re: float representation and precision

In article <d5ijlb$mbi$03$ 1@news.t-online.com>,
Rolf Magnus <ramagnus@t-online.de> writes:[color=blue]
> Frederick Bruckman wrote:
>[color=green]
>> In any case, that's not the reason for the loss of precision in
>> converting to floating point and back. What's "stored in binary"
>> is not the number itself, but the natural logarithm of the number.[/color]
>
> Where did you get that idea from? Floating point numbers are usually stored
> in base 2, not base e. I have seen a format where they were stored in base
> 16, but never e.[/color]

I don't know where I got that idea from. It might be a nice way
to represent numbers on a scientific calculator -- many of the
"interestin g" irrational numbers would be repeating decimals in
base _e_, but I don't know that anyone actually does that. Never
mind.

--
Frederick

[Andrey Tarasevich]

Re: float representation and precision

Sebastian wrote:[color=blue]
> ...
> yesterday I discovered something that won't come as a surprise to more experienced C++ - programmers - but I was genuinely amazed. I had this line of code:
>
> float x = 0.9f;
>
> and when I had a look at it in the VS.NET 2003 debugger, it said x had a value of 0.89999998 ! When I printf'ed x to a logfile, it was 0.8999999761581 420900000000000 00. Now, why on earth is that? I understand that the results of floatingpoint operations will always be somewhat imprecise due to machine limitations, but 0.9? Can anybody explain this to me?
> ...[/color]

Decimal 0.9 is

0.1110011001100 11001100... = 0.1(1100)

in binary positional notation. The last part - 1100 - is repeated
indefinitely. This means that a binary machine that uses some form of
positional notation for floating-point numbers and does not provide any
means for representing periodic fractions won't be able to represent 0.9
precisely. The same applies to 0.1, 0.2, 0.8. That's what you observe in
your case.

--
Best regards,
Andrey Tarasevich