Double is doing incorrect subtraction

[Nathan Sokalski]

I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.3329999999999 98

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
--
Nathan Sokalski
njsokalski@hotm ail.com

http://www.nathansokalski.com/

[=?Utf-8?B?RmFtaWx5IFRyZWUgTWlrZQ==?=]

RE: Double is doing incorrect subtraction

Consider using decimal instead of double if you want to get the answer you
expect below.

"Nathan Sokalski" wrote:

I have the following code:
>
Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()
>
The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:
>
0.3329999999999 98
>
I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
--
Nathan Sokalski
njsokalski@hotm ail.com

http://www.nathansokalski.com/

>
>
>

[Erik Funkenbusch]

Re: Double is doing incorrect subtraction

On Sat, 12 Jan 2008 14:53:22 -0500, Nathan Sokalski wrote:

I have the following code:
>
Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()
>
The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:
>
0.3329999999999 98
>
I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?

Double is doing entirely correct subtraction.

The problem is that Double is an implementation of IEEE floating point, and
IEEE floating point is incapable of representing all numbers exactly.

Read this:

http://support.microsoft.com/kb/42980

[Cor Ligthert[MVP]]

Re: Double is doing incorrect subtraction

Nathan,

This is basic

Float, Double or whatever floating poing calculation for mathimatical
programming,
Decimal for business programming,

Cor

[Herfried K. Wagner [MVP]]

Re: Double is doing incorrect subtraction

"Nathan Sokalski" <njsokalski@hot mail.comschrieb :

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()
>
The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:
>
0.3329999999999 98
>
I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?

In addition to the other replies, take a look at these articles:

IEEE Standard 754 Floating Point Numbers
<URL:http://steve.hollasch. net/cgindex/coding/ieeefloat.html>

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://dotnet.mvps.org/dotnet/faqs/>