Meaning of Double.Epsilon

[Daniel]

I am grappling with the idea of double.Epsilon. I have written the
following test:

[Test]
public void FuzzyDivisionTe st()
{
double a = 0.33333d;
double b = 1d / 3d;

Assert.IsFalse( a == b, "Built-in == operator should not be
fuzzy");
Assert.IsTrue( a-b < double.Epsilon) ;
Assert.IsTrue( Math.Abs(a-b) < double.Epsilon) ;
Assert.IsTrue( Math.Abs(b-a) < double.Epsilon) ;
}

However, something is weird. The first 2 assertions pass, however the
last 2 fail. Is Math.Abs() losing my Epsilon value? Is that correct?

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Daniel <dhchait@yahoo. com> wrote:[color=blue]
> I am grappling with the idea of double.Epsilon. I have written the
> following test:
>
> [Test]
> public void FuzzyDivisionTe st()
> {
> double a = 0.33333d;
> double b = 1d / 3d;
>
> Assert.IsFalse( a == b, "Built-in == operator should not be
> fuzzy");
> Assert.IsTrue( a-b < double.Epsilon) ;
> Assert.IsTrue( Math.Abs(a-b) < double.Epsilon) ;
> Assert.IsTrue( Math.Abs(b-a) < double.Epsilon) ;
> }
>
> However, something is weird. The first 2 assertions pass, however the
> last 2 fail. Is Math.Abs() losing my Epsilon value? Is that correct?[/color]

Epsilon is the smallest positive non-zero value that a double can hold.
It will never be true (I *think*) that a != b and
Math.Abs(b-a) < double.Epsilon.

a-b is less than double.Epsilon because it's negative.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Jon]

Re: Meaning of Double.Epsilon

My interpretation of the Visual Studio help on this is that this reply is
partially correct. It is possible that
a != b
and
Math.Abs(b - a) < double.Epsilon

However the difference between a and b in your example is evidently larger
than double.Epsilon - it is 3.3333333(recur ring)e-10, and double.Epsilon ==
4.9406564584124 7e-324

As the previous reply said, the reason the second assertion passes is that
a - b is negative.

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in message
news:MPG.1ba7eb 35102ad3eb98b35 a@msnews.micros oft.com...[color=blue]
> Daniel <dhchait@yahoo. com> wrote:[color=green]
> > I am grappling with the idea of double.Epsilon. I have written the
> > following test:
> >
> > [Test]
> > public void FuzzyDivisionTe st()
> > {
> > double a = 0.33333d;
> > double b = 1d / 3d;
> >
> > Assert.IsFalse( a == b, "Built-in == operator should not be
> > fuzzy");
> > Assert.IsTrue( a-b < double.Epsilon) ;
> > Assert.IsTrue( Math.Abs(a-b) < double.Epsilon) ;
> > Assert.IsTrue( Math.Abs(b-a) < double.Epsilon) ;
> > }
> >
> > However, something is weird. The first 2 assertions pass, however the
> > last 2 fail. Is Math.Abs() losing my Epsilon value? Is that correct?[/color]
>
> Epsilon is the smallest positive non-zero value that a double can hold.
> It will never be true (I *think*) that a != b and
> Math.Abs(b-a) < double.Epsilon.
>
> a-b is less than double.Epsilon because it's negative.
>
> --
> Jon Skeet - <skeet@pobox.co m>
> http://www.pobox.com/~skeet
> If replying to the group, please do not mail me too[/color]

[Niki Estner]

Re: Meaning of Double.Epsilon

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in
news:MPG.1ba7eb 35102ad3eb98b35 a@msnews.micros oft.com...[color=blue]
> Daniel <dhchait@yahoo. com> wrote:[color=green]
>> I am grappling with the idea of double.Epsilon. I have written the
>> following test:
>>
>> [Test]
>> public void FuzzyDivisionTe st()
>> {
>> double a = 0.33333d;
>> double b = 1d / 3d;
>>
>> Assert.IsFalse( a == b, "Built-in == operator should not be
>> fuzzy");
>> Assert.IsTrue( a-b < double.Epsilon) ;
>> Assert.IsTrue( Math.Abs(a-b) < double.Epsilon) ;
>> Assert.IsTrue( Math.Abs(b-a) < double.Epsilon) ;
>> }
>>
>> However, something is weird. The first 2 assertions pass, however the
>> last 2 fail. Is Math.Abs() losing my Epsilon value? Is that correct?[/color]
>
> Epsilon is the smallest positive non-zero value that a double can hold.
> It will never be true (I *think*) that a != b and
> Math.Abs(b-a) < double.Epsilon.[/color]

Yes, I think the docs about Double.Epsilon are actually wrong; Quote:
"Instead [of comparing double values with ==], determine if the two sides of
a comparison are close enough to equal for your purposes by comparing
whether the absolute value of the difference between the left and right-hand
sides is less than Epsilon."

I usually use something like "Math.Abs(a-b) < 1e-10", but I'm not sure if
this is common practice.

Niki

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Jon <nospam@nospame ver.com> wrote:[color=blue]
> My interpretation of the Visual Studio help on this is that this reply is
> partially correct. It is possible that
> a != b
> and
> Math.Abs(b - a) < double.Epsilon[/color]

Could you give an example? I believe that

Math.Abs(b-a) < double.Epsilon => b-a != 0 => b==a

It's the last step that I'm not sure about though - is it possible for
a and b to be different, but their difference to be so close to 0 as to
be unrepresentably small? It feels unlikely, but I know that intuition
is often horribly flawed when it comes to floating point maths.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Niki Estner <niki.estner@cu be.net> wrote:[color=blue][color=green]
> > Epsilon is the smallest positive non-zero value that a double can hold.
> > It will never be true (I *think*) that a != b and
> > Math.Abs(b-a) < double.Epsilon.[/color]
>
> Yes, I think the docs about Double.Epsilon are actually wrong; Quote:
> "Instead [of comparing double values with ==], determine if the two sides of
> a comparison are close enough to equal for your purposes by comparing
> whether the absolute value of the difference between the left and right-hand
> sides is less than Epsilon."
>
> I usually use something like "Math.Abs(a-b) < 1e-10", but I'm not sure if
> this is common practice.[/color]

Yup - that seems a much better idea. Do you want to report the problem
to MS, or shall I?

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Niki Estner]

Re: Meaning of Double.Epsilon

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in
news:MPG.1ba801 913dac652298b35 b@msnews.micros oft.com...[color=blue]
> Jon <nospam@nospame ver.com> wrote:[color=green]
>> My interpretation of the Visual Studio help on this is that this reply is
>> partially correct. It is possible that
>> a != b
>> and
>> Math.Abs(b - a) < double.Epsilon[/color]
>
> Could you give an example? I believe that
>
> Math.Abs(b-a) < double.Epsilon => b-a != 0 => b==a
>
> It's the last step that I'm not sure about though - is it possible for
> a and b to be different, but their difference to be so close to 0 as to
> be unrepresentably small? It feels unlikely, but I know that intuition
> is often horribly flawed when it comes to floating point maths.[/color]

I guess it's possible if a and b are stored in 80-bit FPU registers: b-a (80
bit) could still be different form 0 (using an 80-bit comparison), but the
difference would be smaller than double.Epsilon. Also Math.Abs(b-a) would
actually return 0 (if not inlined) as b-a would get casted to a (64-bit)
double value.
Sample code (compares (double.Epsilon/2) with 0):

using System;
public class MyClass
{
public static void Main()
{
double a = double.Parse("2 ");
Console.WriteLi ne(0 == (double.Epsilon/a));
Console.WriteLi ne((double.Epsi lon/a-0) < double.Epsilon) ;
}
}

Niki

[Niki Estner]

Re: Meaning of Double.Epsilon

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in
news:MPG.1ba801 b7ca273b7298b35 c@msnews.micros oft.com...[color=blue]
> Niki Estner <niki.estner@cu be.net> wrote:[color=green][color=darkred]
>> > Epsilon is the smallest positive non-zero value that a double can hold.
>> > It will never be true (I *think*) that a != b and
>> > Math.Abs(b-a) < double.Epsilon.[/color]
>>
>> Yes, I think the docs about Double.Epsilon are actually wrong; Quote:
>> "Instead [of comparing double values with ==], determine if the two sides
>> of
>> a comparison are close enough to equal for your purposes by comparing
>> whether the absolute value of the difference between the left and
>> right-hand
>> sides is less than Epsilon."
>>
>> I usually use something like "Math.Abs(a-b) < 1e-10", but I'm not sure if
>> this is common practice.[/color]
>
> Yup - that seems a much better idea. Do you want to report the problem
> to MS, or shall I?[/color]

You're the MVP ;-)

Niki

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Niki Estner <niki.estner@cu be.net> wrote:[color=blue][color=green]
> > It's the last step that I'm not sure about though - is it possible for
> > a and b to be different, but their difference to be so close to 0 as to
> > be unrepresentably small? It feels unlikely, but I know that intuition
> > is often horribly flawed when it comes to floating point maths.[/color]
>
> I guess it's possible if a and b are stored in 80-bit FPU registers: b-a (80
> bit) could still be different form 0 (using an 80-bit comparison), but the
> difference would be smaller than double.Epsilon. Also Math.Abs(b-a) would
> actually return 0 (if not inlined) as b-a would get casted to a (64-bit)
> double value.[/color]

Aargh, yes. I knew there was a good reason to be hesitant :)
[color=blue]
> Sample code (compares (double.Epsilon/2) with 0):[/color]

<snip>

Good, thanks. Here's something interesting, based on your code:

using System;
public class MyClass
{
public static void Main()
{
double two = double.Parse("2 ");
double a = double.Epsilon/two;
double b = 0;
Console.WriteLi ne(a==b);
Console.WriteLi ne(Math.Abs(b-a) < double.Epsilon) ;
}
}

That prints out (on my box):
True
True

If you comment out the last line, however, it just prints out
False

Presumably the call to Math.Abs forces a and b to be "normal" 64-bit
values rather than just 80-bit registers, or something like that.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Niki Estner <niki.estner@cu be.net> wrote:[color=blue][color=green]
> > Yup - that seems a much better idea. Do you want to report the problem
> > to MS, or shall I?[/color]
>
> You're the MVP ;-)[/color]

Righto. (Admittedly I didn't have any luck persuading them that
System.Decimal is a floating point type rather than a fixed point type
- although reporting it again with the online bug database for 2.0,
it's been accepted!)

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Niki Estner]

Re: Meaning of Double.Epsilon

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in
news:MPG.1ba815 e2cc7c8ef698b35 e@msnews.micros oft.com...[color=blue]
> Niki Estner <niki.estner@cu be.net> wrote:
> <snip>
> Good, thanks. Here's something interesting, based on your code:
>
> using System;
> public class MyClass
> {
> public static void Main()
> {
> double two = double.Parse("2 ");
> double a = double.Epsilon/two;
> double b = 0;
> Console.WriteLi ne(a==b);
> Console.WriteLi ne(Math.Abs(b-a) < double.Epsilon) ;
> }
> }
>
> That prints out (on my box):
> True
> True
>
> If you comment out the last line, however, it just prints out
> False
>
> Presumably the call to Math.Abs forces a and b to be "normal" 64-bit
> values rather than just 80-bit registers, or something like that.[/color]

Yes, I never really understood when variables get enregistered and when not.
And the fact that the rules change as soon as I attach a debugger doesn't
make it easier either...

Niki

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Niki Estner <niki.estner@cu be.net> wrote:[color=blue][color=green]
> > Presumably the call to Math.Abs forces a and b to be "normal" 64-bit
> > values rather than just 80-bit registers, or something like that.[/color]
>
> Yes, I never really understood when variables get enregistered and when not.
> And the fact that the rules change as soon as I attach a debugger doesn't
> make it easier either...[/color]

I'm glad I'm not the only one - your previous posts suggest that you
know far more about JIT optimisation than I'm ever likely to :)

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Niki Estner]

Re: Meaning of Double.Epsilon

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in
news:MPG.1ba82b de1dee7b1798b36 2@msnews.micros oft.com...[color=blue]
> Niki Estner <niki.estner@cu be.net> wrote:[color=green][color=darkred]
>> > Presumably the call to Math.Abs forces a and b to be "normal" 64-bit
>> > values rather than just 80-bit registers, or something like that.[/color]
>>
>> Yes, I never really understood when variables get enregistered and when
>> not.
>> And the fact that the rules change as soon as I attach a debugger doesn't
>> make it easier either...[/color]
>
> I'm glad I'm not the only one - your previous posts suggest that you
> know far more about JIT optimisation than I'm ever likely to :)[/color]

Thanks a lot :-)
Unfortunately I'm guessing blindly more often than I'd like to, too...

I think it would be a wise decision if MS would at least release a little
more documentation on the internals of the JIT: Many developers simply
assume it doesn't optimize at all, others try to "optimize" their code and
break real optimizations that way (register-variables are quite fragile) or
create unmaintainable code. At least a few rules of thumb (when will it
remove range checking? when will it inline a function? when will it
enregister a variable? etc.) would be very helpful here...

On the other hand, we probably wouldn't have any use for cordbg any more
then; Wouldn't that be a shame?

Niki

[Daniel]

Re: Meaning of Double.Epsilon

Jon - thanks for the reply, I was hoping to hear from you or John
Bentley, the recognized leaders on the internet in this area!

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

[cody]

Re: Meaning of Double.Epsilon

I tend to say that operators == result is undefined for floating point types
(except decimal).
You never know how variables are stored in memory. testing for equality for
a floating point variables never makes sense, even if you do:

float a = 1f;
float b = 1f;
if (a==b) { }

The Jitter may choose to place a and b in difference register types (32, 64
or 80 bits), so a==b may return true or false depending on the machine type,
framework version, debug or release settings or if a debugger is attached or
in which way the variables are used in code or wheather the method is
inlined or not.

So it would be great if the compiler would at least issue a warning if it
spots code which tests for equality with floating point values.

The only thing I could imagine is testing for NaN but in that case one
should use IsNaN() methods of struct Double and Single.

--
cody

Freeware Tools, Games and Humour
http://www.deutronium.de.vu || http://www.deutronium.tk
"Jon Skeet [C# MVP]" <skeet@pobox.co m> schrieb im Newsbeitrag
news:MPG.1ba815 e2cc7c8ef698b35 e@msnews.micros oft.com...[color=blue]
> Niki Estner <niki.estner@cu be.net> wrote:[color=green][color=darkred]
> > > It's the last step that I'm not sure about though - is it possible for
> > > a and b to be different, but their difference to be so close to 0 as[/color][/color][/color]
to[color=blue][color=green][color=darkred]
> > > be unrepresentably small? It feels unlikely, but I know that intuition
> > > is often horribly flawed when it comes to floating point maths.[/color]
> >
> > I guess it's possible if a and b are stored in 80-bit FPU registers: b-a[/color][/color]
(80[color=blue][color=green]
> > bit) could still be different form 0 (using an 80-bit comparison), but[/color][/color]
the[color=blue][color=green]
> > difference would be smaller than double.Epsilon. Also Math.Abs(b-a)[/color][/color]
would[color=blue][color=green]
> > actually return 0 (if not inlined) as b-a would get casted to a (64-bit)
> > double value.[/color]
>
> Aargh, yes. I knew there was a good reason to be hesitant :)
>[color=green]
> > Sample code (compares (double.Epsilon/2) with 0):[/color]
>
> <snip>
>
> Good, thanks. Here's something interesting, based on your code:
>
> using System;
> public class MyClass
> {
> public static void Main()
> {
> double two = double.Parse("2 ");
> double a = double.Epsilon/two;
> double b = 0;
> Console.WriteLi ne(a==b);
> Console.WriteLi ne(Math.Abs(b-a) < double.Epsilon) ;
> }
> }
>
> That prints out (on my box):
> True
> True
>
> If you comment out the last line, however, it just prints out
> False
>
> Presumably the call to Math.Abs forces a and b to be "normal" 64-bit
> values rather than just 80-bit registers, or something like that.
>
> --
> Jon Skeet - <skeet@pobox.co m>
> http://www.pobox.com/~skeet
> If replying to the group, please do not mail me too[/color]