Meaning of Double.Epsilon

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

cody <please_dont.sp am.deutronium@g mx.de> wrote:[color=blue][color=green][color=darkred]
> > > If the result is from a calculation, that's right. I believe that if
> > > it's from a constant, or if it's from a calculation which has an answer
> > > which can be exactly represented in the chosen data size, it's fine.[/color][/color]
>
> But in Jons example the calculation can be left can certainly out without
> altering the meaning the program because it compares the results of two same
> calculations using constants only.[/color]

The calculation *uses* constants, but the result of it isn't
expressable exactly in 64 bits. That's the difference - any constant is
basically defined to be "the 64-bit (or 32-bit in case of float) value
closest to what is literally specified". It would be an error (I
believe) for an enregistered variable to take a more accurate value.
[color=blue][color=green]
> > Constant are ok, but calculation results usually aren't safe:
> > Console.WriteLi ne(Math.Sqrt(2) *Math.Sqrt(2) == 2);
> > prints out false on my machine.[/color]
>
> I wouldn't have expected any other result than a random value from 'true'
> and 'false' :)
>
> I always have been thinking of floating point values as "fuzzy things" where
> the exact representation of the result of a calculation is not foreseeable.
> Maybe there are IEEE standards but who knows where some machines calculates
> floating point in their own way? I do not care as long as the result comes
> very close to the "real" exact result.[/color]

I believe the result of a particular calculation is and can be
foreseeable. The difficulty here isn't with the calculation, or even
*how* the result is truncated to 64 bits - it's *when* the result is
truncated.
[color=blue][color=green][color=darkred]
> > > I suspect it's very, very rarely a good idea to use == for comparisons
> > > with non-constants in the first place though.[/color]
> >
> > I'd definitely agree to that.[/color]
>
> Who wouldn't agree with that (except the microsoft guy who decided that
> operator== for fp do not need a compiler warning).[/color]

I'm in two minds about that, personally. I can see the argument for a
warning, but at the same time it feels odd.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[cody]

Re: Meaning of Double.Epsilon

> I believe the result of a particular calculation is and can be[color=blue]
> foreseeable. The difficulty here isn't with the calculation, or even
> *how* the result is truncated to 64 bits - it's *when* the result is
> truncated.[/color]


This is imho absolutely unforeseeable and depends on the cpu, the jitter and
so on.
[color=blue][color=green][color=darkred]
> > > > I suspect it's very, very rarely a good idea to use == for[/color][/color][/color]
comparisons[color=blue][color=green][color=darkred]
> > > > with non-constants in the first place though.
> > >
> > > I'd definitely agree to that.[/color]
> >
> > Who wouldn't agree with that (except the microsoft guy who decided that
> > operator== for fp do not need a compiler warning).[/color]
>
> I'm in two minds about that, personally. I can see the argument for a
> warning, but at the same time it feels odd.[/color]

It may feel a bit odd but I do not think one of us ever used == on a
floating point value in a real application.
The only thing I can think of is if (a!=a) to test for NaN but we have
IsNan() for that.

--
cody

[Freeware, Games and Humor]
www.deutronium.de.vu || www.deutronium.tk

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

cody <please_dont.sp am.deutronium@g mx.de> wrote:[color=blue][color=green]
> > I believe the result of a particular calculation is and can be
> > foreseeable. The difficulty here isn't with the calculation, or even
> > *how* the result is truncated to 64 bits - it's *when* the result is
> > truncated.[/color]
>
> This is imho absolutely unforeseeable and depends on the cpu, the
> jitter and so on.[/color]

Yes - but the point I'm making is that it's not the same as the
calculation itself varying in its results.
[color=blue][color=green]
> > I'm in two minds about that, personally. I can see the argument for a
> > warning, but at the same time it feels odd.[/color]
>
> It may feel a bit odd but I do not think one of us ever used == on a
> floating point value in a real application.
> The only thing I can think of is if (a!=a) to test for NaN but we have
> IsNan() for that.[/color]

Sure.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Niki Estner]

Re: Meaning of Double.Epsilon

"cody" <please_dont.sp am.deutronium@g mx.de> wrote in
news:Og8paWslEH A.2500@TK2MSFTN GP09.phx.gbl...[color=blue]
> ...[color=green][color=darkred]
>> > Who wouldn't agree with that (except the microsoft guy who decided that
>> > operator== for fp do not need a compiler warning).[/color]
>>
>> I'm in two minds about that, personally. I can see the argument for a
>> warning, but at the same time it feels odd.[/color]
>
> It may feel a bit odd but I do not think one of us ever used == on a
> floating point value in a real application.
> The only thing I can think of is if (a!=a) to test for NaN but we have
> IsNan() for that.[/color]

That's the point I've already tried to say: I *do* use tests like "if (a
== -1) a = DoLazyCalculati on();" and the like; I'd also guess the equality
operator is implicitly used when you use doubles as keys in a hashtable, or
sort an array of doubles (at least with generics, things like this should
trigger your warning, too). I don't think it would be a good idea to have a
warning *every* time the == operator is used on fp values;

BTW: If you "forbid" the == operator, wouldn't it make sense to apply the
same strictness to other comparison operators? After all, everything said
applies also to comparisons like "Math.Sqrt(2)*M ath.Sqrt(2) < 2" as well.

Niki

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Niki Estner <niki.estner@cu be.net> wrote:[color=blue][color=green]
> > It may feel a bit odd but I do not think one of us ever used == on a
> > floating point value in a real application.
> > The only thing I can think of is if (a!=a) to test for NaN but we have
> > IsNan() for that.[/color]
>
> That's the point I've already tried to say: I *do* use tests like "if (a
> == -1) a = DoLazyCalculati on();" and the like; I'd also guess the equality
> operator is implicitly used when you use doubles as keys in a hashtable, or
> sort an array of doubles (at least with generics, things like this should
> trigger your warning, too).[/color]

I don't think the equality operator will be used - the Equals method
will be used. There are subtle differences.
[color=blue]
> I don't think it would be a good idea to have a
> warning *every* time the == operator is used on fp values;
>
> BTW: If you "forbid" the == operator, wouldn't it make sense to apply the
> same strictness to other comparison operators? After all, everything said
> applies also to comparisons like "Math.Sqrt(2)*M ath.Sqrt(2) < 2" as well.[/color]

Hmm... interesting point.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[Niki Estner]

Re: Meaning of Double.Epsilon

"Jon Skeet [C# MVP]" <skeet@pobox.co m> wrote in
news:MPG.1bab7e aa3a234c7198b3c 1@msnews.micros oft.com...[color=blue]
> Niki Estner <niki.estner@cu be.net> wrote:[color=green][color=darkred]
>> > It may feel a bit odd but I do not think one of us ever used == on a
>> > floating point value in a real application.
>> > The only thing I can think of is if (a!=a) to test for NaN but we have
>> > IsNan() for that.[/color]
>>
>> That's the point I've already tried to say: I *do* use tests like "if (a
>> == -1) a = DoLazyCalculati on();" and the like; I'd also guess the
>> equality
>> operator is implicitly used when you use doubles as keys in a hashtable,
>> or
>> sort an array of doubles (at least with generics, things like this should
>> trigger your warning, too).[/color]
>
> I don't think the equality operator will be used - the Equals method
> will be used. There are subtle differences.[/color]

Yes, you're probably right: After all, support for operators was one of the
weaknesses of .net generics. I just remembered how many compiler
errors/warnings I used to have with C++ generics if someone decided to
implement standard operators in a non-standard way...

But if "==" would issue a warning, while double.Equals would not, wouldn't
that be even more strange? After all, double.Equals will always return the
same value.

The more I think about it, the more I dislike that
"fp-equality-warning"-idea...

Niki

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

Niki Estner <niki.estner@cu be.net> wrote:

<snip other stuff>
[color=blue]
> But if "==" would issue a warning, while double.Equals would not, wouldn't
> that be even more strange? After all, double.Equals will always return the
> same value.[/color]

Nope - Double.Equals is different to == for NaNs. IEEE specifies that
NaN != NaN, but Object.Equals specifies that x.Equals(x) is true for
all x.

From section 8.2.5.2 of the CLI spec:

<quote>
Note: Although two floating point NaNs are defined by IEC 60559:1989 to
always compare as unequal, the contract for System.Object.E quals,
requires that overrides must satisfy the requirements for an
equivalence operator. Therefore, System.Double.E quals and
System.Single.E quals return True when comparing two NaNs, while the
equality operator returns False in that case, as required by the
standard.
</quote>

Very odd.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[cody]

Re: Meaning of Double.Epsilon

> That's the point I've already tried to say: I *do* use tests like "if (a[color=blue]
> == -1) a = DoLazyCalculati on();" and the like; I'd also guess the equality
> operator is implicitly used when you use doubles as keys in a hashtable,[/color]
or[color=blue]
> sort an array of doubles (at least with generics, things like this should
> trigger your warning, too). I don't think it would be a good idea to have[/color]
a[color=blue]
> warning *every* time the == operator is used on fp values;[/color]

using -1 is imho a bad idea. what if you actually need -1 in your app?
Negative numbers are very common in floating point (in contrast to
integers). What if you inadvertantly use the float variable with its -1
value?
NaN is just for that purpose: If a value is NaN it stays NaN even if used in
calculations or converted from double to float and vice versa and this way,
you know that something gone wrong if you see NaN in the output.
The following code displays 4 times NaN:

float f = float.NaN;
double d = double.NaN;
Console.WriteLi ne(double.IsNaN ((float)d));
Console.WriteLi ne(double.IsNaN ((double)f));
Console.WriteLi ne(float.IsNaN( (float)d));
Console.WriteLi ne(float.IsNaN( (float)(double) f));

The funny thing is that if you compare a float with and int value -1 the
result returns true even if its value is not -1. The following prints
"hello":

float f1 = -1.00000001f;
float f2 = -0.99999999f;
if (f1==-1 && f2==-1)
{
Console.WriteLi ne("hello");
}

Btw: using floating point values as hashtable keys seems to be a bad idea.
Even when flooring and ceiling values it doesn't work. The following outputs
2 times false:

float f1 = -0.99999991f;
float f2 = -0.99999999f;
float f3 = 0.99999991f;
float f4 = 0.99999999f;
Console.WriteLi ne(Math.Ceiling (f1)==Math.Ceil ing(f2));
Console.WriteLi ne(Math.Floor(f 3)==Math.Floor( f4));

Only using Math.Round() seems always to work correctly.
[color=blue]
> BTW: If you "forbid" the == operator, wouldn't it make sense to apply the
> same strictness to other comparison operators? After all, everything said
> applies also to comparisons like "Math.Sqrt(2)*M ath.Sqrt(2) < 2" as well.[/color]

Very strange I'd expect the value be smaller as 2 than bigger but it shows
that you cannot rely on exact representation. But I'd never go so far to
issue warnings on all comparison operators for floating point because it
would be simply plain stupid.

Keep in mind that if (f1 < f2) is used very commonly and works fine in
almost every situation (except the programmer makes a wrong assumtion as you
example shows), But in constrast if (f1==f2) will be undefined unless you
use *constants* for *both* values and I cannot imagine a scenario which
requires this which cannot be solved with a better method.

Whatever is said in this thread, one should never assume an exact
representation of floating point variables
even if it seems to work with some values if could fail with other ones or
fail on another platform.

--
cody

[Freeware, Games and Humor]
www.deutronium.de.vu || www.deutronium.tk

[cody]

Re: Meaning of Double.Epsilon

> > But if "==" would issue a warning, while double.Equals would not,
wouldn't[color=blue][color=green]
> > that be even more strange? After all, double.Equals will always return[/color][/color]
the[color=blue][color=green]
> > same value.[/color]
>
> Nope - Double.Equals is different to == for NaNs. IEEE specifies that
> NaN != NaN, but Object.Equals specifies that x.Equals(x) is true for
> all x.[/color]


If == isssues a warning, double.Equals() also should.

Also bare in mind that a Warning doesn't mean "It is wrong" but "It could
possibly be wrong", otherwise it would be an Error.

--
cody

[Freeware, Games and Humor]
www.deutronium.de.vu || www.deutronium.tk

[Niki Estner]

Re: Meaning of Double.Epsilon

"cody" <please_dont.sp am.deutronium@g mx.de> wrote in
news:e0VDug2lEH A.1652@TK2MSFTN GP09.phx.gbl...[color=blue][color=green]
>> That's the point I've already tried to say: I *do* use tests like "if (a
>> == -1) a = DoLazyCalculati on();" and the like; I'd also guess the
>> equality
>> operator is implicitly used when you use doubles as keys in a hashtable,[/color]
> or[color=green]
>> sort an array of doubles (at least with generics, things like this should
>> trigger your warning, too). I don't think it would be a good idea to have[/color]
> a[color=green]
>> warning *every* time the == operator is used on fp values;[/color]
>
> using -1 is imho a bad idea. what if you actually need -1 in your app?
> Negative numbers are very common in floating point (in contrast to
> integers). What if you inadvertantly use the float variable with its -1
> value?
> NaN is just for that purpose: If a value is NaN it stays NaN even if used
> in
> calculations or converted from double to float and vice versa and this
> way,
> you know that something gone wrong if you see NaN in the output.
> The following code displays 4 times NaN:[/color]

I think this is going slightly off-topic now, but of course "special values"
like these have to be values that cannot be result of a calculation.
Depending on the meaning of the value this could be a negative value (e.g
distances), NaN, a value above 100 (percentages), or whatever. Although NaN
is a good "special value", too, it does have disadvantages, sometimes: AFAIK
Double.Parse can't parse it, converting it to an int will throw an
exception, you only have one single special value, and sometimes NaN can be
result of a calculation...

Niki

[Niki Estner]

Re: Meaning of Double.Epsilon

"cody" <please_dont.sp am.deutronium@g mx.de> wrote in
news:%23v3Mwk2l EHA.3260@TK2MSF TNGP09.phx.gbl. ..[color=blue][color=green][color=darkred]
>> > But if "==" would issue a warning, while double.Equals would not,[/color][/color]
> wouldn't[color=green][color=darkred]
>> > that be even more strange? After all, double.Equals will always return[/color][/color]
> the[color=green][color=darkred]
>> > same value.[/color]
>>
>> Nope - Double.Equals is different to == for NaNs. IEEE specifies that
>> NaN != NaN, but Object.Equals specifies that x.Equals(x) is true for
>> all x.[/color]
>
>
> If == isssues a warning, double.Equals() also should.
>
> Also bare in mind that a Warning doesn't mean "It is wrong" but "It could
> possibly be wrong", otherwise it would be an Error.[/color]

I think that's a misunderstandin g. Here are a few warnings I've randomly
picked:
CS0028 'function declaration' has the wrong signature to be an entry point
CS0105 The using directive for 'namespace' appeared previously in this
namespace
CS0162 Unreachable code detected
CS0251 Indexing an array with a negative index (array indices always start
at zero)
CS0628 'member' : new protected member declared in sealed class
CS0649 Field 'field' is never assigned to, and will always have its default
value 'value'
CS1522 Empty switch block
CS1591 Missing XML comment for publicly visible type or member
'Type_or_Member '
CS5000 Unknown compiler option '/option'

These do suggest that a warning acually means "this code can be compiled,
but it makes no sense and will either do nothing, or crash". I don't think a
fp comparison would fit in that category.

Niki

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

cody <please_dont.sp am.deutronium@g mx.de> wrote:[color=blue]
> The funny thing is that if you compare a float with and int value -1 the
> result returns true even if its value is not -1. The following prints
> "hello":
>
> float f1 = -1.00000001f;
> float f2 = -0.99999999f;
> if (f1==-1 && f2==-1)
> {
> Console.WriteLi ne("hello");
> }[/color]

There's nothing funny about that - the nearest float to those literals
is exactly -1. Have a look in the IL. That explains the Ceiling/Floor
problems you were having too.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too

[cody]

Re: Meaning of Double.Epsilon

> >> That's the point I've already tried to say: I *do* use tests like "if
(a[color=blue][color=green][color=darkred]
> >> == -1) a = DoLazyCalculati on();" and the like; I'd also guess the
> >> equality
> >> operator is implicitly used when you use doubles as keys in a[/color][/color][/color]
hashtable,[color=blue][color=green]
> > or[color=darkred]
> >> sort an array of doubles (at least with generics, things like this[/color][/color][/color]
should[color=blue][color=green][color=darkred]
> >> trigger your warning, too). I don't think it would be a good idea to[/color][/color][/color]
have[color=blue][color=green]
> > a[color=darkred]
> >> warning *every* time the == operator is used on fp values;[/color]
> >
> > using -1 is imho a bad idea. what if you actually need -1 in your app?
> > Negative numbers are very common in floating point (in contrast to
> > integers). What if you inadvertantly use the float variable with its -1
> > value?
> > NaN is just for that purpose: If a value is NaN it stays NaN even if[/color][/color]
used[color=blue][color=green]
> > in
> > calculations or converted from double to float and vice versa and this
> > way,
> > you know that something gone wrong if you see NaN in the output.
> > The following code displays 4 times NaN:[/color]
>
> I think this is going slightly off-topic now, but of course "special[/color]
values"[color=blue]
> like these have to be values that cannot be result of a calculation.
> Depending on the meaning of the value this could be a negative value (e.g
> distances), NaN, a value above 100 (percentages), or whatever. Although[/color]
NaN[color=blue]
> is a good "special value", too, it does have disadvantages, sometimes:[/color]
AFAIK[color=blue]
> Double.Parse can't parse it, converting it to an int will throw an
> exception,[/color]

If you use it as special value to mean "uninitialized" , you certainly won't
parse it or convert it to int.
[color=blue]
> you only have one single special value,[/color]

otherwise it wouldn't be special :)
[color=blue]
> and sometimes NaN can be result of a calculation...[/color]

Indeed, NaN can be result of a calculation whereas -1 can't :)))

In fact, special values(besides NaN,NegativeInf ,PositiveInf) in floating
point sounds a bit like an oxymoron (ok it isn't). But maybe special ranges
are ok.

--
cody

[Freeware, Games and Humor]
www.deutronium.de.vu || www.deutronium.tk

[cody]

Re: Meaning of Double.Epsilon

> > float f1 = -1.00000001f;[color=blue][color=green]
> > float f2 = -0.99999999f;
> > if (f1==-1 && f2==-1)
> > {
> > Console.WriteLi ne("hello");
> > }[/color]
>
> There's nothing funny about that - the nearest float to those literals
> is exactly -1. Have a look in the IL. That explains the Ceiling/Floor
> problems you were having too.[/color]


I'd have expected a simple truncation which would have returned true for
f1==-1 but false for f2==-1,
since -0.999999f would have been truncated to 0 which obviously wasn't the
case. And who guarantees that every platform will handle this the same way?
Even if the standard exactly dictates what should happen such edge cases are
imho always a good candidate for implementation errors.

I stick to my opinion - never trust a floating point value.

--
cody

[Freeware, Games and Humor]
www.deutronium.de.vu || www.deutronium.tk

[Jon Skeet [C# MVP]]

Re: Meaning of Double.Epsilon

cody <please_dont.sp am.deutronium@g mx.de> wrote:[color=blue][color=green][color=darkred]
> > > float f1 = -1.00000001f;
> > > float f2 = -0.99999999f;
> > > if (f1==-1 && f2==-1)
> > > {
> > > Console.WriteLi ne("hello");
> > > }[/color]
> >
> > There's nothing funny about that - the nearest float to those literals
> > is exactly -1. Have a look in the IL. That explains the Ceiling/Floor
> > problems you were having too.[/color]
>
> I'd have expected a simple truncation which would have returned true for
> f1==-1 but false for f2==-1,
> since -0.999999f would have been truncated to 0 which obviously wasn't the
> case.[/color]

No - both f1 and f2 are exactly -1.
[color=blue]
> And who guarantees that every platform will handle this the same way?[/color]

The C# specification states:

<quote>
The value of a real literal having type float or double is determined
by using the IEEE "round to nearest" mode.
</quote>

I have no reason to doubt that applying that to both -1.00000001 and
-0.99999999 would show that both should have a value of -1.
[color=blue]
> Even if the standard exactly dictates what should happen such edge
> cases are imho always a good candidate for implementation errors.
>
> I stick to my opinion - never trust a floating point value.[/color]

Well, only trust a floating point value if you know what's going on. My
point is that there was nothing "funny" going on in your code - it was
perfectly predictable.

--
Jon Skeet - <skeet@pobox.co m>

http://www.pobox.com/~skeet

If replying to the group, please do not mail me too