problem with a calculation

[Michael G]

double A = floor((2447297. 0 - 122.1)/365.25);
i get differing results. It varies between 6699 and 6700. The correct
answer is 6699.
The 6700 result really throws the entire group of calculations off.

Any suggestions welcome.
Thanks Mike




-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

[Niels Dybdahl]

Re: problem with a calculation

> double A = floor((2447297. 0 - 122.1)/365.25);[color=blue]
> i get differing results. It varies between 6699 and 6700. The correct
> answer is 6699.
> The 6700 result really throws the entire group of calculations off.[/color]

Do you have details on the system that returns 6700 ? (Hardware, OS,
Compiler).

Doing the calculation in 32 bit floats would probably return 6700, while
using 64 bit or 80 bit would return 6699.

Niels Dybdahl

[Jacek Dziedzic]

Re: [OT] problem with a calculation

Michael G wrote:
[color=blue]
> double A = floor((2447297. 0 - 122.1)/365.25);
> i get differing results. It varies between 6699 and 6700. The correct
> answer is 6699.[/color]

Is it? I'd say the 'correct' answer would depend on the type
of the argument of this floor() function you're using.

As (2447297.0-122.1)/365.25 is 6699.999726... then *if* the
argument of floor() can't hold it with enough precision the
result will be 6700.0

HTH,
- J.

PS. It's not really a proper group to ask this.

[Michael G]

Re: problem with a calculation


"Niels Dybdahl" <ndy@fjern.dett eesko-graphics.com> wrote in message
news:408e0c31$0 $159$edfadb0f@d text02.news.tel e.dk...[color=blue][color=green]
> > double A = floor((2447297. 0 - 122.1)/365.25);
> > i get differing results. It varies between 6699 and 6700. The correct
> > answer is 6699.
> > The 6700 result really throws the entire group of calculations off.[/color]
>
> Do you have details on the system that returns 6700 ? (Hardware, OS,
> Compiler).[/color]


I did some more checking and the problem is in the substraction.

The same system returns varying results in different contexts.
The OS is XP Home Edition. The hardware is P4 in a Toshiba laptop. The
compiler is VC++ 6.0.
We use julian day numbers for calculating dates within a game. If I unit
test the class the floor function returns 6699. But when the class is in
game, the flooring function returns 6700.

Mike




-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

[Peter van Merkerk]

Re: problem with a calculation

"Michael G" <mike-g@montana.com> wrote in message
news:408e6d5a_5 @corp.newsgroup s.com...[color=blue]
>
> "Niels Dybdahl" <ndy@fjern.dett eesko-graphics.com> wrote in message
> news:408e0c31$0 $159$edfadb0f@d text02.news.tel e.dk...[color=green][color=darkred]
> > > double A = floor((2447297. 0 - 122.1)/365.25);
> > > i get differing results. It varies between 6699 and 6700. The[/color][/color][/color]
correct[color=blue][color=green][color=darkred]
> > > answer is 6699.
> > > The 6700 result really throws the entire group of calculations off.[/color]
> >
> > Do you have details on the system that returns 6700 ? (Hardware, OS,
> > Compiler).[/color]
>
>
> I did some more checking and the problem is in the substraction.
>
> The same system returns varying results in different contexts.
> The OS is XP Home Edition. The hardware is P4 in a Toshiba laptop. The
> compiler is VC++ 6.0.
> We use julian day numbers for calculating dates within a game. If I unit
> test the class the floor function returns 6699. But when the class is in
> game, the flooring function returns 6700.[/color]

Are you sure you are using the same compiler setting for the unit test and
in game? If you make a fully optimized build rounding of floating point
number may be different than in debug builds. In release builds the
compiler will try to keep the intermediate results in the 80-bit floating
point registers as long as possible. This means there is no intermediate
rounding to 64-bit or 32-bit precision. With debug builds the intermediate
results are always written to and read from memory, and at that point the
intermediate result will be rounded to 64-bit or 32-bit precision. You can
fix this in release builds at the expense of performance by enabling the
"Improve Float Consistency" (/Op) option. See also: http://tinyurl.com/9an0

--
Peter van Merkerk
peter.van.merke rk(at)dse.nl

[tom_usenet]

Re: problem with a calculation

On Tue, 27 Apr 2004 08:24:38 -0600, "Michael G" <mike-g@montana.com>
wrote:
[color=blue]
>
>"Niels Dybdahl" <ndy@fjern.dett eesko-graphics.com> wrote in message
>news:408e0c31$ 0$159$edfadb0f@ dtext02.news.te le.dk...[color=green][color=darkred]
>> > double A = floor((2447297. 0 - 122.1)/365.25);
>> > i get differing results. It varies between 6699 and 6700. The correct
>> > answer is 6699.
>> > The 6700 result really throws the entire group of calculations off.[/color]
>>
>> Do you have details on the system that returns 6700 ? (Hardware, OS,
>> Compiler).[/color]
>
>
>I did some more checking and the problem is in the substraction.
>
>The same system returns varying results in different contexts.
>The OS is XP Home Edition. The hardware is P4 in a Toshiba laptop. The
>compiler is VC++ 6.0.
>We use julian day numbers for calculating dates within a game. If I unit
>test the class the floor function returns 6699. But when the class is in
>game, the flooring function returns 6700.[/color]

Why are you using doubles for date calculations? They are inaccurate
by nature (although not as inaccurate usually as you seem to be
experiencing), and integer maths will surely do the job. If inaccurate
dates are ok, I don't see the problem.

Note that (2447297.0 - 122.1)/365.25 will likely produce a different
result than if the same calculation if performed on variables, but the
difference will be small, and you should code to expect small
inaccuracies in any doubles (use <limits> to get more quantitive
information about the inaccuracies).

Tom
--
C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Dave Moore]

Re: problem with a calculation

"Niels Dybdahl" <ndy@fjern.dett eesko-graphics.com> wrote in message news:<408e0c31$ 0$159$edfadb0f@ dtext02.news.te le.dk>...[color=blue][color=green]
> > double A = floor((2447297. 0 - 122.1)/365.25);
> > i get differing results. It varies between 6699 and 6700. The correct
> > answer is 6699.
> > The 6700 result really throws the entire group of calculations off.[/color]
>
> Do you have details on the system that returns 6700 ? (Hardware, OS,
> Compiler).
>
> Doing the calculation in 32 bit floats would probably return 6700, while
> using 64 bit or 80 bit would return 6699.[/color]

Yes, you can typically only be sure of 7-digits of precision with
32-bit floating point values.

In general, addition and subtraction of numbers that differ by several
orders of magnitude should be a flag to watch out for precision errors
like the one the OP is having. It is usually possible to formulate
the original expression in a way that makes the precision problem more
tractable .. for example, if the numbers in the expression above are
really literals, the OP could remove the trivial integer division from
the problem to give:

double A = 6700.0 - ceil(0.1/365.25);

which will yield 6699 on a wider range of architectures.

Of course the problem is not likely to be so simple, but general
techniques can often be devised if you can say some things about the
arguments in advance. For example,

double func(double arg1, double arg2, double divisor) {
// assume arg1, arg2 non-negative and divisor > 1
double tmp1=floor(arg1/divisor);
double tmp2=floor(arg2/divisor);
double residue=(arg1-tmp1*divisor) - (arg2-tmp2*divisor);
return tmp1 - tmp2 - (residue<0) ? 1 : 0;
}

will carry out the schematic calculation from the OP's example,
yielding results that are somewhat less prone to precision error. The
key point is that the values that are subtracted are guaranteed to be
closer in magnitude than the original arguments. I expect there is a
more efficient way to write it, but it was just a quick example ...
also, sometimes efficiency must ultimately be sacrificed for the sake
of precision.

Check out a book like "The Art of Scientific Computing" for more info.

HTH, Dave Moore

[Michael G]

Re: [OT] problem with a calculation


"Jacek Dziedzic" <jacek__NOSPAM_ _@janowo.net> wrote in message
news:c6lkca$laa $1@korweta.task .gda.pl...
[color=blue]
> PS. It's not really a proper group to ask this.[/color]

My apologies.

Mike




-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

[Michael G]

Re: problem with a calculation


"Peter van Merkerk" <merkerk@deadsp am.com> wrote in message
news:c6lser$d9m l9$1@ID-133164.news.uni-berlin.de...[color=blue]
> "Michael G" <mike-g@montana.com> wrote in message
> news:408e6d5a_5 @corp.newsgroup s.com...[color=green]
> >
> > "Niels Dybdahl" <ndy@fjern.dett eesko-graphics.com> wrote in message
> > news:408e0c31$0 $159$edfadb0f@d text02.news.tel e.dk...[/color][/color]
[color=blue]
> fix this in release builds at the expense of performance by enabling the
> "Improve Float Consistency" (/Op) option. See also:[/color]
http://tinyurl.com/9an0[color=blue]
>[/color]

Didn't help.
Any other suggestions?

Thanks, Mike




-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----

[Peter van Merkerk]

Re: problem with a calculation

"Michael G" <mike-g@montana.com> wrote in message
news:408eda97_5 @corp.newsgroup s.com...[color=blue][color=green]
> > fix this in release builds at the expense of performance by enabling[/color][/color]
the[color=blue][color=green]
> > "Improve Float Consistency" (/Op) option. See also:[/color]
> http://tinyurl.com/9an0
>
> Didn't help.
> Any other suggestions?[/color]

If building with exactly the same compiler settings still produces
different results, I don't know. Unless there is something in the game code
that changes the rounding mode of the FPU (for example with the
non-standard functions _controlfp() or _control87()), I have no idea what
else can cause this. In the past I written very floating point intensive
programs (which were far more complex than your example), and by enabling
the "Improve Float Consistency" option I could eliminate the differences
between debug and release builds, and even (to my surprise) with the MatLab
version of the programs.

Because with the VC 6.0 compiler computes the number passed to the floor()
function during compile time in both debug and release builds, the only
thing that appears to behave differently is the floor() function.

--
Peter van Merkerk
peter.van.merke rk(at)dse.nl

[Michael G]

Re: problem with a calculation


"Peter van Merkerk" <merkerk@deadsp am.com> wrote in message
news:c6nqdc$de0 45$1@ID-133164.news.uni-berlin.de...[color=blue]
> "Michael G" <mike-g@montana.com> wrote in message
> news:408eda97_5 @corp.newsgroup s.com...[color=green][color=darkred]
> > > fix this in release builds at the expense of performance by enabling[/color][/color]
> the[color=green][color=darkred]
> > > "Improve Float Consistency" (/Op) option. See also:[/color]
> > http://tinyurl.com/9an0
> >
> > Didn't help.
> > Any other suggestions?[/color]
>
> If building with exactly the same compiler settings still produces
> different results, I don't know. Unless there is something in the game[/color]
code[color=blue]
> that changes the rounding mode of the FPU (for example with the
> non-standard functions _controlfp() or _control87()), I have no idea what
> else can cause this. In the past I written very floating point intensive
> programs (which were far more complex than your example), and by enabling
> the "Improve Float Consistency" option I could eliminate the differences
> between debug and release builds, and even (to my surprise) with the[/color]
MatLab[color=blue]
> version of the programs.
>
> Because with the VC 6.0 compiler computes the number passed to the floor()
> function during compile time in both debug and release builds, the only
> thing that appears to behave differently is the floor() function.[/color]

I know this is OT but for the benefit of those who have posted. My
apologies to the rest.

I found the solution to the floating point problem. One thing I did not
mention in my previous posts was that we are using DirectX. I would have
never imagined that that would be an issue but DirectX modifies how the
Floating Point Unit (FPU) handles floating point. If accuracy is desired the
flag D3DCREATE_FPU_P RESERVE must be used whenever CreateDevice is called.

Mike




-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----== Over 100,000 Newsgroups - 19 Different Servers! =-----