Pricision of float

[Benny]

Hi,

Just wonder why

PRINT CAST(0.05735425 67654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

thanks






*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!

[oj]

Re: Pricision of float

Float is never exact. You will need decimal or numeric instead.

--
-oj

http://www.rac4sql.net

"Benny" <anonymous@devd ex.com> wrote in message
news:3ff9ec52$0 $202$75868355@n ews.frii.net...[color=blue]
> Hi,
>
> Just wonder why
>
> PRINT CAST(0.05735425 67654 AS float)
>
> will give the rounded reult
>
> 0.0573543
>
> rather than the original number?
>
> "float" should be 'big' enough to hold numbers that have even more
> decimal places. How come it round up at the 7 decimal place?
>
> since I need to do some calculations with accumulated values. The
> rounded figure will cause significant error after a number of opertions.
>
> Are there any way to work around it?
>
> thanks
>
>
>
>
>
>
> *** Sent via Developersdex http://www.developersdex.com ***
> Don't just participate in USENET...get rewarded for it![/color]

[Erland Sommarskog]

Re: Pricision of float

Benny (anonymous@devd ex.com) writes:[color=blue]
> Just wonder why
>
> PRINT CAST(0.05735425 67654 AS float)
>
> will give the rounded reult
>
> 0.0573543
>
> rather than the original number?
>
> "float" should be 'big' enough to hold numbers that have even more
> decimal places. How come it round up at the 7 decimal place?
>
> since I need to do some calculations with accumulated values. The
> rounded figure will cause significant error after a number of opertions.
>
> Are there any way to work around it?[/color]

PRINT does not work on floats, it works with string data, so there is
an implicit conversion to varchar, and that conversion gives you
seven decimals by default.

Here is one option:

PRINT convert(varchar (30), CAST(0.05735425 67654 AS float), 2)

You can also use the str() function to format floats as string. Look
up this function in Books Online.


--
Erland Sommarskog, SQL Server MVP, sommar@algonet. se

Books Online for SQL Server SP3 at

http://www.microsoft.com/sql/techinfo/productdoc/2000/books.asp