Weird round off error in C program

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • amishera
    New Member
    • Jan 2009
    • 1

    Weird round off error in C program

    Hi,

    I have this piece of code in a C program where I trying to convert a double to a long long:

    Code:
    double x =  0.507462687;
    
    double a = pow(10,k+j);
    double b = pow(10,k);
    
    double c = a*x;
    double d = (long long) (b*x);
    long long num,denum;
    
    num = (long long) (c-d); // the value of num becomes = ........418 instead of ....419 ie the last digit 9 becomes 8.
    I am not sure what I should do to make the thing working ie avoid this peculiar round off error. Please suggest something.

    Thanks.
  • JosAH
    Recognized Expert MVP
    • Mar 2007
    • 11453

    #2
    Read this: Floating Point Arithmetic.

    kind regards,

    Jos

    Comment

    Working...