Decimal issues

[Mazdak]

Hey guys!

I recently bumped into a problem when I was calculating with simple additions. I had declared a variable eariler that we'll name "range" here.
In a loop range was added "+1". If a certain value was reached ( another variable ) then range was added with "+0.1" instead. Now this is where the issue occured.
The problem was identified as a "floating point issue" through several hours of google'ing, in other words the value I wanted wasn't for example -45.1 but something close to -45.1 like
-45.1000000004.

So I tried different solutions like type-casting with string. But then the instrument I was communicating with had problems (the range variable was used for input to a instrument). I really needed the double/float variable to be with one decimal (e.g 45.1) and not as a string or a float with bit issues.
So I went after the Decimal class, but I kept getting annoyed by the fact that you couldn't get the value only but you had to receive the whole class signature ( for example Decimal("-45.1" ). Now that is a really bad input for my instrument.
So I was wondering if you guys know a solution?, is it possible to recieve the value only, after calculations with the Decimal module. Or do I have to pull some dirty tricks on this one? :-)

[jlm699]

I think this may take dirty tricks... Python isn't the most friendly floating point language and Decimal is nice as long as you're keeping it "in the family", as in not converting back to any other types (except string). So if there is a way for your instrument to accept a string representation of a value you could simply wrap str() around your result... other wise you'll have to do some dirty work.

You could possibly overload the __repr__ function of the Decimal class so that instead of Decimal("-45.1") you only get "-45.1", but that's still not exactly what you're looking for anyway, so just ignore that

Perhaps you could try converting the result back to float() and then using round() with a precision of 1 ?

[Mazdak]

Oh yeah, that dirty trick really sounds good. The problem is that Python really doesnt like rounding when you have the floating point issue:

from decimal import *

>>> range = Decimal("-65.0")
>>> range = range + 0.1
>>> range
-64.90000000000006
>>>str(range)
'-64.9'
>>> float(ras)
-64.900000000000006
>>> round(ras,1)
-64.90000000000006

This is a terribly annoying thing that really destroys the market for Python. People who use instuments that depend on specific inputs (in my case, a float/double/int with a resolution of 0.1) must find another programming language that don't have this error. You don't have anymore tricks do you? :-)

[jlm699]

You don't have anymore tricks do you? :-)

Maybe wait for a new version of Python?? :(
This is for sure frustrating for many, and I have a feeling that the guys working on each new version of Python keep this in mind. What version of Python do you use, for curiosity's sake?

[Mazdak]

What version of Python do you use, for curiosity's sake?

Currently using Python 2.5 or 2.5.1, don't know which one =/. Do you know perhaps if there's a new version releasing anytime soon?

:: At the code block above I suddenly started to use a variable named 'ras', thats a minor error and should be 'range' instead ::

[Mazdak]

A quicky note for those that totally lost faith in the float datatype: Python is still able to represent decimals in the base of 2, for example 51.5.. 51,25 and 51.125.

Hopefully we'll see a future Python with the same easy goin' way and no float issues. Huzzah for that! =D

[jlm699]

A quicky note for those that totally lost faith in the float datatype: Python is still able to represent decimals in the base of 2, for example 51.5.. 51,25 and 51.125.

Hopefully we'll see a future Python with the same easy goin' way and no float issues. Huzzah for that! =D

FYI, Base 2 = Binary. Base 10 = Decimal. What are you trying to say in that first line there? Are you trying to say that Python handles two-digit precision floating-point numbers? And if so, how?

[kudos]

I must admit that adding a decimal with a float should probably give errors in my version of python. But what happens if you do this instead:

Decimal("-65.0") + Decimal("0.1")

-kudos

Oh yeah, that dirty trick really sounds good. The problem is that Python really doesnt like rounding when you have the floating point issue:

from decimal import *

>>> range = Decimal("-65.0")
>>> range = range + 0.1
>>> range
-64.90000000000006
>>>str(range)
'-64.9'
>>> float(ras)
-64.900000000000006
>>> round(ras,1)
-64.90000000000006

This is a terribly annoying thing that really destroys the market for Python. People who use instuments that depend on specific inputs (in my case, a float/double/int with a resolution of 0.1) must find another programming language that don't have this error. You don't have anymore tricks do you? :-)

[Mazdak]

I must admit that adding a decimal with a float should probably give errors in my version of python. But what happens if you do this instead:

Decimal("-65.0") + Decimal("0.1")

-kudos

The Decimal module returns the whole instance instead of extracting the value I want. I spoke with some proffessionals and they gave a bit of a lecture. All programming languages have this float-issue, it's just that they present it better than Python. Now if you type:

>>> a = 51.1 + 2.1
>>> print a
53.2

Dunno if this is correct but Python "hides" away the decimal values. Printing works! The real value however is alot longer than that. Now many of you are kinda "Hey, I knew that already and oh my gosh I'm gonna show this guy for not listening". But hey, I was so convinced that it was Python making errors and not the instrument, sorry! It is probably a bug in the instrument, thats what I've concluded.

The instruments documentation describes that the "error" variable I use as input through a method, allows decimal values aswell. That's the only thing I'm changing between my loops and as soon as I add decimal values to the variable it throws an exception.
FYI I'm kinda angry at the instrument right now and I'm trying to contact their programmers.

Thanks for the help guys and info about Python

[cybervegan]

These types of issues are trials and tribulations of the IEEE floating point numbers standard.

When precision matters, one solution is to use integer maths, after multiplying by a suitably large number (e.g. 1000000) and then, right at the end, converting back to floating point or decimal, after dividing back down. Python uses infinite precision integers, so you can multiply by almost as large a number as you like (but at an obvious performance/storage hit). If you still get rounding errors, you can use string operations to extract the exact representation you want, and stuff *that* into a decimal.

By the way, best to avoid using a variable called "range" as this conflicts with the python looping builtin function called the same:

for n in range(100): print n

This could cause some very strange and difficult to trace problems later on.

Hope that helps,
-cybervegan