Alternative to Decimal type

[Frank Millman]

Hi all

I have a standard requirement for a 'decimal' type, to instantiate and
manipulate numeric data that is stored in a database. I came up with a
solution long before the introduction of the Decimal type, which has
been working well for me. I know the 'scale' (number of decimal
places) of the number in advance. When I read the number in from the
database I scale it up to an integer. When I write it back I scale it
down again. All arithmetic is done using integers, so I do not lose
accuracy.

There is one inconvenience with this approach. For example, if I have
a product quantity with a scale of 4, and a price with a scale of 2,
and I want to multiply them to get a value with a scale of 2, I have
to remember to scale the result down by 4. This is a minor chore, and
errors are quickly picked up by testing, but it does make the code a
bit messy, so it would be nice to find a solution.

I am now doing some refactoring, and decided to take a look at the
Decimal type. My initial impressions are that it is quite awkward to
use, that I do not need its advanced features, and that it does not
help solve the one problem I have mentioned above.

I therefore spent a bit of time experimenting with a Number type that
suits my particular requirements. I have come up with something that
seems to work, which I show below.

I have two questions.

1. Are there any obvious problems in what I have done?

2. Am I reinventing the wheel unnecessarily? i.e. can I do the
equivalent quite easily using the Decimal type?

--------------------
from __future__ import division

class Number(object):
def __init__(self,v alue,scale):
self.factor = 10.0**scale
if isinstance(valu e,Number):
value = value.value / value.factor
self.value = long(round(valu e * self.factor))
self.scale = scale

def __add__(self,ot her):
if isinstance(othe r,Number):
other = other.value / other.factor
return Number((self.va lue/self.factor)+ot her,self.scale)

def __sub__(self,ot her):
if isinstance(othe r,Number):
other = other.value / other.factor
return Number((self.va lue/self.factor)-other,self.scal e)

def __mul__(self,ot her):
if isinstance(othe r,Number):
other = other.value / other.factor
return Number((self.va lue/self.factor)*ot her,self.scale)

def __truediv__(sel f,other):
if isinstance(othe r,Number):
other = other.value / other.factor
return Number((self.va lue/self.factor)/other,self.scal e)

def __radd__(self,o ther):
return self.__add__(ot her)

def __rsub__(self,o ther):
return Number(other-(self.value/self.factor),se lf.scale)

def __rmul__(self,o ther):
return self.__mul__(ot her)

def __rtruediv__(se lf,other):
return Number(other/(self.value/self.factor),se lf.scale)

def __cmp__(self,ot her):
if isinstance(othe r,Number):
other = other.value / other.factor
this = self.value / self.factor
if this < other:
return -1
elif this other:
return 1
else:
return 0

def __str__(self):
s = str(self.value)
if s[0] == '-':
minus = '-'
s = s[1:].zfill(self.sca le+1)
else:
minus = ''
s = s.zfill(self.sc ale+1)
return '%s%s.%s' % (minus, s[:-self.scale], s[-self.scale:])
--------------------

Example usage -

>>>qty = Number(12.5,4)
>>>price = Number(123.45,2 )

>>>print price * qty

1543.13 [scale is taken from left-hand operand]

>>>print qty * price

1543.1250 [scale is taken from left-hand operand]

>>>print Number(qty * price,2)

1543.13 [scale is taken from Number instance]
--------------------

At this stage I have not built in any rounding options, but this can
be done later if I find that I need it.

Any comments will be welcome.

Thanks

Frank Millman

[Paul Hankin]

Re: Alternative to Decimal type

On Jun 9, 5:11 pm, Frank Millman <fr...@chagford .comwrote:

I have a standard requirement for a 'decimal' type, to instantiate and
manipulate numeric data that is stored in a database. I came up with a
solution long before the introduction of the Decimal type, which has
been working well for me. I know the 'scale' (number of decimal
places) of the number in advance. When I read the number in from the
database I scale it up to an integer. When I write it back I scale it
down again. All arithmetic is done using integers, so I do not lose
accuracy.
>
There is one inconvenience with this approach. For example, if I have
a product quantity with a scale of 4, and a price with a scale of 2,
and I want to multiply them to get a value with a scale of 2, I have
to remember to scale the result down by 4. This is a minor chore, and
errors are quickly picked up by testing, but it does make the code a
bit messy, so it would be nice to find a solution.
>
I am now doing some refactoring, and decided to take a look at the
Decimal type. My initial impressions are that it is quite awkward to
use, that I do not need its advanced features, and that it does not
help solve the one problem I have mentioned above.
>
I therefore spent a bit of time experimenting with a Number type that
suits my particular requirements. I have come up with something that
seems to work, which I show below.
>
I have two questions.
>
1. Are there any obvious problems in what I have done?
>
2. Am I reinventing the wheel unnecessarily? i.e. can I do the
equivalent quite easily using the Decimal type?

Hi Frank,
I don't know why you think Decimal is complicated: it has some
advanced features, but for what you seem to be doing it should be easy
to replace your 'Number' with it. In fact, it makes things simpler
since you don't have to worry about 'scale'.

Your examples convert easily:

from decimal import Decimal
qty = Decimal('12.5')
price = Decimal('123.45 ')

print price * qty
print qty * price
print (qty * price).quantize (Decimal('0.01' ))

--
Paul Hankin

[Frank Millman]

Re: Alternative to Decimal type

On Jun 9, 10:54 am, Paul Hankin <paul.han...@gm ail.comwrote:

>
Hi Frank,
I don't know why you think Decimal is complicated: it has some
advanced features, but for what you seem to be doing it should be easy
to replace your 'Number' with it. In fact, it makes things simpler
since you don't have to worry about 'scale'.
>
Your examples convert easily:
>
from decimal import Decimal
qty = Decimal('12.5')
price = Decimal('123.45 ')
>
print price * qty
print qty * price
print (qty * price).quantize (Decimal('0.01' ))
>

I thought I might be missing something obvious. This does indeed look
easy. Thanks, Paul

Frank

[Frank Millman]

Re: Alternative to Decimal type

On Jun 9, 4:06 pm, Frank Millman <fr...@chagford .comwrote:

>
Thanks for the reply, Mel. I don't quite understand what you mean.

As so often happens, after I sent my reply I re-read your post and I
think I understand what you are getting at.

One problem with my approach is that I am truncating the result down
to the desired scale factor every time I create a new instance. This
could result in a loss of precision if I chain a series of instances
together in a calculation. I think that what you are suggesting avoids
this problem.

I will read your message again carefully. I think it will lead to a
rethink of my approach.

Thanks again

Frank

P.S. Despite my earlier reply to Paul, I have not abandoned the idea
of using my Number class as opposed to the standard Decimal class.

I did a simple test of creating two instances and adding them
together, using both methods, and timing them. Decimal came out 6
times slower than Number.

Is that important? Don't know, but it might be.

[Diez B. Roggisch]

Re: Alternative to Decimal type

Frank Millman wrote:

On Jun 9, 4:06 pm, Frank Millman <fr...@chagford .comwrote:

>>
>Thanks for the reply, Mel. I don't quite understand what you mean.

>
As so often happens, after I sent my reply I re-read your post and I
think I understand what you are getting at.
>
One problem with my approach is that I am truncating the result down
to the desired scale factor every time I create a new instance. This
could result in a loss of precision if I chain a series of instances
together in a calculation. I think that what you are suggesting avoids
this problem.
>
I will read your message again carefully. I think it will lead to a
rethink of my approach.
>
Thanks again
>
Frank
>
P.S. Despite my earlier reply to Paul, I have not abandoned the idea
of using my Number class as opposed to the standard Decimal class.
>
I did a simple test of creating two instances and adding them
together, using both methods, and timing them. Decimal came out 6
times slower than Number.
>
Is that important? Don't know, but it might be.

It is because it uses arbitrary precision integer literals instead of ieee
floats. It pays this price so you get decimal rounding errors instead of
binary. Yet rounding errors you get...

If you are in money calculations with your Number-class - you certainly want
Decimal instead.

If all you want is auto-rounding... then you might not care.

Diez

[Frank Millman]

Re: Alternative to Decimal type

Thanks to all for the various replies. They have all helped me to
refine my ideas on the subject. These are my latest thoughts.

Firstly, the Decimal type exists, it clearly works well, it is written
by people much cleverer than me, so I would need a good reason not to
use it. Speed could be a good reason, provided I am sure that any
alternative is 100% accurate for my purposes.

My approach is based on expressing a decimal number as a combination
of an integer and a scale, where scale means the number of digits to
the right of the decimal point.

Therefore 0.04 is integer 4 with scale 2, 1.1 is integer 11 with scale
1, -123.456 is integer -123456 with scale 3. I am pretty sure that any
decimal number can be accurately represented in this form.

All arithmetic is carried out using integer arithmetic, so although
there may be rounding differences, there will not be the spurious
differences thrown up by trying to use floats for decimal arithmetic.

I use a class called Number, with two attributes - an integer and a
scale. My first attempt required these two to be provided every time
an instance was created. Then I realised that this would cause loss of
precision if I chain a series of instances together in a calculation.
The constructor can now accept any of the following forms -

1. A digit (either integer or float) and a scale. It uses the scale
factor to round up the digit to the appropriate integer.

2. Another Number instance. It takes the integer and scale from the
other instance.

3. An integer, with no scale. It uses the integer, and assume a scale
of zero.

4. A float in string format (e.g. '1.1') with no scale. It uses the
number of digits to the right as the scale, and scales the number up
to the appropriate integer.

For addition, subtraction, multiplication and division, the 'other'
number can be any of 2, 3, or 4 above. The result is a new Number
instance. The scale of the new instance is based on the following rule
-

For addition and subtraction, the new scale is the greater of the two
scales on the left and right hand sides.

For multiplication, the new scale is the sum of the two scales on the
left and right hand sides.

For division, I could not think of an appropriate rule, so I just hard-
coded a scale of 9. I am sure this will give sufficient precision for
any calculation I am likely to encounter.

My Number class is now a bit more complicated than before, so the
performance is not as great, but I am still getting a four-fold
improvement over the Decimal type, so I will continue running with my
version for now.

My main concern is that my approach may be naive, and that I will run
into situations that I have not catered for, resulting in errors. If
this is the case, I will drop this like a hot potato and stick to the
Decimal type. Can anyone point out any pitfalls I might be unaware of?

I will be happy to show the code for the new Number class if anyone is
interested.

Thanks

Frank

[Ethan Furman]

Re: Alternative to Decimal type

Frank Millman wrote:

Thanks to all for the various replies. They have all helped me to
refine my ideas on the subject. These are my latest thoughts.
>
Firstly, the Decimal type exists, it clearly works well, it is written
by people much cleverer than me, so I would need a good reason not to
use it. Speed could be a good reason, provided I am sure that any
alternative is 100% accurate for my purposes.

[snip]

For addition, subtraction, multiplication and division, the 'other'
number can be any of 2, 3, or 4 above. The result is a new Number
instance. The scale of the new instance is based on the following rule
>
For addition and subtraction . . .
For multiplication . . .
For division . . .

Out of curiosity, what is the purpose of these numbers? Do they
represent money, measurements, or something else? The reason I ask is
way back in physics class (or maybe chemistry... it was way back :) I
was introduced to the idea of significant digits -- that idea being that
a measured number is only accurate to a certain degree, and calculations
using that number therefore could not be more accurate. Sort of like a
built-in error range.

I'm thinking of developing the class in the direction of maintaining the
significant digits through calculations... mostly as I think it would be
fun, and it also seems like a good test case to get me in the habit of
unit testing. I'll call it something besides Number, though. :)

Is anybody aware of such a class already in existence?
--
Ethan

[Frank Millman]

Re: Alternative to Decimal type

On Jun 11, 4:39 pm, Ethan Furman <et...@stonelea f.uswrote:

Frank Millman wrote:

Thanks to all for the various replies. They have all helped me to
refine my ideas on the subject. These are my latest thoughts.

>

Out of curiosity, what is the purpose of these numbers?  Do they
represent money, measurements, or something else?

I am writing a business/accounting application. The numbers represent
mostly, but not exclusively, money.

Examples -

Multiply a selling price (scale 2) by a product quantity (scale 4) to
get an invoice value (scale 2), rounded half-up.

Multiply an invoice value (scale 2) by a tax rate (scale 2) to get a
tax value (scale 2), rounded down.

Divide a currency value (scale 2) by an exchange rate (scale 6) to get
a value in a different currency (scale 2).

Divide a product quantity (scale 4) by a pack size (scale 2) to get an
equivalent quantity in a different pack size.

In my experience, the most important thing is to be consistent when
rounding. If you leave the rounding until the presentation stage, you
can end up with an invoice value plus a tax amount differing from the
invoice total by +/- 0.01. Therefore I always round a result to the
required scale before 'freezing' it, whether in a database or just in
an object instance.

Frank

[Nick Craig-Wood]

Re: Alternative to Decimal type

Frank Millman <frank@chagford .comwrote:

Thanks to all for the various replies. They have all helped me to
refine my ideas on the subject. These are my latest thoughts.
>
Firstly, the Decimal type exists, it clearly works well, it is written
by people much cleverer than me, so I would need a good reason not to
use it. Speed could be a good reason, provided I am sure that any
alternative is 100% accurate for my purposes.
>
My approach is based on expressing a decimal number as a combination
of an integer and a scale, where scale means the number of digits to
the right of the decimal point.
>
Therefore 0.04 is integer 4 with scale 2, 1.1 is integer 11 with scale
1, -123.456 is integer -123456 with scale 3. I am pretty sure that any
decimal number can be accurately represented in this form.
>
All arithmetic is carried out using integer arithmetic, so although
there may be rounding differences, there will not be the spurious
differences thrown up by trying to use floats for decimal
arithmetic.

I used an identical scheme in a product some time ago (written in C
not python). It was useful because it modelled the problem domain
exactly.

You might want to investigate the rational class in gmpy which might
satisfy your requirement for exact arithmetic :-

>>import gmpy
>>gmpy.mpq(123, 1000)

mpq(123,1000)

>>a = gmpy.mpq(123,10 00)
>>b = gmpy.mpq(12,100 )
>>a+b

mpq(243,1000)

>>a*b

mpq(369,25000)

>>a/b

mpq(41,40)

>>>

It is also *very* fast being written in C.

--
Nick Craig-Wood <nick@craig-wood.com-- http://www.craig-wood.com/nick

[Terry Reedy]

Re: Alternative to Decimal type


"Frank Millman" <frank@chagford .comwrote in message
news:f7cc4f0f-3e37-43e7-927a-0ed14a063930@25 g2000hsx.google groups.com...
| Thanks to all for the various replies. They have all helped me to
| refine my ideas on the subject. These are my latest thoughts.
|
| Firstly, the Decimal type exists, it clearly works well, it is written
| by people much cleverer than me, so I would need a good reason not to
| use it. Speed could be a good reason, provided I am sure that any
| alternative is 100% accurate for my purposes.

The Decimal module is a Python (now C coded in 2.6/3.0, I believe)
implementation of a particular IBM-sponsored standard. The standard is
mostly good, but it is somewhat large with lots of options (such as
rounding modes) and a bit of garbage (the new 'logical' operations, for
instance) added by IBM for proprietary purposes. Fortunately, one can
ignore the latter once you recognize them for what they are.

As Nick indicated, it is not the first in its general category. And I
believe the Decimal implementation could have been done differently.

By writing you own class, you get just what you need with the behavior you
want. I think just as important is the understanding of many of the issues
involved, even if you eventually switch to something else. That is a
pretty good reason in itself.

tjr

[Aahz]

Re: Alternative to Decimal type

In article <f7cc4f0f-3e37-43e7-927a-0ed14a063930@25 g2000hsx.google groups.com>,
Frank Millman <frank@chagford .comwrote:

>
>My approach is based on expressing a decimal number as a combination
>of an integer and a scale, where scale means the number of digits to
>the right of the decimal point.

You should probably use one written by an expert:

http://www.pythoncraft.com/FixedPoint.py

--
Aahz (aahz@pythoncra ft.com) <* http://www.pythoncraft.com/

"as long as we like the same operating system, things are cool." --piranha

[Frank Millman]

Re: Alternative to Decimal type

On Jun 11, 11:48 am, Frank Millman <fr...@chagford .comwrote:

Thanks to all for the various replies. They have all helped me to
refine my ideas on the subject. These are my latest thoughts.
>

[snip]

>
My main concern is that my approach may be naive, and that I will run
into situations that I have not catered for, resulting in errors. If
this is the case, I will drop this like a hot potato and stick to the
Decimal type. Can anyone point out any pitfalls I might be unaware of?
>
I will be happy to show the code for the new Number class if anyone is
interested.
>

Thanks again for all the really useful replies.

I will have a look at gmpy, and I will study FixedPoint.py closely.

Frank

[Grant Edwards]

Re: Alternative to Decimal type

On 2008-06-12, Dennis Lee Bieber <wlfraed@ix.net com.comwrote:

If you were to follow the footsteps of COBOL, you'd be using
BCD internally, with a special code to represent the decimal
point (and maybe, to save space, the sign too)
>
Old mainframes had instructions to work with packed BCD as a
register format (the Xerox Sigma series used four 32-bit
registers to represent a 31 digit packed BCD number). Okay, I
think even the Pentium's still have a BCD operation set,

IIRC, most general purpose microprocessors I've used had
special instructions to make packed BCD operations easier, and
back in the day (when Pascal was much more popular than C)
compilers for microprocessors usually offered a BCD floating
point data type.

--
Grant Edwards grante Yow! The SAME WAVE keeps
at coming in and COLLAPSING
visi.com like a rayon MUU-MUU ...