insufficient incrementation

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Dormilich
    Recognized Expert Expert
    • Aug 2008
    • 8694

    insufficient incrementation

    Hi,

    I’ve got a mathematical script, which is currently running fine*, although it bothers me that JS does not always increment correctly. sometimes it happens that the incremented value is slightly bigger/smaller (around the 12th decimal place) than it should be (I’m usually incrementing by 0.1). because of that it affects the accuracy of the results.

    incrementation:
    - Fn.search() line 239 & 248
    value assignment:
    - Fn() line 83-85

    full code

    can anyone shed some light on this matter?

    * - even if you can’t read German, after some trials you should know what each button/field is for.
  • gits
    Recognized Expert Moderator Expert
    • May 2007
    • 5390

    #2
    that is due to javascript's number representation - taken from MDC here:

    Numbers in JavaScript are "double-precision 64-bit format IEEE 754 values", according to the spec. This has some interesting consequences. There's no such thing as an integer in JavaScript, so you have to be a little careful with your arithmetic if you're used to maths in C or Java. Watch out for stuff like:

    Code:
    0.1 + 0.2 = 0.30000000000000004
    so basicly there are some things to do when you do math with decimal values in JavaScript ... you could round the values or implement some function that fix that issue, in case you need to.

    kind regards

    Comment

    • Dormilich
      Recognized Expert Expert
      • Aug 2008
      • 8694

      #3
      besides rounding, do you have any idea how to do this?

      Comment

      • gits
        Recognized Expert Moderator Expert
        • May 2007
        • 5390

        #4
        not really ... i don't use JavaScript math with that precision ... mostly in business cases where simple rounding rules will do the job. but i guess there are some theoretical concepts that could handle it ... may be you could start further research from here?

        kind regards

        Comment

        • Dormilich
          Recognized Expert Expert
          • Aug 2008
          • 8694

          #5
          actually, I hardly need more than a 3 digits precision.* (I use it for solving dissociation equations when comparing experiment data with theoretical values)


          * - besides that, it’s a fun/self-training project

          [EDIT]
          do you see anything, where I should do improvements?

          Comment

          • gits
            Recognized Expert Moderator Expert
            • May 2007
            • 5390

            #6
            :) are you asking for a code review? ... besides of the mixing of different ways to build/declare objects/methods i don't see anything that i would change. personally i always prefer the 'classical' way to build JavaScript objects and would always use the prototype for their methods, but as i said ... that is my personal view ... i think that methods shouldn't alloc memory everytime an instance of the obj is created while member-variables that store 'plain-data' need to do so ...

            kind regards

            Comment

            • Dormilich
              Recognized Expert Expert
              • Aug 2008
              • 8694

              #7
              Originally posted by gits
              besides of the mixing of different ways to build/declare objects/methods i don't see anything that i would change.
              that’s already quite helpful.

              Comment

              Working...