Integer Division Broken?

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Ben
    #1

    Integer Division Broken?

    Hi, I have an interesting example from my debugger. I have 2 variables:
    sourcewidthnet and targetwidthnet. Notice the results in the debugger.
    I'm going to be forced to use the int function of the decimal.toint32
    since the \ operator doesn't appear to work, or I greatly misunderstand
    the documentation on the \ operator.

    decimal.Remaind er(sourcewidthn et, targetwidthnet) 0.375D
    sourcewidthnet 96D
    targetwidthnet 2.125D
    SourceWidthNet / TargetWidthNet
    45.176470588235 294117647058824 D(SourceWidthNe t \ TargetWidthNet)
    48 Long
    decimal.ToInt32 ((SourceWidthNe t / TargetWidthNet) ) 45 Integer

    I can't justify the: 96 \ 2.125 = 48 answer at all!

    Thanks for your input.

    Tags: None
    • Robin Tucker
      #2
      Re: Integer Division Broken?

      It is correct. 96 \ 2 = 48. When you do an integer division, the divisor
      is the integer part, which in this case is 2.

      "Ben" <benhunsberger@ bankscorporatio n.com> wrote in message
      news:1117631057 .785829.306980@ z14g2000cwz.goo glegroups.com.. .[color=blue]
      > Hi, I have an interesting example from my debugger. I have 2 variables:
      > sourcewidthnet and targetwidthnet. Notice the results in the debugger.
      > I'm going to be forced to use the int function of the decimal.toint32
      > since the \ operator doesn't appear to work, or I greatly misunderstand
      > the documentation on the \ operator.
      >
      > decimal.Remaind er(sourcewidthn et, targetwidthnet) 0.375D
      > sourcewidthnet 96D
      > targetwidthnet 2.125D
      > SourceWidthNet / TargetWidthNet
      > 45.176470588235 294117647058824 D(SourceWidthNe t \ TargetWidthNet)
      > 48 Long
      > decimal.ToInt32 ((SourceWidthNe t / TargetWidthNet) ) 45 Integer
      >
      > I can't justify the: 96 \ 2.125 = 48 answer at all!
      >
      > Thanks for your input.
      >[/color]


        Comment

        • Ben
          #3
          Re: Integer Division Broken?

          Ok, I finally see the documentation in the remarks section:

          Remarks
          Before division is performed, any floating-point numeric expressions
          are coerced to Long if Option Strict is Off. If Option Strict is On, a
          compiler error results

          Silly me, thanks.

          Still to get what I'm looking for the int() and decimal.tointxx
          routines will work, thanks again.

            Comment

            Previous template Next
            Working...