PRINT 1/(1+26)=0?

Collapse
This topic is closed.
X
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • mpaine@htxml.com
    #1

    PRINT 1/(1+26)=0?

    Why does this T-SQL round incorrectly?

    PRINT 1/(1+26)

    when it should be 0.037037037?

    Thanks,
    Michael

    Tags: None
    • mpaine@htxml.com
      #2
      Re: PRINT 1/(1+26)=0?

      I should have been more specific:


      DECLARE @A int
      DECLARE @B int
      DECLARE @C float

      SET @A = 1
      SET @B = 26
      SET @C = @A / (@A + @B)
      PRINT @C



      mpa...@htxml.co m wrote:
      Why does this T-SQL round incorrectly?
      >
      PRINT 1/(1+26)
      >
      when it should be 0.037037037?
      >
      Thanks,
      Michael

        Comment

        • mpaine@htxml.com
          #3
          Re: PRINT 1/(1+26)=0?

          Ok, I just figured it out:



          DECLARE @A int
          DECLARE @B int
          DECLARE @C float

          SET @A = 1
          SET @B = 26
          SET @C = CAST(@A AS float) / (CAST(@A AS float) + CAST(@B AS float))
          PRINT @C

          This is unlike any other programming language that I have used before
          -- so strange.

          mpaine@htxml.co m wrote:
          Why does this T-SQL round incorrectly?
          >
          PRINT 1/(1+26)
          >
          when it should be 0.037037037?
          >
          Thanks,
          Michael

            Comment

            • Roy Harvey
              #4
              Re: PRINT 1/(1+26)=0?

              If any one of the operators had a decimal point then the results would
              have had a decimal point. Performaing pure integer arithmatic, as in
              the original example, returns an integer result.

              Roy Harvey
              Beacon Falls, CT

              On 20 Nov 2006 12:57:04 -0800, mpaine@htxml.co m wrote:
              >Ok, I just figured it out:
              >
              >
              >
              >DECLARE @A int
              >DECLARE @B int
              >DECLARE @C float
              >
              >SET @A = 1
              >SET @B = 26
              >SET @C = CAST(@A AS float) / (CAST(@A AS float) + CAST(@B AS float))
              >PRINT @C
              >
              >This is unlike any other programming language that I have used before
              >-- so strange.
              >
              >mpaine@htxml.c om wrote:
              >Why does this T-SQL round incorrectly?
              >>
              >PRINT 1/(1+26)
              >>
              >when it should be 0.037037037?
              >>
              >Thanks,
              >Michael

                Comment

                • mpaine@htxml.com
                  #5
                  Re: PRINT 1/(1+26)=0?

                  So is PRINT CAST(1 AS float)/(CAST(1 AS float)+CAST(26 AS float)) the
                  best way to get the correct answer for PRINT 1/(1+26)?

                  - Michael

                  Roy Harvey wrote:
                  If any one of the operators had a decimal point then the results would
                  have had a decimal point. Performaing pure integer arithmatic, as in
                  the original example, returns an integer result.
                  >
                  Roy Harvey
                  Beacon Falls, CT
                  >
                  On 20 Nov 2006 12:57:04 -0800, mpaine@htxml.co m wrote:
                  >
                  Ok, I just figured it out:



                  DECLARE @A int
                  DECLARE @B int
                  DECLARE @C float

                  SET @A = 1
                  SET @B = 26
                  SET @C = CAST(@A AS float) / (CAST(@A AS float) + CAST(@B AS float))
                  PRINT @C

                  This is unlike any other programming language that I have used before
                  -- so strange.

                  mpaine@htxml.co m wrote:
                  Why does this T-SQL round incorrectly?
                  >
                  PRINT 1/(1+26)
                  >
                  when it should be 0.037037037?
                  >
                  Thanks,
                  Michael

                    Comment

                    • Roy Harvey
                      #6
                      Re: PRINT 1/(1+26)=0?

                      On 20 Nov 2006 14:53:34 -0800, mpaine@htxml.co m wrote:
                      >So is PRINT CAST(1 AS float)/(CAST(1 AS float)+CAST(26 AS float)) the
                      >best way to get the correct answer for PRINT 1/(1+26)?
                      >
                      >- Michael
                      If a value is a constant, simply adding a decimal point to it is
                      sufficient, and only one of the operators needs the decimal point:

                      PRINT 1.0/(1+26)

                      Alternately, if none of the values are constants, converting any one
                      of them is sufficient. However, I would not choose to convert to
                      FLOAT, but to an appropriately sized DECIMAL.

                      PRINT CONVERT(DECIMAL (15,6),1)/(1+26)

                      Roy Harvey
                      Beacon Falls, CT

                        Comment

                        Previous template Next
                        Working...