floating point over-/under-flow

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • Mark L Pappin

    floating point over-/under-flow

    <puts on Compiler Vendor hat>

    I've recently discovered that [at least two of] our compilers don't
    make any attempt to handle floating point overflow in add/subtract/
    multiply/divide, with the result that programmers who don't range-
    check their data can e.g. multiply two very tiny values and end up
    with a very large one. This is (AFAICT) quite fine according to the
    Standard but is certainly a QoI issue and I'd like to raise ours.

    I believe that it's the programmer's job to know what possible ranges
    his values could take, and to check they're sane before performing
    operations upon them. If this is done, then overflow and underflow
    can never happen. Rarely is this done, of course, so the compromise
    suggested by TPTB is to offer a safety net in the form of adding
    inexpensive checks to the generated code which can propagate status
    back to the user.

    The approach I'm taking is to detect overflow or underflow and set a
    flag (in the implementation namespace) as appropriate, but leave the
    [invalid] value in the result. This way, if it really is important to
    the user to eke that last bit (pun not intended) of information out of
    the operation then the way is clear for them to do so. For example, a
    multiplication which overflows will end up with a value pow(2,256)
    smaller than the correct [unrepresentable] value, thanks to wraparound
    of our 8-bit signed binary exponent, but the mantissa will still have
    full precision - some recovery may be possible.

    An alternative approach (since we use an IEEE-754-compatible
    representation already) is to go the whole hog and implement
    Infinities, NaNs, and Denormals. At this time we don't want to do this
    because it appears to be a significant cost (see below) for not much
    return. In any case, how to deal with those types of value in C is
    still not defined, so any solution of this type would still be Not C.


    I'm interested to hear the c.l.c set of viewpoints on silent vs. noisy
    underflow and overflow - in this case, coercing values to 0 or Inf,
    vs. leaving them as is and setting a user-checkable flag.
    ('Very-noisy' would be throwing an exception of some kind, also
    undefined by the Standards.) What practices to you use to avoid
    hitting overflow and underflow? Is our plan to let the dodgy value
    continue to exist of any conceivable value?


    For those who've read this far: we make cross-compilers for [often
    tiny; <256 bytes of RAM is common] CPUs commonly used in embedded
    systems, and we do our damnedest to make them meet C90 and are inching
    toward C99 in places. Floating point has become popular, some
    deficiencies have been found, and Muggins put his hand up to fix them.

    mlp
  • Tim Prince

    #2
    Re: floating point over-/under-flow


    "Mark L Pappin" <mlp@acm.org> wrote in message
    news:m3pt1sucoy .fsf@Claudio.Me ssina...[color=blue]
    > <puts on Compiler Vendor hat>
    >
    > I've recently discovered that [at least two of] our compilers don't
    > make any attempt to handle floating point overflow in add/subtract/
    > multiply/divide, with the result that programmers who don't range-
    > check their data can e.g. multiply two very tiny values and end up
    > with a very large one.[/color]
    .....[color=blue]
    >
    > The approach I'm taking is to detect overflow or underflow and set a
    > flag (in the implementation namespace) as appropriate, but leave the
    > [invalid] value in the result. This way, if it really is important to
    > the user to eke that last bit (pun not intended) of information out of
    > the operation then the way is clear for them to do so. For example, a
    > multiplication which overflows will end up with a value pow(2,256)
    > smaller than the correct [unrepresentable] value, thanks to wraparound
    > of our 8-bit signed binary exponent, but the mantissa will still have
    > full precision - some recovery may be possible.
    >[/color]
    ....[color=blue]
    > I'm interested to hear the c.l.c set of viewpoints on silent vs. noisy
    > underflow and overflow - in this case, coercing values to 0 or Inf,
    > vs. leaving them as is and setting a user-checkable flag.
    > ('Very-noisy' would be throwing an exception of some kind, also
    > undefined by the Standards.) What practices to you use to avoid
    > hitting overflow and underflow? Is our plan to let the dodgy value
    > continue to exist of any conceivable value?[/color]
    Long, long ago, before C, the usual practice for such hardware was to throw
    an exception. If the application wished to continue, it had to provide an
    exception handler. Thus, the application would make the choice whether to
    set 0 or Inf and continue silently, or issue a diagnostic, or take a branch
    to process the out of range. Continuing without fixing it up seems unlikely
    to be useful. When IEEE-754 came in, the argument was this exception
    handling is an unnecessary burden.


    Comment

    • Kevin Bracey

      #3
      Re: floating point over-/under-flow

      In message <m3pt1sucoy.fsf @Claudio.Messin a>
      Mark L Pappin <mlp@acm.org> wrote:
      [color=blue]
      > An alternative approach (since we use an IEEE-754-compatible
      > representation already) is to go the whole hog and implement
      > Infinities, NaNs, and Denormals. At this time we don't want to do this
      > because it appears to be a significant cost (see below) for not much
      > return. In any case, how to deal with those types of value in C is
      > still not defined, so any solution of this type would still be Not C.[/color]

      Actually, C99 does fully define how to deal with all those values for
      IEEE754, in Annex F. Only "signalling NaNs" aren't covered.

      The main problem with implementation of NaNs etc is the extra cost
      of handling them as INPUTs to operations. All your operations will need
      to be able to detect them and propagate them accordingly, otherwise there's
      no point generating them in the first place. This can be expensive.

      I'd be tempted to say that the basic operations should signal SIGFPE in the
      event of an error (ie enable Invalid Operation, Divide By Zero and Overflow
      traps, in IEEE754 terminology), and kill the program. This means NaNs and
      Infinities can't get into the system. I've used such systems quite
      extensively.

      If you intend to just set a flag and continue execution, then I feel it's
      important that some sort of indicator value be left in the result, rather
      than just leaving it wrapped. If you can't manage "NaN" or "Inf", then at the
      very least it should be "HUGE_VAL", as per the <math.h> functions.

      As for denormals, flushing all tiny results to zero is a reasonable thing to
      do if you can't implement denormals properly. Kahan wouldn't approve, as it
      leaves a massive hole in the number line, but it's far better than your
      current "tiny * tiny -> huge".

      By the way, the overflow flags you're describing are defined in <fenv.h> in
      C99 - there's no need for you to invent your own, I believe.

      You may want to consider the FENV #pragmas. I'm not sure that FENV_ACCESS
      will give you any leeway on shrinking your code size, but you should have a
      look at that area. Otherwise some sort of compiler option or private #pragma
      to suppress the extra code bloat of proper range checking will probably be in
      order in your situation.
      [color=blue]
      > Floating point has become popular, some deficiencies have been found, and
      > Muggins put his hand up to fix them.[/color]

      Well, this will be an important life lesson for you.

      --
      Kevin Bracey, Principal Software Engineer
      Tematic Ltd Tel: +44 (0) 1223 503464
      182-190 Newmarket Road Fax: +44 (0) 1728 727430
      Cambridge, CB5 8HE, United Kingdom WWW: http://www.tematic.com/

      Comment

      • Gordon Burditt

        #4
        Re: floating point over-/under-flow

        >I've recently discovered that [at least two of] our compilers don't[color=blue]
        >make any attempt to handle floating point overflow in add/subtract/
        >multiply/divide, with the result that programmers who don't range-
        >check their data can e.g. multiply two very tiny values and end up
        >with a very large one. This is (AFAICT) quite fine according to the
        >Standard but is certainly a QoI issue and I'd like to raise ours.[/color]

        What floating point hardware / software allows you to multiply two
        small floating-point numbers and end up with a huge one? The
        implementations I know of either end up with zero or a small number.
        IEEE-754 certainly requires that. You also can't multiply two large
        numbers and end up with a small one on any hardware I know of.
        Overflow sticks at either Inf or the largest possible value, or
        traps. Now you can still end up with an answer that is a large
        number of orders of magnitude wrong mathematically, but it won't
        appear to be a small, reasonable result when it overflows.

        I consider your hardware / software emulation broken, although ANSI
        C doesn't.
        [color=blue]
        >I believe that it's the programmer's job to know what possible ranges
        >his values could take, and to check they're sane before performing
        >operations upon them.[/color]

        The problem here is that individually reasonable values may produce
        a collectively unreasonable result. For example, linear interpolation
        using two points that are nearly identical. Roundoff error in
        decimal conversion can also change a reasonable situation to an
        unreasonable one (e.g. change division by zero, which you've handled,
        to division by what was supposed to be zero but for roundoff error,
        which you didn't because in a corner case it was more than expected).
        [color=blue]
        >If this is done, then overflow and underflow
        >can never happen. Rarely is this done, of course, so the compromise
        >suggested by TPTB is to offer a safety net in the form of adding
        >inexpensive checks to the generated code which can propagate status
        >back to the user.[/color]
        [color=blue]
        >The approach I'm taking is to detect overflow or underflow and set a
        >flag (in the implementation namespace) as appropriate, but leave the
        >[invalid] value in the result.[/color]

        I believe the appropriate action is to substitute +Inf, -Inf,
        +DBL_MAX, or -DBL_MAX for overflow, or cause a trap.

        Underflow to zero is problematical as it provides no clear way to
        check for an error, but then it isn't always obvious when an underflow
        *IS* an error. Is 0.10 - 0.10 underflowing to zero an error?
        Or is it a case of "I had a dime and I spent it"?
        [color=blue]
        >This way, if it really is important to
        >the user to eke that last bit (pun not intended) of information out of
        >the operation then the way is clear for them to do so. For example, a
        >multiplicati on which overflows will end up with a value pow(2,256)
        >smaller than the correct [unrepresentable] value, thanks to wraparound
        >of our 8-bit signed binary exponent, but the mantissa will still have
        >full precision - some recovery may be possible.[/color]

        Do you know if anyone is actually trying to do that kind of recovery?
        It's very, very system-specific. How often do you deal in numbers
        that are within 10 orders of magnitude of DBL_MAX? I consider it
        dangerous.

        [color=blue]
        >An alternative approach (since we use an IEEE-754-compatible
        >representati on already) is to go the whole hog and implement
        >Infinities, NaNs, and Denormals.[/color]

        Overflow to +/- DBL_MAX might cover most of the issues if the
        whole IEEE implementation is too expensive.

        [color=blue]
        >At this time we don't want to do this
        >because it appears to be a significant cost (see below) for not much
        >return. In any case, how to deal with those types of value in C is
        >still not defined, so any solution of this type would still be Not C.[/color]

        How costly is it to check for the binary exponent overflow
        and overflow to +/- DBL_MAX and underflow to 0?
        [color=blue]
        >I'm interested to hear the c.l.c set of viewpoints on silent vs. noisy
        >underflow and overflow - in this case, coercing values to 0 or Inf,
        >vs. leaving them as is and setting a user-checkable flag.
        >('Very-noisy' would be throwing an exception of some kind, also
        >undefined by the Standards.) What practices to you use to avoid
        >hitting overflow and underflow?[/color]

        In most cases, range-checking the result to within sane values is
        sufficient, given overflow to +/- Inf or +/- DBL_MAX. Legitimate
        values are usually smaller in absolute value than the 10th root of
        DBL_MAX.

        It would be really nasty having 1e+200 * 1e+200 end up being 1e+4
        (two outrageous values ending up with a sane-looking product).
        [color=blue]
        >Is our plan to let the dodgy value
        >continue to exist of any conceivable value?[/color]

        I think it is of negative value, unless someone really needs to
        use the full range. How often do you deal with numbers like 1e+300?

        If someone really wants to check this hidden flag, could they not
        get the dodgy value from some other hidden place, where the
        normal result is DBL_MAX?
        [color=blue]
        >For those who've read this far: we make cross-compilers for [often
        >tiny; <256 bytes of RAM is common] CPUs commonly used in embedded
        >systems, and we do our damnedest to make them meet C90 and are inching
        >toward C99 in places. Floating point has become popular, some
        >deficiencies have been found, and Muggins put his hand up to fix them.[/color]

        Gordon L. Burditt

        Comment

        Working...