pickle and infinity

[Bart Ogryczak]

Hello,
I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error). What could I do about it? Example code:

>>x = 1e310 #actually it would be a result of calculations
>>type(x)

<type 'float'>

>>x

1.#INF

>>import pickle
>>pickle.loads( pickle.dumps(x) )

[...]
ValueError: invalid literal for float(): 1.#INF

[Bart Ogryczak]

Re: pickle and infinity

To make things more interesting -- Solaris version:

>>x = 1e310
>>x

Infinity

>>import pickle
>>pickle.dumps( x)

'FInfinity\n.'

>>pickle.loads( _)

Infinity

>>pickle.dumps( x,1)

[...]
SystemError: frexp() result out of range

[Grant Edwards]

Re: pickle and infinity

On 2006-11-29, Bart Ogryczak <B.Ogryczak@gma il.comwrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return
overflow/underflow error). What could I do about it?

Here's what I did. I'm sure it'll fall down on some systems,
but it works on Linux and Windows.

_______________ _______________ _______________ _______________ __________
# unpickle can't handle nan/inf/ind floats, so we need to
# handle that ourselves

def myload_float(se lf):
s = self.readline()[:-1]
try:
f = float(s)
except ValueError:
s = s.upper()
if s in ["1.#INF", "INF"]:
f = 1e300*1e300
elif s in ["-1.#INF", "-INF"]:
f = -1e300*1e300
elif s in ["NAN","1.#QNAN" ,"-1.#QNAN","QNAN" ,"1.#IND","IND" ,"-1.#IND"]:
f = (1e300*1e300)/(1e300*1e300)
else:
raise ValueError, "Don't know what to do with "+`s`
self.append(f)

# unpickle routine that overrides the float load method

def unpickle(f):
unpickler = pickle.Unpickle r(f)
unpickler.dispa tch[pickle.FLOAT] = myload_float
return unpickler.load( )
_______________ _______________ _______________ _______________ __________

--
Grant Edwards grante Yow! I represent a
at sardine!!
visi.com

[Fredrik Lundh]

Re: pickle and infinity

Bart Ogryczak wrote:

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

</F>

[Bart Ogryczak]

Re: pickle and infinity


Fredrik Lundh wrote:

Bart Ogryczak wrote:
>

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

>
Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

Actually I´d be very happy, if Python would throw FloatPointExept ion,
rather then allow infinite values. Haven´t got idea how to do it.
fpectl would do the trick, but it´s not present nither on Solaris, nor
on Windows platform. And I´d need it on both.

[Grant Edwards]

Re: pickle and infinity

On 2006-11-29, Bart Ogryczak <B.Ogryczak@gma il.comwrote:

>
Fredrik Lundh wrote:

>Bart Ogryczak wrote:
>>

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

>>
>Python 2.X relies on the C library to serialize floats, and, as you've
>noticed, some C libraries can produce values that they themselves cannot
>read.

>
Actually I´d be very happy, if Python would throw FloatPointExept ion,
rather then allow infinite values.

I'd be very unhappy. I use both NaNs and infinities in many
programs, so it needs to be selectable should that feature be
added.

Haven´t got idea how to do it. fpectl would do the trick, but
it´s not present nither on Solaris, nor on Windows platform.
And I´d need it on both.

In my experience anything than needs to be portable and
involves NaN, inf, or denormals is rather painful.

--
Grant Edwards grante Yow! I wish I was a
at sex-starved manicurist
visi.com found dead in the Bronx!!

[Bart Ogryczak]

Re: pickle and infinity


Grant Edwards wrote:

On 2006-11-29, Bart Ogryczak <B.Ogryczak@gma il.comwrote:

Fredrik Lundh wrote:

Bart Ogryczak wrote:
>
I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).
>
Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

Actually I´d be very happy, if Python would throw FloatPointExept ion,
rather then allow infinite values.

>
I'd be very unhappy. I use both NaNs and infinities in many
programs, so it needs to be selectable should that feature be
added.

Like I´ve said. I´d be satisfied with something like fpectl, which
gives you option to either throw FloatPointExept ion or go on with NaNs
and infs. NaNs and infinities are pain in the a** if you´re
integrating with C libraries or you´re using pickle, pyro, whatever.
The problem is, that the error pops out in the middle of the code that
really has nothing to do with the actual erroneous calculation. Its
impossible to trace back where the hell the NaN or infinite value comes
from.

Haven´t got idea how to do it. fpectl would do the trick, but
it´s not present nither on Solaris, nor on Windows platform.
And I´d need it on both.

>
In my experience anything than needs to be portable and
involves NaN, inf, or denormals is rather painful.

That´s not very comforting :-/