PyExcelerator

**John Machin** · Jun 2 '06, 10:05 PM

struct.pack('d' ,...) -> "frexp( ) result out of range" (was Re: PyExcelerator)

On 3/06/2006 3:39 AM, tkpmep@hotmail. com wrote:[color=blue]
> I write data to Excel files using PyExcelerator 0.6.3.a and have done
> so successfully for small files (10-15 cells). I'm experiencing an
> error when writing a big chunk of data (10,000 cells) to Excel. By way
> of comparison, the same data writes perfectly well to a csv file using
> Python's built in csv module. I run the program in PyScripter, and the
> traceback shows the following sequence of calls:
>
> main (my routine)
> writeData (my routine)
> save Line 563
> get_biff_data Line 548
> get_biff_data Line 1357
> __row_blocks_re c Line 1276
> get_cells_biff_ data Line 200
> get_biff_data Line 106[/color]

Grumble: When you need to post a traceback, please supply one that shows
on each line which *file* the source line is in. In this case there are
multiple files, and 9 classes spread over 4 files have a method called
get_biff_data.
[color=blue]
>
> SystemError: frexp() result out of range
>
> The line it stops at in get_biff_data is the line that starts with
> packed =
>
> def get_biff_data(s elf):
> rk_encoded = 0
>
> packed = struct.pack('<d ', self.__number)
>
> Any thoughts on what the problem could be?
>[/color]

It is calling struct.pack (see structmodule.c in the Python source) to
encode one of your cell values as an 8-byte little-endian IEEE 754
floating-point number. This task is handed off to _PyFloat_Pack8( ) (see
floatobject.c). Here's the relevant bit:

"""
if (x < 0) {
sign = 1;
x = -x;
}
else
sign = 0;

f = frexp(x, &e);

/* Normalize f to be in the range [1.0, 2.0) */
if (0.5 <= f && f < 1.0) {
f *= 2.0;
e--;
}
else if (f == 0.0)
e = 0;
else {
PyErr_SetString (PyExc_SystemEr ror,
"frexp() result out of range");
return -1;
}
"""
where x is your cell value as a C double.

frexp() is a standard C library routine. The above code is just checking
that it has done its job correctly. AFAICT from reading K&R v2 p251, the
code is correct.

Aside: isn't open source wonderful? Well it is for me. It could be
wonderful for you too :-)

I'm not an IEEE 754 guru by any means, so I'm guessing your value could
be an Inf (infinity) or a NaN (not a number) -- there's no explicit test
for those in the code -- or you have stumbled on a value that's a corner
case on your platform. Maybe it's -0.0 (zero with the sign bit set).

Grumble: When you have a problem like this, you should state what
version of Python you are using, on what platform. The line you get when
you run Python at the command line is a start e.g.
"""Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)]
on win32"""

Please tell us this, plus explicitly what OS and what hardware you are
running on.

OK, now what's this dodgy number?? Unfortunately, the error is happening
right at the end of your process, in the workbook save method. The
only info about which row/column is buried deep in pyExcelerator. We
don't (yet) want to start poking about there ...

Try these steps:
1. Search your alternative CSV output file for the text "Inf" or "NaN"
(case insensitive).
2. Put some code in your cell write routine to filter the values you are
writing:

absval = abs(value)
SMALL = 0.000001 # depends on what is reasonable for your app
LARGE = 10.0 ** 10 # depends ...
if not (value == 0.0 or (SMALL <= absval <= LARGE)):
print "strange value %r in row=%d col=%d" % (value, rownum, colnum)

Then come back and tell us what if anything you've found.

HTH,
John

PyExcelerator

PyExcelerator

Comment