LinearAlgebra incredibly slow for eigenvalue problems

**David M. Cooke** · Jul 18 '05, 08:49 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

"drife" <daranrife@yaho o.com> writes:
[color=blue]
> Hello,
>
> I need to calculate the eigenvectors and eigenvalues for a 3600 X 3600
> covariance matrix.
>
> The LinearAlgebra package in Python is incredibly slow to perform the
> above calculations (about 1.5 hours). This in spite of the fact that
> I have installed Numeric with the full ATLAS and LAPACK libraries.
>
> Also note that my computer has dual Pentium IV (3.1 GHz) processors
> with 2Gb ram.
>
> Every Web discussion I have seen about such issues indicates that
> one can expect huge speed ups if one compiles and installs Numeric
> linked against the ATLAS and LAPACK libraries.[/color]

Are you *sure* that Numeric is linked against these?
[color=blue]
> Even more perplexing is that the same calculation takes a mere 7 min
> in Matlab V6.5. Matlab uses both ATLAS and LAPACK.
>
> Moreover, the above calculation takes the same amount of time for
> Numeric to complete with --and-- without ATLAS and PACK. I am certain
> that I have done the install correctly.[/color]

This is good evidence that Numeric *isn't* linked to them.

If you're on a Linux system, you can check with ldd:
cookedm@arbutus $ ldd /usr/lib/python2.3/site-packages/Numeric/lapack_lite.so
liblapack.so.3 => /usr/lib/atlas/liblapack.so.3 (0x0000002a9567 7000)
libblas.so.3 => /usr/lib/atlas/libblas.so.3 (0x0000002a95e5 5000)
libg2c.so.0 => /usr/lib/libg2c.so.0 (0x0000002a9672 1000)
libpthread.so.0 => /lib/libpthread.so.0 (0x0000002a9684 2000)
libc.so.6 => /lib/libc.so.6 (0x0000002a9695 7000)
libm.so.6 => /lib/libm.so.6 (0x0000002a96b9 6000)
/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000000552aaa a000)

You can see that lapack and blas (the Atlas versions) are linked to
the lapack_lite.so.

To install Numeric using Lapack:
- remove the build/ directory in your Numeric sources, so you don't
any old binaries
- edit setup.py and follow the comments on using Lapack (you need to
comment out a few lines, and set some directories)
Also set use_dotblas to 1.
- do the 'python setup.py build', 'python setup.py install' dance.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)phy sics(dot)mcmast er(dot)ca

**drife** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

Hi David,

I performed the above check, and sure enough, Numeric
is --not-- linked to the ATLAS libraries.

I followed each of your steps outlined above, and Numeric
still is not linking to the ATLAS libraries.

My setup.py file is attached below.

Thanks ,

Daran

--#!/usr/bin/env python
# To use:
# python setup.py install
# or:
# python setup.py bdist_rpm (you'll end up with RPMs in dist)
#
import os, sys, string, re
from glob import glob
if not hasattr(sys, 'version_info') or sys.version_inf o <
(2,0,0,'alpha', 0):
raise SystemExit, "Python 2.0 or later required to build Numeric."
import distutils
from distutils.core import setup, Extension

# Get all version numbers
execfile(os.pat h.join('Lib','n umeric_version. py'))
numeric_version = version

execfile(os.pat h.join('Package s', 'MA', 'Lib', 'MA_version.py' ))
MA_version = version

headers = glob (os.path.join ("Include","Num eric","*.h"))
extra_compile_a rgs = [] # You could put "-O4" etc. here.
mathlibs = ['m']
define_macros = [('HAVE_INVERSE_ HYPERBOLIC',Non e)]
undef_macros = []
# You might need to add a case here for your system
if sys.platform in ['win32']:
mathlibs = []
define_macros = []
undef_macros = ['HAVE_INVERSE_H YPERBOLIC']
elif sys.platform in ['mac', 'beos5']:
mathlibs = []

# delete all but the first one in this list if using your own
LAPACK/BLAS
sourcelist = [os.path.join('S rc', 'lapack_litemod ule.c')]
# set these to use your own BLAS;

library_dirs_li st = ['/d2/lib/atlas']
libraries_list = ['lapack', 'ptcblas', 'ptf77blas', 'atlas', 'g2c']

# set to true (1), if you also want BLAS optimized
matrixmultiply/dot/innerproduct
use_dotblas = 1
include_dirs = ['/d2/include']
# You may need to set this to find cblas.h
# e.g. on UNIX using ATLAS this should be
['/usr/include/atlas']
extra_link_args = []

# for MacOS X to link against vecLib if present
VECLIB_PATH = '/System/Library/Frameworks/vecLib.framewor k'
if os.path.exists( VECLIB_PATH):
extra_link_args = ['-framework', 'vecLib']
include_dirs = [os.path.join(VE CLIB_PATH, 'Headers')]

# The packages are split in this way to allow future optional inclusion
# Numeric package
packages = ['']
package_dir = {'': 'Lib'}
include_dirs.ap pend('Include')
ext_modules = [
Extension('_num py',
[os.path.join('S rc', '_numpymodule.c '),
os.path.join('S rc', 'arrayobject.c' ),
os.path.join('S rc', 'ufuncobject.c' )],
extra_compile_a rgs = extra_compile_a rgs),
Extension('mult iarray',
[os.path.join('S rc', 'multiarraymodu le.c')],
extra_compile_a rgs = extra_compile_a rgs),
Extension('umat h',
[os.path.join('S rc', 'umathmodule.c' )],
libraries = mathlibs,
define_macros = define_macros,
undef_macros = undef_macros,
extra_compile_a rgs = extra_compile_a rgs),
Extension('arra yfns',
[os.path.join('S rc', 'arrayfnsmodule .c')],
extra_compile_a rgs = extra_compile_a rgs),
Extension('ranl ib',
[os.path.join('S rc', 'ranlibmodule.c '),
os.path.join('S rc', 'ranlib.c'),
os.path.join('S rc', 'com.c'),
os.path.join('S rc', 'linpack.c')],
extra_compile_a rgs = extra_compile_a rgs),
Extension('lapa ck_lite', sourcelist,
library_dirs = library_dirs_li st,
libraries = libraries_list,
extra_link_args = extra_link_args ,
extra_compile_a rgs = extra_compile_a rgs)
]

# add FFT package (optional)
packages.append ('FFT')
package_dir['FFT'] = os.path.join('P ackages','FFT', 'Lib')
include_dirs.ap pend(os.path.jo in('Packages',' FFT','Include') )
ext_modules.app end(Extension(' FFT.fftpack',
[os.path.join('P ackages','FFT', 'Src',
'fftpackmodule. c'),
os.path.join('P ackages', 'FFT', 'Src',
'fftpack.c')],
extra_compile_a rgs = extra_compile_a rgs))

# add MA package (optional)
packages.append ('MA')
package_dir['MA'] = os.path.join('P ackages', 'MA', 'Lib')

# add RNG package (optional)
packages.append ('RNG')
packages.append ('RNG')
package_dir['RNG'] = os.path.join('P ackages', 'RNG', 'Lib')
include_dirs.ap pend(os.path.jo in('Packages', 'RNG', 'Include'))
ext_modules.app end(Extension(' RNG.RNG',
[os.path.join('P ackages', 'RNG', 'Src',
'RNGmodule.c'),
os.path.join('P ackages', 'RNG', 'Src',
'ranf.c'),
os.path.join('P ackages', 'RNG', 'Src',
'pmath_rng.c')],
extra_compile_a rgs = extra_compile_a rgs))
# add dotblas package (optional)
if use_dotblas:
packages.append ('dotblas')
package_dir['dotblas'] = os.path.join('P ackages', 'dotblas',
'dotblas')
ext_modules.app end(Extension(' _dotblas',
[os.path.join('P ackages', 'dotblas',
'dotblas', '_dotblas.c')],
library_dirs = library_dirs_li st,
libraries = libraries_list,

extra_compile_a rgs=extra_compi le_args))

long_descriptio n = """
Numerical Extension to Python with subpackages.

The authors and maintainers of the subpackages are:

FFTPACK-3.1
maintainer = "Numerical Python Developers"
maintainer_emai l = "numpy-discussion@list s.sourceforge.n et"
description = "Fast Fourier Transforms"
url = "http://numpy.sourcefor ge.net"

MA-%s
author = "Paul F. Dubois"
description = "Masked Array facility"
maintainer = "Paul F. Dubois"
maintainer_emai l = "dubois@users.s f.net"
url = "http://sourceforge.net/projects/numpy"

RNG-3.1
author = "Lee Busby, Paul F. Dubois, Fred Fritsch"
maintainer = "Paul F. Dubois"
maintainer_emai l = "dubois@users.s f.net"
description = "Cray-like Random number package."
""" % (MA_version, )

# Oops, another bug in Distutils!?
# Write rpm_build.sh pointing to this python
rpm_build_text= """env CFLAGS="$RPM_OP T_FLAGS" %setup.py build\n""" %
sys.executable
rpm_script = open("rpm_build .sh", "w")
rpm_script.writ e(rpm_build_tex t)
rpm_script.clos e()

# Write rpm_install.sh pointing to this python
rpm_install_tex t=sys.executabl e +""" setup.py install
--root=$RPM_BUILD _ROOT

cat >INSTALLED_FILE S <<EOF
%doc Demo
EOF
find $RPM_BUILD_ROOT -type f | sed -e "s|$RPM_BUILD_R OOT||g"[color=blue][color=green]
>>INSTALLED_FIL ES[/color][/color]

"""
rpm_script = open("rpm_insta ll.sh", "w")
rpm_script.writ e(rpm_install_t ext)
rpm_script.clos e()

setup (name = "Numeric",
version = numeric_version ,
maintainer = "Numerical Python Developers",
maintainer_emai l = "numpy-developers@list s.sourceforge.n et",
description = "Numerical Extension to Python",
long_descriptio n = long_descriptio n,
url = "http://numpy.sourcefor ge.net",

# distutils allows you to fix or fudge anything :-)
extra_path = 'Numeric',
packages = packages,
package_dir = package_dir,
headers = headers,
include_dirs = include_dirs,
ext_modules = ext_modules
)

print 'MA Version', MA_version
print 'Numeric Version', numeric_version

**John Hunter** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

>>>>> "drife" == drife <daranrife@yaho o.com> writes:

drife> Hi David, I performed the above check, and sure enough,
drife> Numeric is --not-- linked to the ATLAS libraries.

drife> I followed each of your steps outlined above, and Numeric
drife> still is not linking to the ATLAS libraries.

drife> My setup.py file is attached below.

Are you sure that you

1) 'rm -rf build' before rebuilding

2) are using the python / Numeric that you think you are, eg, might
you have one python in /usr/bin and another in /usr/local. root
paths can differ from user paths which may effect which python is
used at install time versus run time

**drife** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

Hi John,

I do have more than one version of Python laying around.

To do the build and install I am typing:

/d2/python/bin/python setup.by build > &! build.out
/d2/python/bin/python setup.by install > &! install.out
Should I be doing something different?

Daran

**David M. Cooke** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

"drife" <daranrife@yaho o.com> writes:
[color=blue]
> Hi David,
>
> I performed the above check, and sure enough, Numeric
> is --not-- linked to the ATLAS libraries.
>
> I followed each of your steps outlined above, and Numeric
> still is not linking to the ATLAS libraries.
>
> My setup.py file is attached below.[/color]
[color=blue]
> # delete all but the first one in this list if using your own LAPACK/BLAS
> sourcelist = [os.path.join('S rc', 'lapack_litemod ule.c')]
> # set these to use your own BLAS;
>
> library_dirs_li st = ['/d2/lib/atlas']
> libraries_list = ['lapack', 'ptcblas', 'ptf77blas', 'atlas', 'g2c']
>
> # set to true (1), if you also want BLAS optimized matrixmultiply/dot/innerproduct
> use_dotblas = 1
> include_dirs = ['/d2/include'][/color]

This all look right (assuming you've got the right stuff in /d2).

When it compiles, does it look like it's actually doing the linking?
After doing python setup.py build, you can run ldd on the libraries in
the build directory (something like build/lib.linux-i386-2.3/lapack_lite.so) .
If that's linked, then it's not being installed right.

You don't have a previous Numeric installation that's being picked up
instead of the one you're trying to install, do you?

At the interpreter prompt, check that[color=blue][color=green][color=darkred]
>>> import Numeric
>>> Numeric.__file_ _[/color][/color][/color]

gives you something you're expecting, and not something else.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)phy sics(dot)mcmast er(dot)ca

**drife** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

Hi David,

Yes, when Numeric compiles it does look like the linking is being
done properly. I captured the build to a file and see a few lines
similar to:

gcc -pthread -shared build/temp.linux-i686-2.4/Src/lapack_litemodu le.o
-L/d2/lib/atlas -llapack -lptcblas -lptf77blas -latlas -lg2c -o
build/lib.linux-i686-2.4/lapack_lite.so

I checked the files in build/lib.linux-i686-2.4 and none have
dependencies on ATLAS.

When I run Python interpreter with:
[color=blue][color=green][color=darkred]
>>> import Numeric
>>> Numeric.__file_ _[/color][/color][/color]

I get the answer I expect.
I am totall baffled.

Any other ideas?

Thanks for your help,

Daran

**drife** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

David,

One more thing. I checked to see if the SciPy libraries had
dependencies on ATLAS. They do not, however, the eigenvector
calculation is still much faster than Numeric?
This is very strange.

Daran

**drife** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

David,

I noticed that the libraries that ATLAS builds are not shared
objects (e.g., liblapack.a). Should these be shared objects?
I see nothing in the ATLAS documentation about building
things as shared objects.
Wondering if this is why the Numeric install is failing.

Daran

**Robert Kern** · Jul 18 '05, 08:50 PM

Re: LinearAlgebra incredibly slow for eigenvalue problems

drife wrote:[color=blue]
> David,
>
> I noticed that the libraries that ATLAS builds are not shared
> objects (e.g., liblapack.a). Should these be shared objects?
> I see nothing in the ATLAS documentation about building
> things as shared objects.
> Wondering if this is why the Numeric install is failing.[/color]

No, they should work just fine as static libraries.

--
Robert Kern
rkern@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

LinearAlgebra incredibly slow for eigenvalue problems

LinearAlgebra incredibly slow for eigenvalue problems

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment