1. The ScaLAPACK routines have a systematic naming scheme which tells you alot about a function. For example PDDBSV is a double precision routine (PD) that solves a linear system (SV).
2. The ScaLAPACK routines are grouped into single precision, double precision, complex, complex16 categories at the bottom of the page. These links connect to an index of routines. So to see the code for PDDBSV click on the double precision link, and search for PDDBSV or to find single precision eigenvalue routines go to the single precision index and search on the key word eigenvalues.
3. The ScaLAPACK User's Guide is difficult to use, since the index and table of contents are not complete and often the active links don't have any contents. I found that doing a search on google was often an effect way to find information.
4.Finally, the site gives instructions for getting started with ScaLAPACK; Please note that all necessary libraries have already been installed on the cluster, so it is not necessary for you to compile any libraries in your home directory. Our installation of ScaLAPACK uses the ATLAS libraries installed on the cluster (these are optimized BLAS libraries).
SLmake.inc : This defines the necessary variables for the Makefile and will be the same for any ScaLAPACK program that you write. It contains all the library and header information that is used by the Makefile when linking your program.
Makefile : This make file has targets that create the executable xsmsolve xsolve xeig. If you are new to makefiles you can easily modify this simple example to accommodate any ScaLAPACK program that you create.
pdlaread.f: One specified process reads a matrix stored as a text file and either writes the data to its own submatrix or broadcasts the data to the appropriate process. On completion each process in the process grid has a submatrix. The global matrix exists only as a collection of submatrices in the grid, in other words no process in the grid ever has the whole global matrix as defined in the file -- it is a 2D block cyclically distributed matrix.
pdlawrite.f : If a 2D block cyclically distributed matrix is stored on the processes in a process grid one can use pdlawrite to have one process receive the values of the matrix from the other process and write them to a file. The result is the whole global matrix goes into one file.
pdscaexinfo.f : This simple program uses one process in the grid to read the header information from HEADER.dat and broadcasts this information to all other processes in the grid.
solve.f : This code calls pdlaread and scaexinfo to get the data needed to solve a large square system. It solves the system with a call to pdgesv, and writes the answer to the file sol.dat with a call to pdlawrite. The answer is also checked and the result is reported in the log file. It also shows how to make ScaLAPACK calls to calculate norms and perform Matrix vector multiplications.
smallsolve.f : This code does the same things as solve.f, but it also writes the whole matrix A, b and x to the log file returned at runtime. Memory problems occur if this is used for large files.
eig.f : This code calls pdlaread and
scaexinfo to get the Matrix. It finds some of the eigenvalues for a
general dense matrix. It calls PDGEHRD to reduce a real general
matrix A to
upper Hessenberg form H by an orthogonal similarity
transformation H = Q^T*A*Q. It then calculates the eigenvalues of H
with a call to the subroutine PDGEHQR.
PDGEHQR takes the upper Hessenberg form H of the real general matrix A
and calculates H's eigenvalues
which are the same as those of A. The eigenvalues are written to two
files realeig.dat and imageig.dat where each file contains the
respective parts of a complex vector;
eigenvalue = realeig + i*imageig.
ScaLAPACK documentation requires the block size (NB) >= 6 otherwise PDGEHQR doesn't work properly.
RHS.dat: This contains the write hand side of the system of equations. The first line contains the row and column size of the vector, each additional line contains one element from the vector.
MAT.dat: This contains the matrix of the linear system or the matrix whose eigenvalues are to be computed. The first line contains the row and column size of the Matrix, each additional line contains one element from the Matrix, organized a column at a time; the first n elements are the first column of A, the second n elements are the second column of A,....
HEADER.dat: A small file that contains basic information needed by all codes
listed here, including the process grid info. The file format is self
explanatory and an
explanation of the input/output parameters follows:
N : The order of the matrix A.
NRHS : The number of right hand sides.
NB : The size of the square blocks the matrices A and B are split
into.
P : The number of process rows.
Q : The number of process columns.
write.m: This is a simple matlab code that will take a matrix or vector from Matlab and write it to a file that is readable by pdlaread.f
Compiling: make [eig, solve, smsolve]
Running: scasub -mpimon -np [process columns * process rows] [xeig, xsolve, xsmsolve]
Error Codes:
When a ScaLAPACK routine returns the error code is stored in the variable INFO. The programs listed above print out this code in the log file for each of the major ScaLAPACK computations. INFO = 0 means the call was successfully completed, otherwise something is wrong.For example you might get the message:
PDGESV para 605 illegal value
This mens that INFO = -605 = -(100*i) + j so i = 6 and j = 5 implies that there is a problem with the 5th element of the array DESCA which is the 6th argument passed to PDGESV.
Work Space: A segmentation violation and INFO = 0 indicates that one of the ScaLAPACK routines needs more workspace. A fast and reliable method is to have the routine do an LWORK Query.