Introduction to Clusters: Build Yourself a PC Cluster NOW!

SuperComputing 2001, Denver, Colorado

Joint Institute For Computational Science
K.Wong, M.Grubert, and C.Halloy
November 11th,  2001

 

Lab #3:

Software Installation on a PC Cluster

 

---

Table of Contents
 

1. Objective
2. Preparation
3. Installation of Several Popular Packages
4. Acknowledgements

Objective

Preparation

• Root Privilege

You will need root privilege for most of the installations. Log on as root.
 
• Package Source

Many of the following software packages can be obtained in a Redhat RPM format from the UTK DII site
• ftp://helios.dii.utk.edu/pub/linux/redhat/redhat/redhat-7.1-en/os/i386/RedHat/RPMS/


You will need to come to this repository several times in this part of the lab.
 

• Central Storage

You may want to keep the packages you downloaded in a central storage area for later use. One way to do this is to create an archives directory under /usr/local/ and put every original downloaded packages there. You may want to do the same for this part of the lab.

Installation of Several Popular Packages

• BLAS
1. Obtain the blas-3.0-9.i386.rpm and blas-man-3.0-9.i386.rpm from the UTK DII side and save it in your /usr/local directory

$ rpm -ivh blas-3.0-9.i386.rpm
$ rpm -ivh blas-man-3.0-9.i386.rpm
 

2. The achieve file should have been put in the /usr/lib directory.
   
• LAPACK
1. Obtain the lapack-3.0-9.i386.rpm and lapack-man-3.0-9.i386.rpm from the UTK DII side and put it into /usr/local.

$ rpm -ivh lapack-3.0-9.i386.rpm
$ rpm -ivh lapack-man-3.0-9.i386.rpm
 

2. The archive file should have been put in the /usr/lib directory.
   
• Gnuplot
1. Obtain the gnuplot-3.7.1-12.i386.rpm  from the UTK DII side and put it into /usr/local.

$ rpm -ivh gnuplot-3.7.1-12.i386.rpm
 

2. The archive file should have been put in the /usr/lib directory.
   
• Octave
1. Download the octave-2.1.33-2.i386.rpm  from the UTK DII side and put it into /usr/local.

$ rpm -ivh octave-2.1.33-2.i386.rpm
 

2. The archive file should have been put in the /usr/lib directory.
   
• Java
1. Obtain the j2sdk-1_3_1_01-linux-i386-rpm.bin  from the UTK DII side and put it into /usr/local.

$ ./j2sdk-1_3_1_01-linux-i386-rpm.bin

Go through the license agreement and type yes, then press Enter

$rpm -ivh j2sdk-1_3_1_01.i386.rpm


2. The archive file should have been put in the /usr/lib directory.
   
• Atlas
1. You will need to install Atlas for the Parallel Matrix Multiplication Benchmark package to work properly. Go to the following site
1. http://www.netlib.org/atlas/archives
2. Get the atlas3.2.0_Linux_PIIISSE1256.tgz package and save it under /usr/local/scalapack.
3. Unpack it:

$ tar zxvf atlas3.2.0_Linux_PIIISSE1256.tgz
$ cd Linux_PIIISSE1256
$ mv libatlas.a ..
 

4. Now the libatlas.a library is in /usr/local/scalapack directory ready for use.
   
• Parallel Virtual Machine (PVM)
1. Create a directory /usr/local/pvm3
2. Obtain the Redhat RPM package for PVM, pvm3-4.0-27.i386.rpm, from the UTK DII site and save it in your /usr/export/archives directory.

$ rpm -ivh pvm3-4.0-27.i386.rpm
 

3. Note:

You don't have to do this now, but remember, several environment variables need to be set up in a user account (e.g. ypuser) before the user can start to use PVM. The way to do this is:
 
1. Append the following lines to the .bashrc file in the user's home directory:

export PVM_ROOT=/usr/pvm3
export PVM_ARCH=LINUX
export MANPATH=$PVM_ROOT:$MANPATH
export PATH=$PATH:$PVM_ROOT/lib:$HOME/pvm3/bin/$PVM_ARCH:.
 

2. Update the .bashrc file

$ source .bashrc
 

3. PVM can then be invoked by

$pvm
 

4. Make sure to kill pvm by typing halt in the pvm console

$ pvm> halt

4. For programming in PVM, please refer to JICS PVM workshop for details.
   
• MPICH
1. Get version 1.2.2.2 of MPICH from the following site:

http://www.mcs.anl.gov/mpi
2. Download mpich.tar.gz to /usr/export, then

$ gunzip -c mpich.tar.gz | tar xovf -
$ ln -s mpich-1.2.2.2 mpich (-- for convenience)
$ cd mpich
$ ./configure
$ make >& make.log &
(-- don't do "$ make testing" here. It won't work since you are the root.)
 

3. Note:

Remember, to make MPI work for a particular user, you need to set the appropriate environment variables first. The way to do it:
1. Append the following lines to the .bashrc file in the user's home directory:
# SETUP F0R MPI
export MPI_HOME=/usr/export/mpich
export PATH=$PATH:$MPI_HOME/bin:$MPI_HOME/lib
export MANPATH=$MPI_HOME/man:$MANPATH
 
1. Update the .bashrc file

$ source .bashrc
 

1. Edit the file ${MPIR_HOME}/util/machines/machines.LINUX. This is the default machine file and should be edited to contain the names of all machines in the cluster, listed one per line.
2. Example files can be found in /usr/export/mpich/basic/examples
3. For programming in MPI, refer to the JICS MPI workshop for details.
   
• ScaLAPACK
1. It is easier to download the prebuilt Scalapack version for LINUX from following site
• http://www.netlib.org/scalapack/scalapack_home.html
2. Make a scalapack directory under /usr/local.
3. Go to the Prebuilt Scalapack page in the site above and download scalapack.tgz for LINUX to your /usr/local/scalapack directory.
4. Go to the Prebuilt BLACS page and download blacs_MPI-LINUX-0.tgz and blacs_PVM-LINUX-0.tgz to /usr/local/scalapack
5. Go to the Example Programs to obtain readme and example1.f
6. Installation :

$ tar zxvf scalapack.tgz
$ tar zxvf blacs_MPI_LINUX-0.tgz
$ tar zxvf blacs_PVM_LINUX-0.tgz
1. The archive files are in the working directory /usr/export/scalapack
2. You can move those .tgz packages to /usr/export/archives directory now.
   
• Serial Matrix Multiplication Benchmark
1. Get MM kernel from the following site:

http://www.jics.utk.edu/2001/lib

2. Download MMPIII.tar.gz into the /usr/local directory 

$ gunzip -c MMPIII.tar.gz | tar -xovf -
$ cd Matmul
$ make matmul0 (or $make matmul1, or make matmul2)
$ ./matmul
 

3. Make sure that everything goes on alright. Then move the .tar files to /usr/local/archives.
   
• Parallel Matrix Multiplication Benchmark
1. First get the precompiled libraries from www.jics.utk.edu/SC2001/lib. Download SC2001-lib.tar.gz into /usr/local and extract it: 

$ gunzip -c SC2001-lib.tar.gz | tar -xovf -


2. Important: To fully test this installation, you need to logout and then logon as a user (e.g. ypuser). Do this now.
3. Get Parallel MM kernel from the following site:

http://www.jics.cs.utk.edu/SC2001/lib

4. Create a Bench directory in your home directory.
5. Download PMMPIII.tar.gz to ~ypuser//Bench, then
6. Important: Edit the .bashrc file and update the environment variables for the user as specified in the MPICH installation part.
7. Install the package and test it.
$gunzip -c PMMPIII.tar.gz | tar xovf -
$ cd pblas
(--change PBLAS.dat file for size of matrices and number of processors
$ make
$ make run2 (since you may be using a two-node cluster, change the number if you are using a different number of nodes)
(--used the given time in seconds to calculate the MFLOPS)
8. Check that everything is working.

Acknowledgments

• Argonne National Laboratory
• Oak Ridge National Laboratory
• Redhat Linux
• Kachina Technologies Inc.
• GNU Project