Getting started on Iridis

Access to Iridis

Users with an account can connect to the  Iridis login node with ssh using the Internet address iridis2.soton.ac.uk. Access via telnet & rlogin has been discontinued for security reasons. See our ssh page for details of how to transfer files to and from your local computer and how to display a GUI from Iridis to your local machine.

File Store and Disk Space

Please note that your home filestore on Iridis2 is NOT your normal iSolutions home filestore. On Iridis2 you have an entirely new filestore in the directory /rhome/ (/rhome stands for research home). This directory is backed-up, but a quota limit of 10 GB per user has been applied on this filesystem. This limit will be strictly adhered to, so any filestore required in excess of this will need to use a /scratch directory. New users who think that they will need a directory in /scratch will need to request this. The recommended usage is that valuable files such as source files, input data files and results summaries should be kept in your /rhome directory. /scratch would then be used for working space for large files produced by jobs in progress, or recent job output that had not been analysed as yet. Important Note: while /scratch directories are backed up periodically, we are not in currently in a position to do a regular full backup, so the integrity of data in /scratch should not be relied on.

 A consequence of the different home filesystem is that any shell initialisation files in your normal iSolutions filestore, such as .login or .cshrc, will not be read on Iridis2. So if you rely on these you will need to copy them across to your /rhome idirectory and modify them as appropriate.

Applications with intensive I/O requirements should consider using the /tmp area on the disks local to each node for best performance. There should normally be at least 25-30 GB available on each node, but note that this area is cleared at the end of each job so users should copy any files they wish to save back to non-local filestore before the end of the job.

Editing and viewing files, working with directories

Users who are already familiar with Linux/Unix and the vi or emacs editors will need to read no further.  If you do not have any previous experience of using command line editors in Linux you will probably want to use the emacs or gedit text editors to modify files. The command emacs my_file displays a new window in which you can view and modify the contents of the text file my_file in an intuitive manner, without having to know anything about the many more powerful capabilities of advanced emacs use.  (Note that you must be able to display new windows back to your local system - see our page on displaying a GUI. If you want to use a simple command-line editor in your original login window then nano is a good choice.)

To list which files and subdirectories you have in a directory use the ls command. (Directories are the equivalent of windows folders.)

To view or search the contents of  the text file some_file use the command less some_file. To learn how to move around in the file or search with less type the letter "h" for help while in less.  

Use the mkdir command to create a new directory, eg. mkdir new_dir to create a directory called new_dir. To change the directory use the cd command, eg. cd new_dir. Two dots are used to represent the directory above, so cd .. will change to the directory above. The pwd command will show you where you are in the directory structure.

Further information can be found on our introduction to Unix page.

 Compiling

The use of the Portland Group compilers is recommended on Iridis. For serial codes these are accessed as pgf77, pgf90, pgcc and pgCC, for f77, f90, C and C++ respectively. pghpf provides an HPF compiler. The gnu compilers gcc and g77 are also available. Access to the Intel compilers may also be arranged where this is essential.

When compiling MPI codes it is recommended to use the appropriate wrapper for the compilers to ensure that the correct libraries and include files are selected. These are mpif77 or mpif90 for Fortran and mpicc and mpicxx for C and C++.


Important Note: To use the MPICH versions you must either specify the full path to the compiler wrapper, eg. /local/mpich-pgi/bin/mpif90 or modify your default path so that the MPICH bin directory precedes /usr/bin. The unqualified use of these commands will normally pick up a LAM version using the gnu compilers which comes bundled with the O/S and is in /usr/bin on the default path.

Parallel Environments

MPI is the preferred environment for running parallel jobs. MPICH 1.2.6 with the PGI compilers is installed in /local/mpich-pgi. Alternatively a version using the Gnu compilers is installed in /local/mpich-pgi. The use of the full path to mpirun is essential as explained in the note on compilers above.

More recently, OpenMPI 1.2.4 has been installed and can generate a single executable that will run on both the GigE network and the Myrinet networks.  This is best used in conjunction with the appropriate environment module, to set up paths and runtime libraries.

Support for shared-memory parallelism using OpenMP is available within the Portland Compilers, but this can only be used between the dual processors within each node. If you prefer a data parallel approach, or have inherited such a code, there is also an HPF compiler pghpf bundled with the Portland Compilers, which is licensed to run on upto 16 CPUs.

Many applications such as Fluent, CFX, Ansys or Abaqus have their own provision for running in parallel over multiple processor-cores (if sufficient licences are available), more details are given in the software pages for each application.

Running Jobs

All long-running compute-intensive applications be should be run on the compute nodes (not the login node as this degrades performance for all users and makes you quite unpopular). There are currently over 370 computational nodes which are normally accessed via the PBS/Torque resource manager (though some users may prefer to access the system via a GRID interface). Most jobs will be run in batch mode, though interactive jobs are possible. The order in which jobs are run is determined by the Maui scheduler. More documentation on how to submit jobs is given in the links below.

An additional interactive node is available, as an alternative to the login node. The functionality is almost identical to the login node. It is intended to be used for shortish sessions where CPU-intensive interactive access is essential(in conjunction with a GUI interface perhaps), but not as an alternative to the batch queue. Please be responsible and considerate to other users in your usage of this node. This node may be accessed by connecting to blue14.iridis.soton.ac.uk. For long compute-intensive interactive jobs it may be more appropriate to run an interactive PBS job on the compute nodes.

User Support

System problems and general queries should be directed primarily to ServiceLine.
Dedicated support is available to assist users with running and developing applications on Iridis. Application-related queries and requests for assistance can be sent to Ivan Wolton. email: icw@soton.ac.uk

Further Information

We are still working on the best way to integrate the recent additions of dual-core core nodes  and Myrinet-connected nodes into the job-submission system. Interim information on how to submit jobs for these nodes is available.

The following documents are still under development. They will be periodically revised as the service matures.