About this QuickStart Guide

Note - although essentially still correct, this guide has been superceded by an expanded version with updated information at http://hpc.ucalgary.ca/quickstart/terminus

This QuickStart guide gives a brief overview of the Terminus cluster at the University of Calgary. 

It is intended to be read by WestGrid account holders who are considering whether to move to the Terminus system from Matrix or who have otherwise been advised to run jobs on Terminus.  For more detailed information about the Terminus hardware and performance characteristics, available software, usage policies and how to log in and run jobs, see the main Terminus documentation at http://hpc.ucalgary.ca/quickstart/terminus.

Introduction

A large equipment donation from HP Labs, along with additional contributions from project partners Alberta Innovation & Science (AI&S), Western Economic Diversification (WED) Canada, HP Canada, and the University of Calgary, led to the installation of a cluster with more than 1000 cores at the University of Calgary in the fall of 2008. The primary purpose of the acquisition was for research into grid and utility computing. However, a portion of the cluster is being made available for High Performance Computing (HPC) cycles to WestGrid and other approved researchers.  Terminus is the name given to that part of the machine configured for HPC use.

The size of the Terminus cluster will vary depending upon the cycles required by researchers and by the awards of the HP Labs Resource Allocation Committee (HPL-RAC).  We expect at least 25% of the cycles on Terminus to be used by Alberta HPC researchers, and a further 25% to be used by our colleagues throughout WestGrid.  Please note that Terminus is not governed by WestGrid.  Although it can be used for signficant computations, keep in mind that it is essentially an experimental environment that may be subject to reconfiguration from time to time.

Terminus,  a Linux Opteron cluster similar to the WestGrid Matrix cluster, is intended for parallel jobs that can take advantage of its high-capacity, low-latency Infiniband interconnect.

Accounts

If you are a WestGrid account holder and would like a Terminus account, please write to support@hpc.ucalgary.ca with the subject line "Terminus account requested (username)", filling in your WestGrid username in the brackets.  In the body of the message, briefly indicate that you would like your Terminus account activated and give your name, department and institution.  If computations you intend to carry out on Terminus are not the same as described in your WestGrid application, please include a few sentences indicating the nature of your proposed work.

If you are not currently a WestGrid account holder, but are eligible to use WestGrid, as described in the Information for Prospective Users, apply for a WestGrid account at http://www.westgrid.ca/support/accounts.  After your WestGrid account has been set up, then, follow the directions above to request activation of your Terminus account.

There may be cases in which Terminus accounts are appropriate for those who are not eligible for WestGrid (such as in an undergraduate teaching situation). If you think this applies to you, please write to support@hpc.ucalgary.ca to discuss your requirements.

Hardware

Processors

Terminus is a cluster comprised of up to twenty HP C7000 chassis.  Each chassis houses sixteen BL465c G1 CTO Blades.  Each blade (compute node) contains two dual-core 2.4 GHz AMD Opteron processors.  So, if fully deployed towards an HPC task load, Terminus could have more than 1000 cores (20 chassis x 16 blades/chassis x 2 processors/blade x 2 cores/processor).

Approximately half the nodes have 4GB of memory and about half have 8 GB. One node has 16 GB.

Interconnect

The compute nodes communicate via Infiniband, a high-bandwidth, low-latency network.

Storage

There is about 9.5 TB of disk space allocated for home directories and global scratch space. Each user has a subdirectory in /scratch.

The compute nodes are connected to an HP SFS Storage Array by Infiniband.

Application software

Look for installed software under /usr/apps.  MATLAB and Gaussian are available only for University of Calgary researchers.  FLUENT and VASP are available only to approved license holders.  VMD access requires agreement to certain license conditions.  Write to support@hpc.ucalgary.ca if you need access to any of these restricted packages or additional software installed.

GROMACS 4 performs well on Terminus and may be used by any WestGrid researchers.

Support

Send Terminus-specific questions to support@hpc.ucalgary.ca. if issues being discussed overlap with work being done on WestGrid systems, you may write to support@westgrid.ca, as Terminus support personnel are also on the WestGrid list.