The CTIT computing lab is an environment that consists of two compute clusters, several compute servers and a small OpenNebula cloud.
Machines that you may log on to directly.¶
Note that you can only connect to these servers via SSH when you have a 130.89* address. Which means you have to be on campus or logged in via VPN.
|Hadoop/Yarn||wegdam/oldemeule||ctithead1/ctithead2||head node||do not run heavy jobs directly on this machine|
|HPC/Slurm||korenvliet||head node||do not run heavy jobs directly on this machine|
|Compute Server||everloo||Xeon Phi host||different OS than other machines|
|Compute Server||fmtmini||OS X build/test environment||FMT members only & only password less login|
We have several cluster partitions:Hadoop/Yarn
- 8 nodes with FDR Infiniband
- 32 nodes with QDR Infiniband
- 12 nodes
- 10 nodes with DDR Infiniband
- 1 (big)node
- 4 nodes with dual gpu's
The Hadoop cluster can be used for runnnig large scale computations,
but because of the nature of hadoop it should not be used for benchmarking.
To run benchmarks, use the HPC cluster.
For a complete listing of all hardware see Full list of hardware¶
Known Problems and Issues.¶
- Infiniband RDMA has problems on the R415 nodes (contact: Wytse Oortwijn)
- The R415's run two schedulers: YARN and SLURM. If the SLURM users are careless and bring down YARN/Hadoop then SLURM will be shut down on these nodes.
Who has access?¶
To get access, you need to have an AD account of the University of Twente. All students and employees have such an account and they can be arranged for external persons. To get your AD account enabled for the CTIT cluster, you need to contact one of the contact persons.
- Freark van den Berg for FMT
- Jan Flokstra for DB
- Geert Jan Laanstra for SCS
For staff, the username is probably your family name followed by your initials, for students its your student number starting with the "s".
CTIT Computing Lab does not store your password and we are unable to reset your password. If you require password assistance, please see the ICTS/LISA Servicedesk
Connecting with SSH¶
Access to CTIT Computing Lab resources is provided via secure shell (SSH) login. Most Unix-like operating systems (Mac OS X, Linux, etc) provide an ssh utility by default that can
be accessed by typing the command ssh in a terminal window.
You can login to one of the head nodes.
Note : Login nodes are not intended to be used for computationally intensive work. For running computationally intensive programs, see the Usage pages .
Linux & Mac¶
To login to a headnode from a Linux or Mac computer, open a terminal and at the command line enter:
$ ssh <UserId>@<NodeName>.ewi.utwente.nl
Note : replace UserId with your login name, nodename with one of the listed login nodes.
Logging in to ctithead1(/wegdam) or ctithead2(/oldemeule) will connected you to a “login node” of the Hadoop/Yarn Cluster.
Logging in to korenvliet will connect you to the "login node" of the HPC/SLURM Cluster.
Windows users will first need to download an ssh client(*PuTTY) which will allow you to interact with the remote Unix command line. When setting up your connection to the cluster in Putty, use the following information:
Logging in on fmtmini¶
Logging in on the fmtmini machine can only be done with SSH keys. The reason being that OS X does not support Linux UIDs. To login, upload your SSH public key to your home directory on Wegdam. The use of the fmtmini is restricted to fmt members.
To display the gui applications on your desktop, you need to enable (trusted) X11 forwarding. When connecting through ssh, add the "-Y" parameter.
$ ssh -Y <UserId>@<NodeName>.ewi.utwente.nl
Windows users needs to set this option in the Session Configuration (Connection->SSH->X11->Enable X11 forwarding).
_Note : Add the remote server to the X0.hosts file, to authorize displaying on your computer.
Data transfer (SCP)¶
Secure copy or SCP is a means of securely transferring computer files between a local host and a remote host. It is based on the Secure Shell (SSH) and Secure File Transfer (SFTP) protocols.
Most UNIX-like operating systems (Mac OS X, Linux, etc) provide a scp command which can be accessed from the command line. To transfer files from your local computer to your home directory on the cluster, open a terminal window and issue the command:
Single files: $ scp <some file> <UserID>@<NodeName>.ewi.utwente.nl: Directories: $ scp -r <some dir> <UserID>@<NodeName>.ewi.utwente.nl:
Or, to connect to a directory on the cluster (/deepstore/project2, for example) :
$ scp -r <some dir> <UserID>@<NodeName>.ewi.utwente.nl:/deepstore/project2
When prompted, enter your password.
WinSCP GUI for Windows Clients:¶
WinSCP is a scp client software that can be used to move files to and from the cluster and a Windows machine. WinSCP can be obtained from http://www.winscp.net.
When setting up your connection to the cluster in WinSCP, use the following information:
After connecting, if you are prompted to accept the server’s host key, select “yes.”
Upon successfully connecting, the main WinSCP window allows you to move files from your local machine (left side) to the cluster (right side).
The first cluster machines run Ubuntu Server 14.04 LTS, The second cluster machines run Ubuntu Server 16.04 LTS.
Some basic packages in the repositories have been installed. Additional software is available in the /software folder. See Local Software.
You can store small amount of data within your home folder (/home/username)
Data for the Hadoop cluster can be placed on the Hadoop Distributed File System (HDFS)
Data for the HPC cluster can be placed on the central storage (/deepstore, on request)
You are allowed to create a local folder to storage temporary data (/local/projectname or /local/username)
Keep in mind this is temporary data and may be wiped if the disk gets to full.
See the HPC info page for more information.
for more information contact Geert Jan Laanstra.
Analysing experiments (not related to SLURM)¶
Also attached is "models.tar.gz". This is an archive of a large set of models we benchmarked LTSmin with. Furthermore we have attached "analyse-experiments.php". This script can be used to analyse std out and std err output from thousands of experiments in seconds. This script can be used as reference (it may or may not suit your needs). The main result of this script are CSV files with results of the experiments. Also some Latex code is generated to quickly include these CSV files in your Latex documents.
A mailing list for cluster users has been created on the UTwente list server: CTIT_CLUS_USERS.
For the HPC/SLURM cluster, two mailing lists are created, CTIT-Cluster-Users (all the users) and CTIT-Cluster-Managers (all the managers).