First Steps

The steps in this section are aimed at new users who would like to do any of the following tasks:

Run their own experiment
Analyse model data outputs
Browse observational data catalogues
Evaluate model performance
Perform other tasks involving ACCESS models or evaluation tools

Create an NCI user account

Most of the data and models you will need are available at the National Computing Infrastructure (NCI) .
To access these, you need an NCI account. If you do not have one, sign up here.

You will need an institutional email address with an organisation that allows access to NCI (e.g., an Australian university, ACCESS-NRI, CSIRO, BoM, CLEX, etc.).

Once you sign up, you will be assigned a username (e.g., ab1234).

Join relevant NCI projects

NCI provides multiple services that are necessary for climate research. These include the access to supercomputing resources, data storage, and data collections management.

For technical reasons, to access either of these services you need to join a specific project. Each project has an ID (e.g. xp65), which is what the term project actually refers to.

If you are interested in datasets and data collections, you can browse the NCI Data Catalogue and follow the NCI Data Catalogue User Guide.

To run models on Gadi instead, you need to join a project with computing resources, also called Service Units (SU). The project ID will be provided by your supervisor, research project or institution.

To join a project, search for it on NCI website and request membership.

The first project you join will become your default one. If you would like to change this, check out how to change your default project on Gadi.

There are several NCI projects that may be relevant to you, depending on the tasks you want to carry out.
For tasks supported by ACCESS-NRI (e.g., running a supported model configuration, using a supported model evaluation tool, etc.), you will find a list of relevant projects to join in the pages relative to each respective task.

Operations involving model runs and data collections take place on the Gadi supercomputer.

Before you login to Gadi, you need to possess the following prerequisites:

Internet connection
Terminal with built-in SSH
Linux, MacOS and Windows 10 (or later) operative systems already have a terminal with built-in SSH.
Users of Windows 9 (or earlier) can install Windows Subsystems for Linux (WSL).
Alternatively, you can login through the ARE Gadi Terminal.
However, it is recommended that you connect to Gadi from your local machine's terminal without using ARE.

To login to Gadi using SSH, on your local machine's terminal run the following command, replacing <your-NCI-username> with your NCI username (e.g., ab1234):

ssh <your-NCI-username>@gadi.nci.org.au

You will be prompted to enter your NCI password, and then you will be connected to Gadi: ssh <your-NCI-username>@gadi.nci.org.au <NCI-username>@gadi.nci.org.au's password: ############################################################################### # Welcome to the NCI National Facility! # # This service is for authorised clients only. It is a criminal # # offence to: # # - Obtain access to data without permission; # # - Damage, delete, alter or insert data without permission; # # Use of this system requires acceptance of the Conditions of Use; # # published at https://nci.org.au/users/nci-terms-and-conditions-access; # ############################################################################### | gadi.nci.org.au - 260,760 processor InfiniBand x86_64 cluster | =============================================================================== ===============================================================================

To simplify the login and avoid being prompted every time to enter your NCI password, follow these steps:

Create an SSH key
To create an SSH key on your local machine, run:
```
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_gadi
```
You will be prompted to create a passphrase linked to the SSH key, which you will enter twice: ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_gadi Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in <$HOME>/.ssh/id_gadi Your public key has been saved in /Users/davide/.ssh/id_gadi.pub The key fingerprint is: SHA256:<fingerprint-code> <$USER@hostname> The key's randomart image is: +---[RSA 4096]----+ |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| |xxxxxxxxxxxxxxxxx| +----[SHA256]-----+
For security reasons, it is recommended to enter a passphrase rather than leave it empty.
As you will see in the next step, you will not need to enter the passphrase every time you login to Gadi.
Add the SSH key to the SSH-agent
An SSH-agent is an SSH key manager that avoids you having to enter your passphrase every time you connect to a server.
To add the SSH key to the SSH-agent:
1. On your local machine, start the SSH-agent by running:
```
eval "$(ssh-agent -s)"
```
  eval "$(ssh-agent -s)" Agent pid <agent-PID>
2. Add your SSH key to the SSH-agent by running:
```
ssh-add --apple-use-keychain ~/.ssh/id_gadi
```
  You will be prompted to enter your SSH key passphrase, which will be stored inside the SSH-agent: ssh-add --apple-use-keychain ~/.ssh/id_gadi Enter passphrase for <$HOME>/.ssh/id_gadi: Identity added: <$HOME>/.ssh/id_gadi <$USER@hostname>
  If you are using a MacOS version prior to Monterey (12.0), substitute the --apple-use-keychain flag with -K.
```
ssh-add ~/.ssh/id_gadi
```
  You will be prompted to enter your SSH key passphrase, which will be stored inside the SSH-agent: ssh-add ~/.ssh/id_gadi Enter passphrase for <$HOME>/.ssh/id_gadi: Identity added: <$HOME>/.ssh/id_gadi <$USER@hostname>
Create/Update the SSH config file
The ~/.ssh/config is a file where you can store labelled SSH configurations for different remote servers you regularly connect to, so you do not have to remember them all.
To create an SSH config file, run the following command on your local machine:
```
touch ~/.ssh/config
```
If you already have an existing ~/.ssh/config file, the above command will not have any effect.
The following lines should be added to your ~/.ssh/config to describe the SSH configuration for Gadi (replace <your-NCI-username> with your NCI username, e.g., ab1234):
```
Host gadi
    Hostname gadi.nci.org.au
    User <your-NCI-username>
    ForwardX11 true
    ForwardX11Trusted yes
    IdentityFile ~/.ssh/id_gadi
    AddKeysToAgent yes
    UseKeychain yes
```
If you already have an existing ~/.ssh/config file which contains configurations for every Host (e.g., by using Host *), make sure you delete any of the keywords present in that SSH configuration from the Gadi configuration above.
Add the SSH key to the authorised keys
To enable automatic connection to a server, that server needs to recognise the SSH key as authorised. The list of authorised keys for a certain server is stored in the file ~/.ssh/authorized_keys.
To add the newly created SSH key as an authorised key for Gadi, run the following command from your local machine:
```
var=$( cat ~/.ssh/id_gadi.pub ) && ssh gadi "echo $var >> .ssh/authorized_keys"
```
Make sure to use double quotes " in the above command.
You will be prompted to enter your NCI password. If you did all of the above steps correctly, this should be the last time you need to do so.

Once you complete all the above steps, you should be able to connect to Gadi from your local machine's terminal simply by running:

ssh gadi

Change default project on Gadi

It is recommended that you check what your default project on Gadi is set to. The default project should be set to the computational project you will most likely use to run simulations/evaluations and store data.
You can check which is your default project by logging into Gadi and running:

echo $PROJECT

To change your default project on Gadi, you need to manually change the PROJECT field in the ~/.config/gadi-login.conf file.
Alternatively, you can run the following command on Gadi's terminal:

sed "s/\(PROJECT \).*/\1<new-default-project>/" ~/.config/gadi-login.conf

Once this is done, exit from Gadi and log back in.
For example, if you want to change your default project to tm70 on Gadi: echo $PROJECT <old-default-project> sed "s/(PROJECT ).*/\1tm70/" ~/.config/gadi-login.conf exit logout Connection to gadi.nci.org.au closed. ssh gadi ############################################################################### # Welcome to the NCI National Facility! # # This service is for authorised clients only. It is a criminal # # offence to: # # - Obtain access to data without permission; # # - Damage, delete, alter or insert data without permission; # # Use of this system requires acceptance of the Conditions of Use; # # published at https://nci.org.au/users/nci-terms-and-conditions-access; # ############################################################################### | gadi.nci.org.au - 260,760 processor InfiniBand x86_64 cluster | =============================================================================== =============================================================================== echo $PROJECT tm70

References

Last update: April 25, 2024

First Steps

Create an NCI user account

Join relevant NCI projects

Login to Gadi

Auto login

Change default project on Gadi

References