Set Up your NCI Account
The steps in this section are aimed at new users who would like to do any of the following tasks:
- Run their own experiment
- Analyse model data outputs
- Browse observational data catalogues
- Evaluate model performance
- Perform other tasks involving ACCESS models or evaluation tools
Create an NCI user account
Most of the data and models you will need are available at the National Computing Infrastructure (NCI).
To access these, you need an NCI account. If you do not have one, sign up here.
Warning
You will need an institutional email address with an organisation that allows access to NCI (e.g., an Australian university, ACCESS-NRI, CSIRO, BoM, CLEX, etc.).
Once you sign up, you will be assigned a username (e.g., ab1234).
Join relevant NCI projects
NCI provides multiple services that are necessary for climate research. These include the access to supercomputing resources, data storage, and data collections management.
For technical reasons, to access either of these services you need to join a specific project.
Each project has an ID (e.g. xp65), which is what the term project actually refers to.
If you are interested in datasets and data collections, you can browse the NCI Data Catalogue and follow the NCI Data Catalogue User Guide.
To run models on Gadi instead, you need to join a project with computing resources, also called Service Units (SU). The project ID will be provided by your supervisor, research project or institution.
To join a project, search for it on NCI website and request membership.
Tip
The first project you join will become your default one. If you would like to change this, check out how to change your default project on Gadi.
There are several NCI projects that may be relevant to you, depending on the tasks you want to carry out.
For tasks supported by ACCESS-NRI (e.g., running a supported model configuration, using a supported model evaluation tool, etc.), you will find a list of relevant projects to join in the pages relative to each respective task.
Login to Gadi
Operations involving model runs and data collections take place on the Gadi supercomputer.
Before you login to Gadi, you need to possess the following prerequisites:
- Internet connection
-
Terminal with built-in SSH
Linux, MacOS and Windows 10 (or later) operative systems already have a terminal with built-in SSH.
Users of Windows 9 (or earlier) can install Windows Subsystems for Linux (WSL).Tip
Alternatively, you can login through the ARE Gadi Terminal.
However, it is recommended that you connect to Gadi from your local machine's terminal without using ARE.
To login to Gadi using SSH, on your local machine's terminal run the following command, replacing <your-NCI-username> with your NCI username (e.g., ab1234):
ssh <your-NCI-username>@gadi.nci.org.au
You will be prompted to enter your NCI password, and then you will be connected to Gadi:
Automatic login
To simplify the login and avoid being prompted every time to enter your NCI password, follow these steps:
Create an SSH key
To create an SSH key on your local machine, run:
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_gadi
Warning
For security reasons, it is recommended to enter a passphrase rather than leave it empty.
As you will see in the next step, you will not need to enter the passphrase every time you login to Gadi.
Add the SSH key to the SSH-agent
An SSH-agent is an SSH key manager that avoids you having to enter your passphrase every time you connect to a server.
To add the SSH key to the SSH-agent:
- On your local machine, start the SSH-agent by running:
eval "$(ssh-agent -s)"eval "$(ssh-agent -s)" Agent pid <agent-PID> - Add your SSH key to the SSH-agent by running:
You will be prompted to enter your SSH key passphrase, which will be stored inside the SSH-agent:ssh-add --apple-use-keychain ~/.ssh/id_gadissh-add --apple-use-keychain ~/.ssh/id_gadi Enter passphrase for <$HOME>/.ssh/id_gadi: Identity added: <$HOME>/.ssh/id_gadi <$USER@hostname> If you are using a MacOS version prior to Monterey (12.0), substitute the--apple-use-keychainflag with-K.
You will be prompted to enter your SSH key passphrase, which will be stored inside the SSH-agent:ssh-add ~/.ssh/id_gadissh-add ~/.ssh/id_gadi Enter passphrase for <$HOME>/.ssh/id_gadi: Identity added: <$HOME>/.ssh/id_gadi <$USER@hostname>
Create/Update the SSH config file
The ~/.ssh/config is a file where you can store labelled SSH configurations for different remote servers you regularly connect to, so you do not have to remember them all.
To create an SSH config file, run the following command on your local machine:
touch ~/.ssh/config
Tip
If you already have an existing ~/.ssh/config file, the above command will not have any effect.
The following lines should be added to your ~/.ssh/config to describe the SSH configuration for Gadi (replace <your-NCI-username> with your NCI username, e.g., ab1234):
Host gadi
Hostname gadi.nci.org.au
User <your-NCI-username>
ForwardX11 true
ForwardX11Trusted yes
IdentityFile ~/.ssh/id_gadi
AddKeysToAgent yes
UseKeychain yes
Host gadi
Hostname gadi.nci.org.au
User <your-NCI-username>
ForwardX11 true
ForwardX11Trusted yes
IdentityFile ~/.ssh/id_gadi
AddKeysToAgent yes
Warning
If you already have an existing ~/.ssh/config file which contains configurations for any Host (e.g., by using Host *), make sure you delete any of the keywords present in that SSH configuration from the Gadi configuration above.
Add the SSH key to the authorised keys
To enable automatic connection to a server, that server needs to recognise the SSH key as authorised. The list of authorised keys for a certain server is stored in the file ~/.ssh/authorized_keys.
To add the newly created SSH key as an authorised key for Gadi, run the following command from your local machine:
ssh gadi "mkdir -p .ssh && cat >> .ssh/authorized_keys <<< '$(cat ~/.ssh/id_gadi.pub)'"
You will be prompted to enter your NCI password. If you did all of the above steps correctly, this should be the last time you need to do so.
Once you complete all the above steps, you should be able to connect to Gadi from your local machine's terminal simply by running:
ssh gadi
Change default project on Gadi
It is recommended that you check what your default project on Gadi is set to. The default project should be set to the computational project you will most likely use to run simulations/evaluations and store data.
You can check which is your default project by logging into Gadi and running:
echo $PROJECT
To change your default project on Gadi, you need to manually change the PROJECT field in the ~/.config/gadi-login.conf file.
Alternatively, you can run the following command on Gadi's terminal (by substituting <new-default-project> with the project you want to set as default):
sed -i "s/\(PROJECT \)\w*/\1<new-default-project>/" ~/.config/gadi-login.conf
Once this is done, exit from Gadi and log back in.
For example, if you want to change your default project to tm70 on Gadi: