Danger

This page is tailored to experienced users and collaborators developing ACCESS models.
This step is not required if you only want to run a model. If you are looking for information on how to run a model, refer to the Run a Model section.

Set up Spack for building ACCESS models

Spack is a build-from-source package manager, specifically designed to simplify the installation of scientific software on supercomputers.

To use Spack, please familiarise yourself with the Basic Usage instructions and Environments.

We also recommend that you refer to the Spack 101 Tutorial.

Prerequisites

These instructions are tailored specifically for Gadi. To use Spack on Gadi, you must have an NCI account. For instructions on how to set up an account, refer to Set Up your NCI Account.

Set up Spack on Gadi

Tip

The steps in this section only need to be done once.

Create a directory for Spack

Create a directory on the filesystem where Spack will be installed (e.g. /g/data/$PROJECT/$USER/spack/0.22). Use the /g/data filesystem if you wish to run the binaries on the compute nodes.

mkdir -p /g/data/$PROJECT/$USER/spack/0.22
cd /g/data/$PROJECT/$USER/spack/0.22

Clone the relevant git repositories

Info

ACCESS-NRI maintains a fork of Spack to enable back-porting fixes from more recent spack versions. This fork is the one used in these instructions.

git clone -c feature.manyFiles=true https://github.com/ACCESS-NRI/spack.git --branch releases/v0.22
git clone https://github.com/ACCESS-NRI/spack-packages.git --branch main
git clone https://github.com/ACCESS-NRI/spack-config.git --branch main

Link Spack configuration files to the Spack instance

ln -s -r -v spack-config/v0.22/gadi/* spack/etc/spack/

Success

Your Spack setup is complete!

For instructions on how to build an ACCESS model using Spack, refer to Modify an ACCESS model's source code.

Enable Spack

Warning

This step needs to be carried out for any new login or new shell environment.

cd /g/data/$PROJECT/$USER/spack/0.22
module purge
. spack-config/spack-enable.bash

Warning

There is a space between the . and the path to the file, as we are sourcing the file.

Test Spack (OPTIONAL)

To test that your Spack installation works as expected, we will create an ACCESS-TEST environment and build the relevant packages (this will take approximately 30 minutes). Then, we will uninstall all the packages and remove the environment.

Create a Spack managed environment

git clone https://github.com/ACCESS-NRI/ACCESS-TEST.git
spack env create test ACCESS-TEST/spack.yaml

git clone https://github.com/ACCESS-NRI/ACCESS-TEST.git Cloning into 'ACCESS-TEST'... remote: Enumerating objects: 33, done. remote: Counting objects: 100% (33/33), done. remote: Compressing objects: 100% (20/20), done. remote: Total 33 (delta 12), reused 27 (delta 9), pack-reused 0 (from 0) Receiving objects: 100% (33/33), 15.92 KiB | 1.06 MiB/s, done. Resolving deltas: 100% (12/12), done. spack env create test ACCESS-TEST/spack.yaml ==> Created environment test in: /g/data/$PROJECT/$USER/spack/0.22/environments/test ==> Activate with: spack env -p activate test

Activate the environment

Activate the test Spack environment by running:

spack env activate -p test

spack env activate -p test

Compile packages

spack find
spack concretize -f --fresh
spack install

spack find ==> In environment test ==> 1 root specs - access-test@git.2024.09.20 ==> 0 installed packages spack concretize -f --fresh ==> Concretized access-test@git.2024.09.20 - hmy75yl access-test@git.2024.09.20%intel@2021.10.0~deterministic build_system=bundle arch=linux-rocky8-x86_64 [e] 5elnsoi ^glibc@2.28%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64 - kok5n7h ^oasis3-mct@git.2023.11.09=2023.11.09%intel@2021.10.0~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64 - doeoclg ^gmake@4.4.1%intel@2021.10.0~guile build_system=generic arch=linux-rocky8-x86_64 - ntfunrm ^netcdf-fortran@4.6.1%intel@2021.10.0~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64 - vob7om3 ^netcdf-c@4.9.2%intel@2021.10.0+blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=autotools patches=0161eb8 arch=linux-rocky8-x86_64 - 357vng5 ^bzip2@1.0.8%intel@2021.10.0~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64 - y7n7vkn ^diffutils@3.10%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64 - 55ipnye ^libiconv@1.17%intel@2021.10.0 build_system=autotools libs=shared,static arch=linux-rocky8-x86_64 - zrjfo56 ^c-blosc@1.21.5%intel@2021.10.0+avx2~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64 [e] rldyvqn ^cmake@3.24.2%intel@2021.10.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64 - 7a5olrr ^lz4@1.9.4%intel@2021.10.0+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64 - wk4pvru ^snappy@1.1.10%intel@2021.10.0~ipo+pic+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64 - aumsrgz ^hdf5@1.14.3%intel@2021.10.0~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64 - vrupasu ^pkgconf@2.2.0%intel@2021.10.0 build_system=autotools arch=linux-rocky8-x86_64 - x4hw5jq ^libaec@1.0.6%intel@2021.10.0~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64 - wa2x7rh ^zlib-ng@2.1.6%intel@2021.10.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64 - kzc7pcv ^zstd@1.5.6%intel@2021.10.0~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64 [e] uvea7q2 ^openmpi@4.1.5%intel@2021.10.0 cppflags='-diag-disable=10441' ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64 spack install ==> Installing glibc-2.28-mqjolvbeskcnhz5chvtdshk4x4sfnycs [1/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/glibc-2.28-mqjolvbeskcnhz5chvtdshk4x4sfnycs ==> Installing cmake-3.24.2-vc4y4c64s55j5u6kp37ciw2hcghuxhhc [2/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/cmake-3.24.2-vc4y4c64s55j5u6kp37ciw2hcghuxhhc ==> Installing openmpi-4.0.2-ikhujrkyukytbkxxyk3mub44v63vuzfz [3/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/openmpi-4.0.2-ikhujrkyukytbkxxyk3mub44v63vuzfz ==> Installing gmake-4.4.1-j6yscmmcn3qws7n35klote7rivw7foa6 [4/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-j6yscmmcn3qws7n35klote7rivw7foa6 ==> Installing libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq [5/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq ==> Installing zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6 [6/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6 ==> Installing snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm [7/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm ==> Installing pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa [8/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa ==> Installing zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4 [9/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4 ==> Installing lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4 [10/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4 ==> Installing libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx [11/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx ==> Installing hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7 [12/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7 ==> Installing c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc [13/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc ==> Installing diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l [14/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l ==> Installing bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt [15/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt ==> Installing netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb [16/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb ==> Installing netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy [17/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy ==> Installing oasis3-mct-git.2023.11.09=2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p [18/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/oasis3-mct-git.2023.11.09=2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p ==> Installing access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc [19/19] [+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc ==> Updating view at /g/data/$PROJECT/$USER/spack/0.22/environments/test/.spack-env/view

Info

The full output has been truncated for brevity.

Check installed packages

spack find

spack find

==> In environment test

==> 1 root specs

[+] access-test@git.2024.09.20

==> installed packages

-- linux-rocky8-x86_64 / intel@2021.10.0 ------------------------

access-test@git.2024.09.20 diffutils@3.10 libaec@1.0.6 netcdf-fortran@4.6.1 snappy@1.1.10 bzip2@1.0.8 glibc@2.28 libiconv@1.17 oasis3-mct@git.2023.11.09=2023.11.09 zlib-ng@2.1.6 c-blosc@1.21.5 gmake@4.4.1 lz4@1.9.4 openmpi@4.1.5 zstd@1.5.6 cmake@3.24.2 hdf5@1.14.3 netcdf-c@4.9.2 pkgconf@2.2.0

==> 19 installed packages

Cleanup

spack uninstall --remove --all
spack env deactivate
spack env rm test
rm -rf ACCESS-TEST

Update Spack on Gadi

Keep your Spack instance up-to-date by doing the following:

cd /g/data/$PROJECT/$USER/spack/0.22
git -C spack fetch --all -Pp
git -C spack reset --hard origin/releases/v0.22
git -C spack-config pull
git -C spack-packages pull

- https://spack.readthedocs.io/en/latest/

Last update: November 6, 2024