Danger
This page is tailored to experienced users and collaborators developing ACCESS models.
This step is not required if you only want to run a model. If you are looking for information on how to run a model, refer to the Run a Model section.
Creating Prereleases and Releases for an ACCESS Model
About
The instructions below outline how to trigger automatic prerelease and release builds of an ACCESS model by making use of the build and deployment workflows enabled by the access-nri/build-cd repository.
This allows, for example, quick testing of the development of a new feature in a specific ACCESS Model configuration.
The following instructions outline how to trigger a prerelease build of ACCESS-OM2 after modifying its MOM5 component. All other components of the official ACCESS-OM2 release will remain unchanged.
Tip
The following instructions are valid (with simple tweaks) for all ACCESS models.
Prerequisites
-
Join the required projects specific to the HPC system
For models deployed toGadi, join thevk83project by requesting membership on the respective vk83 NCI project page.For more information on joining specific NCI projects, refer to How to connect to a project.
-
Write permissions to the related model deployment repository
To request write permissions for a specific model deployment repository, please contact ACCESS-NRI.
The model deployment repository
All ACCESS models have their deployments on HPC systems controlled by an ACCESS-NRI's GitHub repository, named according to the specific model.
For example, the deployment of ACCESS-OM2 is controlled by the ACCESS-OM2 GitHub repository.
The model deployment repository also enables automatic prerelease build deployments of the model whenever commits are added to one of the repository's open Pull Requests (PR).
Every commit within the same PR triggers an isolated prerelease build deployment for the same PR (every PR can be seen as a separate development configuration for the model).
When a PR is merged, the related prereleases are deleted and an official release build is created.
Although the structure of different model deployment repositories can slightly differ, their top-level will always contain a:
The spack.yaml file
The spack.yaml file defines the Spack environment that constrains the versions and features (variants) of all dependencies required to build a given ACCESS model. At build time, it is concretized into a single set of dependencies, creating a spack.lock file.
For more information on the spack.yaml file, refer to ACCESS-NRI's DevDocs.
The config directory
The config directory contains a single versions.json file. This file allows customisation of both the version of access-nri/spack used to deploy the model and the version of access-nri/spack-packages that will source the recipes for the Spack packages.
Trigger model prerelease and release build deployments
As mentioned above, prerelease build deployments are triggered by commits within a model deployment repository's open PR. When the PR is merged, a release build deployment is created.
The following sequence of tasks are needed to trigger a prerelease and release build deployment for a model new feature:
1. Clone the repo and create a feature branch
The first step is to clone the model deployment repository and create a feature branch from its main branch, on which new features will be developed.
To clone the ACCESS-OM2 deployment repository and create a feature branch named update_mom5_dev_build run:
git clone https://github.com/ACCESS-NRI/ACCESS-OM2.git
cd ACCESS-OM2
git checkout -b update_mom5_dev_build
2. Apply the modifications and commit the changes
Modifications can then be made to the spack.yaml or config/versions.json files as needed (e.g., update packages versions, add or change variants, etc.).
After making the modifications, commit the changes to the newly-created feature branch and push the changes upstream.
In this example, we will change ACCESS-OM2's MOM5 component by replacing it with the version from MOM5 development branch.
To achieve this, the following modifications will be made:
- Update the version of the
mom5package in thespack.yamlfile with the new version (i.e.,@git.development). -
Update the associated module projection to
{name}/development-{hash:7}.Tip
The
{hash:7}part is used so the module doesn't conflict with other versions. -
It is also recommended to update the overall ACCESS-OM2 version along with its associated module projection.
This is particularly important before merging a PR as it will determine the version tag for the model new release. The format isCALVER_YEAR.CALVER_MONTH.MINOR.
In this example, the overall version will be updated togit.2024.03.1.
After completing the modifications above, the output of the command
git diff
specs listTo stage and commit the changes, run:
git commit -am "Updated mom5 to development version."
Finally, push the changes upstream by executing:
git push --set-upstream origin update_mom5_dev_build
3. Open a PR
To trigger a prerelease build, open a PR with the main branch of the model deployment repository as the base. This will start a model build attempt by GitHub's CI/CD infrastructure. As a result, a comment is added by github-actions[bot] saying that the build is deploying.
Once the deployment is successful, you will be able to access the prerelease on the listed HPC systems by following the instructions specified in the comment's Details and usage instructions.
There are three main statuses for a deployment that can be identified by looking at the GitHub Environment diaglog box in the related PR:
-
Successful
A successful deployment is identified by a green (Active deployment) or white (Inactive deployment) icon, with the Active one being the most recent deployment within the repository.
If a deployment is successful, it can be accessed on HPC systems.
-
In Progress
The deployment is still ongoing. It will soon become successful or fail.
-
Failed
The deployment failed and the CI/CD log can be viewed by clicking on Show environments in the GitHub Environment diaglog box.
If we open a PR to the ACCESS-OM2 deployment repository with our update_mom5_dev_build branch as the base, we will get a comment. Once deployed, the prerelease build can be accessed through the module access-om2/pr94-1:
Tip
If further changes are required within the same PR, additional commits can be pushed onto the feature branch to create more prerelease builds. This does not remove earlier prerelease builds, allowing concurrent testing of multiple different builds.
For example, adding an additional commit to the update_mom5_dev_build branch (while its PR is open) will trigger another prerelease build and allow the use of the module access-om2/pr94-2, once deployed.
4. Merge or close the PR
If the PR gets merged, an official ACCESS-NRI release build of the given model will be created, along with an associated GitHub Release.
Warning
When a PR is closed (this includes merged PRs), regardless if it was merged or not, all the related prereleases deployments get removed.
You can retrieve a single prerelease build by creating a Spack environment using the spack.yaml/spack.lock artifact related to the specific commit.
Comment Commands
Another way to trigger the CI/CD pipeline is by commenting on a model deployment repository's open PR with a specific Comment Command.
This will trigger the CI/CD pipeline and perform certain actions, depending on the specific Comment Command.
Info
All Comment Commands must be at the beginning of the comment, without any leading spaces.
The following Comment Commands are available in all model deployment repositories:
!bump
Info
Requires the commenter to have write permissions within the repository.
Usage
!bump [major|minor]
Description
Convenience function that automatically bumps the overall model version in the spack.yaml file and commits the result to the PR branch.
The overall model version is formatted as YEAR.MONTH.MINOR.
!bump major bumps the model version to the next major version, formatted as YEAR.MONTH.0, where YEAR and MONTH correspond to the current year and month when the comment is issued.
!bump minor bumps the model version to the next minor version, formatted as YEAR.MONTH.(MINOR+1). Here YEAR, MONTH and MINOR are the same as the previous version.
Example
For example, if in Jan 2025 we commented with !bump major on an open PR that has its overall model version set to git.2024.06.2, the overall model version would be bumped to git.2025.01.0.
If, however, we commented the PR with !bump minor, the overall model version would be bumped to git.2024.06.3 regardless of the comment date.
!redeploy
Info
Requires the commenter to have write permissions within the repository.
Usage
!redeploy
Description
Convenience function that triggers a new independent deployment of the HEAD of the model deployment repository PR branch.
This is used to trigger a new deployment when changes are made to a model dependency, but no changes have occurred in the HEAD of the PR branch.
Example
For example, we make the above modifications that updated the ACCESS-OM2 MOM5 component version to the one from MOM5 repository's development branch (by changing the mom5 require version to git.development in the spack.yaml).
We then push the commits to the update_mom5_dev_build branch and open a PR to main, resulting in a successful deployment of our ACCESS-OM2 mom5-development build (version 1).
Subsequently, we decide to make some further changes to the MOM5 development branch and we push the commits upstream.
Now, if we want to test these new MOM5 changes, we would have to redeploy the latest commit in the update_mom5_dev_build branch to reflect the updates to the MOM5 development branch. Since the HEAD of the update_mom5_dev_build did not change, instead of having to create a new "redundant" commit only to trigger the prerelease deployment, we can comment !redeploy in the update_mom5_dev_build PR. This will force the CI/CD pipeline to redeploy the model build with the latest modifications in the MOM5 development branch.
As a result, a new separate deployment of the ACCESS-OM2 mom5-development build (version 2) is produced.
Warning
Before using a !redeploy Comment Command, the user should monitor the state of each dependency's git branch listed in the spack.yaml file. The CI/CD pipeline does not perform any checks and will simply redeploy the HEAD of the model deployment repository's PR branch using the HEAD of all corresponding dependency git branches specified in the spack.yaml file.
Backporting Bugfixes
The main branch of a model deployment repository reflects the most up-to-date changes to the model. Hence, it is always at the same level or ahead of the model's latest major release.
However, it is also essential to provide a way to apply bug fixes to earlier major versions of the model. This is where dedicated backport/YEAR.MONTH branches come into play, allowing bug fixes to be incorporated into previous versions of the model.
For example, let's say there is a bug fix that needs to be incorporated to the ACCESS-OM2 2023.11.23 release.
To do so, a developer should:
- Create a
backport/2023.11branch (if it doesn't already exist) from the2023.11.23tag:git checkout -b backport/2023.11 2023.11.23git checkout -b backport/2023.11 2023.11.23 Switched to a new branch 'backport/2023.11' - Apply the bug fixes, commit and push the changes:
git commit -am "Bug fix." git push --set-upstream origin backport/2023.11git commit -am "Bug fix." [backport/2023.11 cac60f0] Bug fix. 1 file changed, 2 insertions(+), 1 deletions(-) git push --set-upstream origin backport/2023.11 Enumerating objects: 5, done. Counting objects: 100% (5/5), done. Delta compression using up to 8 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 565 bytes | 565.00 KiB/s, done. Total 3 (delta 2), reused 0 (delta 0), pack-reused 0 remote: Resolving deltas: 100% (2/2), completed with 2 local objects. remote: remote: Create a pull request for 'backport/2023.11' on GitHub by visiting: remote: https://github.com/ACCESS-NRI/ACCESS-OM2/pull/new/backport/2023.11 remote: To https://github.com/ACCESS-NRI/ACCESS-OM2.git * [new branch] backport/2023.11 -> backport/2023.11 branch 'backport/2023.11' set up to track 'origin/backport/2023.11'. - Open a PR from the
backport/2023.11branch with the fixes.
When the PR is merged, a release tagged as2023.11.24will be created.