Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature: perfect restart and n-cores CI #778

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Feature: perfect restart and n-cores CI #778

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1bd2f0a
Adding compare-output github action because the comparison code will …
scrasmussen Oct 23, 2024
0d8b67f
Add n-cores and perfect restart tests
scrasmussen Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions .github/actions/compare-output/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
name: "Compare Output"
description: "Performs a series of comparison steps"
runs:
using: "composite"
steps:
- name: Compare HYDRO_RST.* output with xrcmp
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
for file in output_${{ matrix.configuration }}/HYDRO_RST.*; do\
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \
done

- name: Compare RESTART.* output with xrcmp
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
for file in output_${{ matrix.configuration }}/RESTART.*; do\
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \
done

- name: List file
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
ls -t output_${{ matrix.configuration }}/*.CHANOBS_DOMAIN1 | head -n 1)

- name: Head of list of files
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.CHANOBS_DOMAIN1 | head -n 1)
echo "file = ${file}"

- name: Compare last *.CHANOBS_DOMAIN1 output with xrcmp
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.CHANOBS_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1

- name: Compare last *.CHRTOUT_DOMAIN1 output with xrcmp
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.CHRTOUT_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1;

- name: Compare last *.LSMOUT_DOMAIN1 output with xrcmp
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.LSMOUT_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1;

- name: Compare last *.RTOUT_DOMAIN1 output with xrcmp
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.RTOUT_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1;

- name: Compare output with compare_output
if: ${{ always() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
rm -f output_diff
mkdir output_diff
python -c \
"import sys; \
sys.path.append('${GITHUB_WORKSPACE}/candidate/tests/utils'); \
import compare_output; \
from pathlib import Path; \
compare_output.plot_diffs('${GITHUB_WORKSPACE}/candidate/build/Run/output_diff', \
'${GITHUB_WORKSPACE}/candidate/build/Run/output_${{ matrix.configuration }}/', \
'${GITHUB_WORKSPACE}/reference/build/Run/output_${{ matrix.configuration }}/', \
'${{ matrix.configuration }}')"

- name: Copy test results from container
if: ${{ always() }}
shell: bash
run: |
mkdir -p $GITHUB_WORKSPACE/test_report
cp -r $GITHUB_WORKSPACE/candidate/build/Run/output_diff/diff_plots/* $GITHUB_WORKSPACE/test_report/

- name: Attach diff plots to PR
if: ${{ failure() }}
shell: bash
run: |
cd $GITHUB_WORKSPACE/candidate/tests/local/utils
bash attach_all_plots.bash $(jq --raw-output .pull_request.number "$GITHUB_EVENT_PATH") ${{ matrix.configuration }}

- name: Archive test results to GitHub
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-reports
path: |
${{ github.workspace }}/test_report/*

- name: Exit if any tests fail
if: ${{ failure() }}
shell: bash
run: exit 1
115 changes: 20 additions & 95 deletions .github/workflows/test-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,105 +113,30 @@ jobs:
cd $GITHUB_WORKSPACE/candidate/build/Run
make run-croton-${{ matrix.configuration }}

- name: Compare HYDRO_RST.* output with xrcmp
if: ${{ always() }}
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
for file in output_${{ matrix.configuration }}/HYDRO_RST.*; do\
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \
done

- name: Compare RESTART.* output with xrcmp
if: ${{ always() }}
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
for file in output_${{ matrix.configuration }}/RESTART.*; do\
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \
done

- name: Compare last *.CHANOBS_DOMAIN1 output with xrcmp
if: ${{ always() }}
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.CHANOBS_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \

- name: Compare last *.CHRTOUT_DOMAIN1 output with xrcmp
if: ${{ always() }}
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.CHRTOUT_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \

- name: Compare last *.LSMOUT_DOMAIN1 output with xrcmp
if: ${{ always() }}
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.LSMOUT_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \

- name: Compare last *.RTOUT_DOMAIN1 output with xrcmp
if: ${{ always() }}
- name: Compare output of serial runs
uses: ./.github/actions/compare-output

- name: Clean candidate model output
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
file=$(ls -t output_${{ matrix.configuration }}/*.RTOUT_DOMAIN1 | head -n 1)
python ${GITHUB_WORKSPACE}/candidate/tests/utils/xrcmp.py \
--candidate $file \
--reference $GITHUB_WORKSPACE/reference/build/Run/$file \
--log_file $file_diff.txt \
--n_cores 1; \

- name: Compare output with compare_output
if: ${{ always() }}
make clean

- name: Run parallel candidate model
run: |
cd $GITHUB_WORKSPACE/candidate/build/Run
mkdir output_diff
python -c \
"import sys; \
sys.path.append('${GITHUB_WORKSPACE}/candidate/tests/utils'); \
import compare_output; \
from pathlib import Path; \
compare_output.plot_diffs('${GITHUB_WORKSPACE}/candidate/build/Run/output_diff', \
'${GITHUB_WORKSPACE}/candidate/build/Run/output_${{ matrix.configuration }}/', \
'${GITHUB_WORKSPACE}/reference/build/Run/output_${{ matrix.configuration }}/', \
'${{ matrix.configuration }}')"

- name: Copy test results from container
if: ${{ always() }}
run: |
mkdir -p $GITHUB_WORKSPACE/test_report
cp -r $GITHUB_WORKSPACE/candidate/build/Run/output_diff/diff_plots/* $GITHUB_WORKSPACE/test_report/
make run-croton-${{ matrix.configuration }}-parallel

- name: Compare output of n-cores candidate model
uses: ./.github/actions/compare-output

- name: Attach diff plots to PR
if: ${{ failure() }}
# Testing perfect restart, not cleaning candidate model output
- name: Setup and run candidate model perfect restart startup
run: |
cd $GITHUB_WORKSPACE/candidate/tests/local/utils
bash attach_all_plots.bash $(jq --raw-output .pull_request.number "$GITHUB_EVENT_PATH") ${{ matrix.configuration }}
cd $GITHUB_WORKSPACE/candidate/build/Run
sed -i 's|RESTART_FILENAME_REQUESTED = "RESTART/RESTART.2011082600_DOMAIN1"|RESTART_FILENAME_REQUESTED = "./RESTART.2011090100_DOMAIN1"|' namelist.hrldas
sed -i 's/KDAY = 7/KDAY = 1/' namelist.hrldas
rm RESTART.2011090200_DOMAIN1
make run-croton-${{ matrix.configuration }}-parallel

- name: Archive test results to GitHub
if: ${{ failure() }}
uses: actions/upload-artifact@v4
with:
name: test-reports
path: |
${{ github.workspace }}/test_report/*
- name: Compare output of perfect restart candidate model
uses: ./.github/actions/compare-output
Loading