Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add first iteration of tests to repo #42

Merged
merged 50 commits into from
Dec 21, 2023
Merged

Add first iteration of tests to repo #42

merged 50 commits into from
Dec 21, 2023

Conversation

robbibt
Copy link
Member

@robbibt robbibt commented Dec 20, 2023

This adds some very rudimentary tests to the repo!
image

Currently they use a small pickle file of satellite data to test against the elevation function, and downloads a zip of tide modelling data. In the future we can improve it by:

  • Replacing the pickle with prod database access
  • Syncing tide models directly from S3

Ignore the long list of commits - we can merge this with squash and merge. The ECR sync step will fail on this branch, but hopefully work on main.

@robbibt robbibt requested a review from vnewey December 20, 2023 06:46
@codecov-commenter
Copy link

Welcome to Codecov 🎉

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered ☂️

pytz==2023.3
rioxarray
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to add back rioxarray - xarray used it under the hood after all

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pickle file of satellite data - to replace with database access later on

max_cloudcover=90,
skip_broken_datasets=True,
)
if study_area == "testing":
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use pickle file instead of loading data from datacube if study area == "testing"

codecov.yaml Outdated
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore the validation and composites modules in code coverage for now


- name: Build DEA Intertidal image
timeout-minutes: 20
shell: bash
run: |
docker-compose build

- name: Run tests
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This downloads tide data from Dropbox (to replace with S3 access to tide modelling files), then runs some tests using docker compose run. Then finally upload code coverage to codecov

@@ -30,13 +33,33 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
Copy link
Member Author

@robbibt robbibt Dec 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe makes the git clone faster?

@@ -1,4 +1,4 @@
name: DEA Intertidal Image Push
name: Image build and test
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed so it shows up nicely on the readme badges

@@ -77,7 +77,7 @@ def extents(
freq,
dem,
corr,
land_use_mask="/gdata1/data/land_use/ABARES_CLUM/geotiff_clum_50m1220m/clum_50m1220m.tif",
land_use_mask="https://dea-public-data-dev.s3-ap-southeast-2.amazonaws.com/abares_clum_2020/clum_50m1220m.tiff",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Load data from S3 so it works in the tests

@@ -14,8 +14,6 @@
# from pyproj import Transformer
# from scipy.signal import argrelmax, argrelmin

from scipy.interpolate import interp1d
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not used below; removing for now

Copy link
Collaborator

@vnewey vnewey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great Robbi!

@vnewey vnewey merged commit b29dd4b into main Dec 21, 2023
1 of 2 checks passed
@robbibt robbibt deleted the tests branch December 21, 2023 02:36
@robbibt robbibt added the v1.0.0 label Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants