-
Notifications
You must be signed in to change notification settings - Fork 107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate unit and integration tests from TravisCI to GitHub Actions #580
Merged
Merged
Changes from 7 commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
85fade9
Migrate unit and integration tests to GitHub Actions
mfisher87 3015f76
Extract checkout from re-usable action
mfisher87 0038615
Apply shell param to composite action
mfisher87 a5b2609
Run integration tests on PRs
mfisher87 2426af3
Remove erroneous comma
mfisher87 33592a5
Remove redundant word
mfisher87 0daf7ae
Clarify reason for selected Python versions in CI
mfisher87 88bc5d9
Remove earthdata username envvar
mfisher87 39006bd
Fixup PR-from-fork test
mfisher87 417238f
Filter integration test runs by path
mfisher87 a950c87
Attempt to provide correct envvars to integration test
mfisher87 7bb6cb5
Give jobs a clearer name
mfisher87 3115443
Add README badge for GitHub Actions
mfisher87 0b82f05
Merge branch 'development' into test-with-gha
mfisher87 4183ece
Adjust integration test triggers
mfisher87 5a606bc
Add separate badges for unit and integration tests
mfisher87 4e59c92
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 403643b
Apply missing badge references
mfisher87 8ee626a
Only run for reviews that are approving
mfisher87 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
name: 'Install icepyx' | ||
description: 'Install icepyx and dev dependencies' | ||
|
||
inputs: | ||
python-version: | ||
required: true | ||
|
||
runs: | ||
using: "composite" | ||
steps: | ||
- uses: "actions/setup-python@v5" | ||
with: | ||
python-version: "${{ inputs.python-version }}" | ||
|
||
- name: "Install package and test dependencies" | ||
shell: "bash" | ||
run: | | ||
python -m pip install . | ||
python -m pip install -r requirements-dev.txt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
name: "Integration test" | ||
# NOTE: We're just running the tests that require earthdata login here; we | ||
# don't distinguish between unit and integration tests yet. | ||
|
||
on: | ||
pull_request: | ||
push: | ||
branches: | ||
- "main" | ||
- "development" | ||
|
||
|
||
jobs: | ||
test: | ||
name: "Test" | ||
JessicaS11 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if: "github.repository == 'icesat2py/icepyx'" | ||
runs-on: "ubuntu-latest" | ||
|
||
steps: | ||
- uses: "actions/checkout@v4" | ||
with: | ||
fetch-depth: 0 | ||
|
||
- uses: "./.github/actions/install-icepyx" | ||
with: | ||
python-version: "3.12" | ||
|
||
- name: "Run tests" | ||
env: | ||
EARTHDATA_USERNAME: "${{ secrets.EARTHDATA_USERNAME }}" | ||
mfisher87 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
EARTHDATA_PASSWORD: "${{ secrets.EARTHDATA_PASSWORD }}" | ||
mfisher87 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
run: | | ||
pytest icepyx/ --verbose --cov app \ | ||
icepyx/tests/test_behind_NSIDC_API_login.py \ | ||
icepyx/tests/test_auth.py | ||
|
||
- name: "Upload coverage report" | ||
uses: "codecov/[email protected]" | ||
with: | ||
token: "${{ secrets.CODECOV_TOKEN }}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
name: "Unit test" | ||
# NOTE: We're just skipping the tests requiring earthdata login here; we don't | ||
# distinguish yet between unit and integration tests. | ||
|
||
on: | ||
pull_request: | ||
push: | ||
branches: | ||
- "main" | ||
- "development" | ||
|
||
|
||
jobs: | ||
test: | ||
runs-on: "ubuntu-latest" | ||
JessicaS11 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ["3.9", "3.12"] #NOTE: min and max Python versions supported by icepyx | ||
|
||
steps: | ||
- uses: "actions/checkout@v4" | ||
with: | ||
fetch-depth: 0 | ||
|
||
- uses: "./.github/actions/install-icepyx" | ||
with: | ||
python-version: "${{ matrix.python-version }}" | ||
|
||
- name: "Run tests" | ||
run: | | ||
pytest icepyx/ --verbose --cov app \ | ||
--ignore=icepyx/tests/test_behind_NSIDC_API_login.py \ | ||
--ignore=icepyx/tests/test_auth.py | ||
|
||
- name: "Upload coverage report" | ||
uses: "codecov/[email protected]" | ||
with: | ||
token: "${{ secrets.CODECOV_TOKEN }}" |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given these are slightly "expensive" tests since they download files (and running them for every commit could ultimately influence the icepyx metrics), what are our options for restricting how often they run for a given PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These triggers (
branches:
) would be for every merge, not for every commit on PRs. We can customize thepull_request:
triggers. By default, they run onopen
andsynchronize
(effectively, when the PR is pushed to, not on every commit). The full list is here: https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#pull_requestI think the best way we can limit these runs is by adding
paths:
filters. That would enable e.g. docs changes to not trigger the expensive tests!There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we consider this part of this PR, or create an issue to track the work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right - forgot this subtlety - but a PR can have a lot of pushes too!
I would definitely consider it part of this PR. I don't want to have a setup running that will trigger so many runs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't want to push back too hard, because I am fine doing it however, but I do want to raise that Travis CI behaves this way already:
https://github.com/icesat2py/icepyx/blob/development/.travis.yml#L21-L24
I would lean towards breaking this work into two PRs: (1) migrate Travis -> GH; (2) more efficient. When those PRs are merged into
development
, they will produce two clearer commits and entries in the changelog. I'm OK with immediately following this PR up with step 2 if you're OK with dividing it!There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no approved event type. That workflow checks the current state of the PR after triggering the workflow, and if it's not "approved", it sets the workflow to "skipped":
icepyx/.github/workflows/uml_action.yml
Line 10 in a52bc37
If there was an approved event type, then we could avoid the triggering in the first place :(
Since we have other possible triggers we would need something like
${{ github.event.action != "pull_request_review" || github.event.review.state == "approved" }}
. Otherwise, integration tests would be skipped on our release PRs until they were approved, and if we want that behavior we can remove thepull_request
trigger. Thoughts?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Guess that was wishful thinking on my part then.
I'm happy with the latest edits versus this. I don't think we tend to have a lot of official "reviews" until we're getting pretty close to merging, and it would be great to have the tests run on release PRs right away.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I might need to rescind this. I'm noticing the integration test is running, which makes me think that GH considers a comment (regardless of approval) to be a "review", thus triggering the action for any comment made on the PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm thinking the least unpalatable option at this point is your suggestion to remove the PR trigger and add something like
${{ github.event.action != "pull_request_review" || github.event.review.state == "approved" }}
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's try it on for size!