Allow per-campaign tests #594
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gbirke
force-pushed
the
campaign-tests
branch
from
3c700ee
to
8910e05
Compare
Banner tests in `test/banners` should now follow the `CHANNEL_NAME/CAMPAIGN_NAME` structure of the `banners` directory. The vitest configuration will exclude all banner tests that are not in the current configuration. Ticket: https://phabricator.wikimedia.org/T369276
gbirke
force-pushed
the
campaign-tests
branch
from
8910e05
to
aa6340c
Compare
Get a baseline to check if the new vitest configuration works (two kinds of tests in desktop) and to provide a structure for other tests. Renamed the wikipedia.de channels in `campaign_info.toml` to match folder names to channel names. This now must be adjusted in https://github.com/wmde/banner-screenshots/blob/main/src/CommandLine/determineCampaignFromBranchName.ts Turn missing tests into errors.
gbirke
force-pushed
the
campaign-tests
branch
from
aa6340c
to
3e28290
Compare
Use GitHub actions to run daily compatibility checks. We might need to change this workflow and [add a notification step][1]. According to the [official documentation][2], only the last committer of this workflow gets the notification emails. [1]: https://stackoverflow.com/questions/62304258/github-actions-notifications-on-workflow-failure [2]: https://docs.github.com/en/actions/monitoring-and-troubleshooting-workflows/monitoring-workflows/notifications-for-workflow-runs Ticket: https://phabricator.wikimedia.org/T369276
Enforce importing from the same folder to avoid copy-paste errors, running tests on old banners. Ticket: https://phabricator.wikimedia.org/T369276
gbirke
force-pushed
the
campaign-tests
branch
from
3e28290
to
6db472f
Compare
Abban
approved these changes
Oct 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow (and enforce) per-campaign tests instead of reusing the same test folder for every new campaign. Add CI check to check if channel name in test folder matches imports in test files, to avoid forgetting to adapt the tests.
Do a daily run of all banner tests to ensure backwards compatibility (we might need to adjust the notification settings for that in a followup PR)
When this is merged and you rebase, you'll have to adjust current PRs to use new structure, otherwise they'll fail.
Ticket: https://phabricator.wikimedia.org/T369276