Skip to content

Commit

Permalink
Rename npm test-snapshots to npm test-qiskit (#473)
Browse files Browse the repository at this point in the history
Prework for adding visual regression testing to the ecosystem theme in
#470.

`npm run test-snapshots` is only run locally. I know that I'll want to
be able to easily run only the Qiskit theme or only the Ecosystem theme,
since the snapshots only are for what is unique to each theme. Snapshot
tests are slow, so I want to run the minimum amount of tests. We'll have
`npm run test-ecosystem` in a followup.

This also makes `npm run test-qiskit` more useful by first running `tox
-e qiskit`. That reduces the risk of forgetting to rebuild the docs
beforehand.
  • Loading branch information
Eric-Arellano authored Jul 7, 2023
1 parent aba939a commit 1bf4084
Show file tree
Hide file tree
Showing 28 changed files with 10 additions and 11 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ jobs:
name: html_docs
path: artifacts

- name: Run JavaScript and snapshot tests
run: npm run _run-snapshot-tests
- name: Run JavaScript snapshot tests
run: npm run _run-playwright
- name: Upload snapshot results
if: failure()
uses: actions/upload-artifact@v3
Expand Down
7 changes: 3 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,17 +85,16 @@ First, you need to install:
Then, to run the tests locally:

1. `npm install`
2. Build the docs, `tox -e qiskit`
3. `npm run test-snapshots`
2. `npm run test-qiskit`

You must rebuild the docs with `tox -e qiskit` whenever you make changes to the theme or docs folder. The docs will not automatically rebuild.
The docs will rebuild every time you run `npm run test-qiskit`.

### How to update the expected snapshot for intentional changes

First, get the `snapshot_results` folder, either by downloading it from CI or by running the tests locally. Then:

1. Find the "actual" snapshot for the failing test, such as `footer-includes-page-analytics-1-actual.png`.
2. Copy that snapshot into the folder `tests/js/snapshots.test.js-snapshots`. Rename the `-actual.png` file ending to be `-linux.png` and overwrite the prior file.
2. Copy that snapshot into the folder `tests/js/qiskit.test.js-snapshots`. Rename the `-actual.png` file ending to be `-linux.png` and overwrite the prior file.

------
## Updating bundled web components
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ html_context = {

By enabling analytics we will be able to collect information on number of visits to each documentation page. It will also trigger the addition of a `Was this page helpful?` component at the bottom of each documentation page, so users will be able to provide yes/no feedback for each page.

![](tests/js/snapshots.test.js-snapshots/footer-includes-page-analytics-1-linux.png)
![](tests/js/qiskit.test.js-snapshots/footer-includes-page-analytics-1-linux.png)

If you do not enable analytics, no data will be collected and the `Was this page helpful?` component will not show.

Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
"scripts": {
"build": "webpack",
"start": "http-server example_docs/docs/_qiskit_build",
"test-snapshots": "npm run _docker-build && npm run _docker-run",
"test-qiskit": "tox -e qiskit && npm run _docker-build && npm run _docker-run",
"_docker-build": "docker build -t qiskit_sphinx_theme -f tests/js/Dockerfile .",
"_docker-run": "docker run --rm --init -v $(pwd)/snapshot_results:/snapshot_results -v $(pwd)/tests/js/snapshots.test.js-snapshots:/tests/js/snapshots.test.js-snapshots qiskit_sphinx_theme",
"_run-snapshot-tests": "playwright test"
"_docker-run": "docker run --rm --init -v $(pwd)/snapshot_results:/snapshot_results -v $(pwd)/tests/js/qiskit.test.js-snapshots:/tests/js/qiskit.test.js-snapshots qiskit_sphinx_theme",
"_run-playwright": "playwright test"
}
}
2 changes: 1 addition & 1 deletion tests/js/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,4 @@ COPY playwright.config.js ./
COPY tests/js tests/js
COPY example_docs/docs/_qiskit_build example_docs/docs/_qiskit_build

CMD [ "npm", "run", "_run-snapshot-tests" ]
CMD [ "npm", "run", "_run-playwright" ]
File renamed without changes.

0 comments on commit 1bf4084

Please sign in to comment.