Skip to content

Commit

Permalink
feat(roll): roll to ToT Playwright (04-08-23) (#1115)
Browse files Browse the repository at this point in the history
  • Loading branch information
playwrightmachine authored Aug 4, 2023
1 parent 55b3ec5 commit 36418f6
Showing 1 changed file with 20 additions and 10 deletions.
30 changes: 20 additions & 10 deletions nodejs/docs/test-sharding.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,12 @@ export default defineConfig({

Blob report contains information about all the tests that were run and their results as well as all test attachments such as traces and screenshot diffs. Blob reports can be merged and converted to any other Playwright report. By default, blob report will be generated into `blob-report` directory.

To merge reports from multiple shards, put the blob report files into a single directory, for example `all-blob-reports`, and run `merge-reports` tool:
To merge reports from multiple shards, put the blob report files into a single directory, for example `all-blob-reports`. Blob report names contain shard number, so they will not clash.

Afterwards, run `npx playwright merge-reports` command:

```bash
npx playwright merge-reports ./all-blob-reports --reporter html
npx playwright merge-reports --reporter html ./all-blob-reports
```

This will produce a standard HTML report into `playwright-report` directory.
Expand All @@ -48,7 +50,14 @@ This will produce a standard HTML report into `playwright-report` directory.

One of the easiest ways to shard Playwright tests across multiple machines is by using GitHub Actions matrix strategy. For example, you can configure a job to run your tests on four machines in parallel like this:

```yaml
```yaml title=".github/workflows/playwright.yml"
name: "Playwright Tests"

on:
push:
branches:
- main

jobs:
playwright-tests:
strategy:
Expand All @@ -71,14 +80,14 @@ jobs:
if: always()
uses: actions/upload-artifact@v3
with:
name: blob-report-${{ github.run_attempt }}
name: all-blob-reports
path: blob-report
retention-days: 1
```
After all shards have completed, run a separate job that will merge the reports and produce a combined HTML report.
```yaml
```yaml title=".github/workflows/playwright.yml"
jobs:
...
merge-reports:
Expand All @@ -96,17 +105,18 @@ jobs:
- name: Download blob reports from GitHub Actions Artifacts
uses: actions/download-artifact@v3
with:
name: blob-report-${{ github.run_attempt }}
name: all-blob-reports
path: all-blob-reports

- name: Merge into HTML Report
run: npx playwright merge-reports ./all-blob-reports --reporter html
run: npx playwright merge-reports --reporter html ./all-blob-reports

- name: Upload HTML report
uses: actions/upload-artifact@v3
with:
name: html-report-${{ github.run_attempt }}
name: html-report--attempt-${{ github.run_attempt }}
path: playwright-report
retention-days: 14
```
To ensure the execution order, we make `merge-reports` job [depend](https://docs.github.com/en/actions/using-jobs/using-jobs-in-a-workflow#defining-prerequisite-jobs) on our sharded `playwright-tests` job.
Expand Down Expand Up @@ -145,12 +155,12 @@ Supported options:

Which report to produce. Can be multiple reporters separated by comma.

Example: `npx playwright merge-reports ./blob-reports --reporter=html,github`
Example: `npx playwright merge-reports --reporter=html,github ./blob-reports`
- `--config path/to/config/file`

Takes reporters from Playwright configuration file.

Example: `npx playwright merge-reports ./blob-reports --config=merge.config.ts`
Example: `npx playwright merge-reports --config=merge.config.ts ./blob-reports`

```ts title="merge.config.ts"
export default {
Expand Down

0 comments on commit 36418f6

Please sign in to comment.