Skip to content

Commit

Permalink
feat(roll): roll to ToT Playwright (09-10-24)
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] committed Oct 9, 2024
1 parent ba51b71 commit eadbdfc
Show file tree
Hide file tree
Showing 3 changed files with 35 additions and 18 deletions.
20 changes: 10 additions & 10 deletions nodejs/docs/ci.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ jobs:
name: 'Playwright Tests'
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.47.2-noble
image: mcr.microsoft.com/playwright:v1.48.0-noble
options: --user 1001
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -340,7 +340,7 @@ trigger:
pool:
vmImage: ubuntu-latest
container: mcr.microsoft.com/playwright:v1.47.2-noble
container: mcr.microsoft.com/playwright:v1.48.0-noble
steps:
- task: NodeTool@0
Expand All @@ -364,7 +364,7 @@ Running Playwright on CircleCI is very similar to running on GitHub Actions. In
executors:
pw-noble-development:
docker:
- image: mcr.microsoft.com/playwright:v1.47.2-noble
- image: mcr.microsoft.com/playwright:v1.48.0-noble
```
Note: When using the docker agent definition, you are specifying the resource class of where playwright runs to the 'medium' tier [here](https://circleci.com/docs/configuration-reference?#docker-execution-environment). The default behavior of Playwright is to set the number of workers to the detected core count (2 in the case of the medium tier). Overriding the number of workers to greater than this number will cause unnecessary timeouts and failures.
Expand All @@ -387,7 +387,7 @@ Jenkins supports Docker agents for pipelines. Use the [Playwright Docker image](
```groovy
pipeline {
agent { docker { image 'mcr.microsoft.com/playwright:v1.47.2-noble' } }
agent { docker { image 'mcr.microsoft.com/playwright:v1.48.0-noble' } }
stages {
stage('e2e-tests') {
steps {
Expand All @@ -404,7 +404,7 @@ pipeline {
Bitbucket Pipelines can use public [Docker images as build environments](https://confluence.atlassian.com/bitbucket/use-docker-images-as-build-environments-792298897.html). To run Playwright tests on Bitbucket, use our public Docker image ([see Dockerfile](./docker.mdx)).
```yml
image: mcr.microsoft.com/playwright:v1.47.2-noble
image: mcr.microsoft.com/playwright:v1.48.0-noble
```
### GitLab CI
Expand All @@ -417,7 +417,7 @@ stages:
tests:
stage: test
image: mcr.microsoft.com/playwright:v1.47.2-noble
image: mcr.microsoft.com/playwright:v1.48.0-noble
script:
...
```
Expand All @@ -432,7 +432,7 @@ stages:
tests:
stage: test
image: mcr.microsoft.com/playwright:v1.47.2-noble
image: mcr.microsoft.com/playwright:v1.48.0-noble
parallel: 7
script:
- npm ci
Expand All @@ -447,7 +447,7 @@ stages:
tests:
stage: test
image: mcr.microsoft.com/playwright:v1.47.2-noble
image: mcr.microsoft.com/playwright:v1.48.0-noble
parallel:
matrix:
- PROJECT: ['chromium', 'webkit']
Expand All @@ -463,7 +463,7 @@ To run Playwright tests on Google Cloud Build, use our public Docker image ([see
```yml
steps:
- name: mcr.microsoft.com/playwright:v1.47.2-noble
- name: mcr.microsoft.com/playwright:v1.48.0-noble
script:
...
env:
Expand All @@ -481,7 +481,7 @@ type: docker
steps:
- name: test
image: mcr.microsoft.com/playwright:v1.47.2-noble
image: mcr.microsoft.com/playwright:v1.48.0-noble
commands:
- npx playwright test
```
Expand Down
16 changes: 8 additions & 8 deletions nodejs/docs/docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This Docker image is intended to be used for testing and development purposes on
### Pull the image

```bash
docker pull mcr.microsoft.com/playwright:v1.47.2-noble
docker pull mcr.microsoft.com/playwright:v1.48.0-noble
```

### Run the image
Expand All @@ -34,15 +34,15 @@ By default, the Docker image will use the `root` user to run the browsers. This
On trusted websites, you can avoid creating a separate user and use root for it since you trust the code which will run on the browsers.

```bash
docker run -it --rm --ipc=host mcr.microsoft.com/playwright:v1.47.2-noble /bin/bash
docker run -it --rm --ipc=host mcr.microsoft.com/playwright:v1.48.0-noble /bin/bash
```

#### Crawling and scraping

On untrusted websites, it's recommended to use a separate user for launching the browsers in combination with the seccomp profile. Inside the container or if you are using the Docker image as a base image you have to use `adduser` for it.

```bash
docker run -it --rm --ipc=host --user pwuser --security-opt seccomp=seccomp_profile.json mcr.microsoft.com/playwright:v1.47.2-noble /bin/bash
docker run -it --rm --ipc=host --user pwuser --security-opt seccomp=seccomp_profile.json mcr.microsoft.com/playwright:v1.48.0-noble /bin/bash
```

[`seccomp_profile.json`](https://github.com/microsoft/playwright/blob/main/utils/docker/seccomp_profile.json) is needed to run Chromium with sandbox. This is a [default Docker seccomp profile](https://github.com/docker/engine/blob/d0d99b04cf6e00ed3fc27e81fc3d94e7eda70af3/profiles/seccomp/default.json) with extra user namespace cloning permissions:
Expand Down Expand Up @@ -75,10 +75,10 @@ See our [Continuous Integration guides](./ci.mdx) for sample configs.
See [all available image tags].

We currently publish images with the following tags:
- `:v1.47.2` - Playwright v1.47.2 release docker image based on Ubuntu 24.04 LTS (Noble Numbat).
- `:v1.47.2-noble` - Playwright v1.47.2 release docker image based on Ubuntu 24.04 LTS (Noble Numbat).
- `:v1.47.2-jammy` - Playwright v1.47.2 release docker image based on Ubuntu 22.04 LTS (Jammy Jellyfish).
- `:v1.47.2-focal` - Playwright v1.47.2 release docker image based on Ubuntu 20.04 LTS (Focal Fossa).
- `:v1.48.0` - Playwright v1.48.0 release docker image based on Ubuntu 24.04 LTS (Noble Numbat).
- `:v1.48.0-noble` - Playwright v1.48.0 release docker image based on Ubuntu 24.04 LTS (Noble Numbat).
- `:v1.48.0-jammy` - Playwright v1.48.0 release docker image based on Ubuntu 22.04 LTS (Jammy Jellyfish).
- `:v1.48.0-focal` - Playwright v1.48.0 release docker image based on Ubuntu 20.04 LTS (Focal Fossa).

:::note

Expand All @@ -103,7 +103,7 @@ To run Playwright inside Docker, you need to have Node.js, [Playwright browsers]
```Dockerfile
FROM node:20-bookworm

RUN npx -y playwright@1.47.2 install --with-deps
RUN npx -y playwright@1.48.0 install --with-deps
```


Expand Down
17 changes: 17 additions & 0 deletions nodejs/docs/test-sharding.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,23 @@ Now, if you run these shards in parallel on different jobs, your test suite comp

Note that Playwright can only shard tests that can be run in parallel. By default, this means Playwright will shard test files. Learn about other options in the [parallelism guide](./test-parallel.mdx).

## Balancing Shards

Sharding can be done at two levels of granularity depending on whether you use the [testProject.fullyParallel](/api/class-testproject.mdx#test-project-fully-parallel) option or not. This affects how the tests are balanced across the shards.

**Sharding with fullyParallel**

When `fullyParallel: true` is enabled, Playwright Test runs individual tests in parallel across multiple shards, ensuring each shard receives an even distribution of tests. This allows for test-level granularity, meaning each shard will attempt to balance the number of individual tests it runs. This is the preferred mode for ensuring even load distribution when sharding, as Playwright can optimize shard execution based on the total number of tests.

**Sharding without fullyParallel**

Without the fullyParallel setting, Playwright Test defaults to file-level granularity, meaning entire test files are assigned to shards. In this case, the number of tests per file can greatly influence shard distribution. If your test files are not evenly sized (i.e., some files contain many more tests than others), certain shards may end up running significantly more tests, while others may run fewer or even none.

**Key Takeaways:**
- **With** `fullyParallel: true`: Tests are split at the individual test level, leading to more balanced shard execution.
- **Without** `fullyParallel`: Tests are split at the file level, so to balance the shards, it's important to keep your test files small and evenly sized.
- To ensure the most effective use of sharding, especially in CI environments, it is recommended to use `fullyParallel: true` when aiming for balanced distribution across shards. Otherwise, you may need to manually organize your test files to avoid imbalances.

## Merging reports from multiple shards

In the previous example, each test shard has its own test report. If you want to have a combined report showing all the test results from all the shards, you can merge them.
Expand Down

0 comments on commit eadbdfc

Please sign in to comment.