Skip to content

Commit

Permalink
feat(roll): roll to ToT Playwright (10-10-24) (#1559)
Browse files Browse the repository at this point in the history
  • Loading branch information
playwrightmachine authored Oct 10, 2024
1 parent 0c7d9a2 commit fff4a1d
Show file tree
Hide file tree
Showing 5 changed files with 68 additions and 67 deletions.
2 changes: 1 addition & 1 deletion nodejs/docs/api/class-test.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1088,7 +1088,7 @@ Timeout for the currently running test is available through [testInfo.timeout](/
});
```

* Changing timeout from a slow `beforeEach` or `afterEach` hook. Note that this affects the test timeout that is shared with `beforeEach`/`afterEach` hooks.
* Changing timeout from a slow `beforeEach` hook. Note that this affects the test timeout that is shared with `beforeEach` hooks.

```js
test.beforeEach(async ({ page }, testInfo) => {
Expand Down
21 changes: 21 additions & 0 deletions nodejs/docs/api/class-testconfig.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -809,6 +809,27 @@ export default defineConfig({

---

### tsconfig {#test-config-tsconfig}

<font size="2" style={{position: "relative", top: "-20px"}}>Added in: v1.49</font><x-search>testConfig.tsconfig</x-search>

Path to a single `tsconfig` applicable to all imported files. By default, `tsconfig` for each imported file is looked up separately. Note that `tsconfig` property has no effect while the configuration file or any of its dependencies are loaded. Ignored when `--tsconfig` command line option is specified.

**Usage**

```js title="playwright.config.ts"
import { defineConfig } from '@playwright/test';

export default defineConfig({
tsconfig: './tsconfig.test.json',
});
```

**Type**
- [string]

---

### updateSnapshots {#test-config-update-snapshots}

<font size="2" style={{position: "relative", top: "-20px"}}>Added in: v1.10</font><x-search>testConfig.updateSnapshots</x-search>
Expand Down
2 changes: 1 addition & 1 deletion nodejs/docs/test-configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ export default defineConfig({
| [testConfig.globalSetup](/api/class-testconfig.mdx#test-config-global-setup) | Path to the global setup file. This file will be required and run before all the tests. It must export a single function. |
| [testConfig.globalTeardown](/api/class-testconfig.mdx#test-config-global-teardown) |Path to the global teardown file. This file will be required and run after all the tests. It must export a single function. |
| [testConfig.outputDir](/api/class-testconfig.mdx#test-config-output-dir) | Folder for test artifacts such as screenshots, videos, traces, etc. |
| [testConfig.timeout](/api/class-testconfig.mdx#test-config-timeout) | Playwright enforces a [timeout](./test-timeouts.mdx) for each test, 30 seconds by default. Time spent by the test function, fixtures, beforeEach and afterEach hooks is included in the test timeout. |
| [testConfig.timeout](/api/class-testconfig.mdx#test-config-timeout) | Playwright enforces a [timeout](./test-timeouts.mdx) for each test, 30 seconds by default. Time spent by the test function, test fixtures and beforeEach hooks is included in the test timeout. |

## Expect Options

Expand Down
100 changes: 35 additions & 65 deletions nodejs/docs/test-timeouts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,16 @@ import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import HTMLCard from '@site/src/components/HTMLCard';

## Introduction

Playwright Test has multiple configurable timeouts for various tasks.

|Timeout |Default |Description |
|:----------|:----------------|:--------------------------------|
|Test timeout|30000 ms|Timeout for each test, includes test, hooks and fixtures:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set default</span><br/><code>{`config = { timeout: 60000 }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override</span><br/>`test.setTimeout(120000)` |
|Expect timeout|5000 ms|Timeout for each assertion:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set default</span><br/><code>{`config = { expect: { timeout: 10000 } }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override</span><br/>`expect(locator).toBeVisible({ timeout: 10000 })` |
|Test timeout|30_000 ms|Timeout for each test<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in config</span><br/><code>{`{ timeout: 60_000 }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override in test</span><br/>`test.setTimeout(120_000)` |
|Expect timeout|5_000 ms|Timeout for each assertion<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in config</span><br/><code>{`{ expect: { timeout: 10_000 } }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override in test</span><br/>`expect(locator).toBeVisible({ timeout: 10_000 })` |

## Test timeout

Playwright Test enforces a timeout for each test, 30 seconds by default. Time spent by the test function, fixtures, `beforeEach` and `afterEach` hooks is included in the test timeout.
Playwright Test enforces a timeout for each test, 30 seconds by default. Time spent by the test function, fixture setups, and `beforeEach` hooks is included in the test timeout.

Timed out test produces the following error:

Expand All @@ -27,6 +25,8 @@ example.spec.ts:3:1 › basic test ===========================
Timeout of 30000ms exceeded.
```

Additional separate timeout, of the same value, is shared between fixture teardowns and `afterEach` hooks, after the test function has finished.

The same timeout value also applies to `beforeAll` and `afterAll` hooks, but they do not share time with any test.

### Set test timeout in the config
Expand All @@ -35,15 +35,15 @@ The same timeout value also applies to `beforeAll` and `afterAll` hooks, but the
import { defineConfig } from '@playwright/test';

export default defineConfig({
timeout: 5 * 60 * 1000,
timeout: 120_000,
});
```

API reference: [testConfig.timeout](/api/class-testconfig.mdx#test-config-timeout).

### Set timeout for a single test

```js
```js title="example.spec.ts"
import { test, expect } from '@playwright/test';

test('slow test', async ({ page }) => {
Expand All @@ -52,7 +52,7 @@ test('slow test', async ({ page }) => {
});

test('very slow test', async ({ page }) => {
test.setTimeout(120000);
test.setTimeout(120_000);
// ...
});
```
Expand All @@ -61,12 +61,12 @@ API reference: [test.setTimeout()](/api/class-test.mdx#test-set-timeout) and [te

### Change timeout from a `beforeEach` hook

```js
```js title="example.spec.ts"
import { test, expect } from '@playwright/test';

test.beforeEach(async ({ page }, testInfo) => {
// Extend timeout for all tests running this hook by 30 seconds.
testInfo.setTimeout(testInfo.timeout + 30000);
testInfo.setTimeout(testInfo.timeout + 30_000);
});
```

Expand All @@ -76,7 +76,7 @@ API reference: [testInfo.setTimeout()](/api/class-testinfo.mdx#test-info-set-tim

`beforeAll` and `afterAll` hooks have a separate timeout, by default equal to test timeout. You can change it separately for each hook by calling [testInfo.setTimeout()](/api/class-testinfo.mdx#test-info-set-timeout) inside the hook.

```js
```js title="example.spec.ts"
import { test, expect } from '@playwright/test';

test.beforeAll(async () => {
Expand All @@ -89,7 +89,7 @@ API reference: [testInfo.setTimeout()](/api/class-testinfo.mdx#test-info-set-tim

## Expect timeout

Web-first assertions like `expect(locator).toHaveText()` have a separate timeout, 5 seconds by default. Assertion timeout is unrelated to the test timeout. It produces the following error:
Auto-retrying assertions like [expect(locator).toHaveText()](/api/class-locatorassertions.mdx#locator-assertions-to-have-text) have a separate timeout, 5 seconds by default. Assertion timeout is unrelated to the test timeout. It produces the following error:

```txt
example.spec.ts:3:1 › basic test ===========================
Expand All @@ -110,11 +110,23 @@ import { defineConfig } from '@playwright/test';

export default defineConfig({
expect: {
timeout: 10 * 1000,
timeout: 10_000,
},
});
```

API reference: [testConfig.expect](/api/class-testconfig.mdx#test-config-expect).

### Specify expect timeout for a single assertion

```js title="example.spec.ts"
import { test, expect } from '@playwright/test';

test('example', async ({ page }) => {
await expect(locator).toHaveText('hello', { timeout: 10_000 });
});
```

## Global timeout

Playwright Test supports a timeout for the whole test run. This prevents excess resource usage when everything went wrong. There is no default global timeout, but you can set a reasonable one in the config, for example one hour. Global timeout produces the following error:
Expand All @@ -129,12 +141,11 @@ Running 1000 tests using 10 workers

You can set global timeout in the config.

```js
// playwright.config.ts
```js title="playwright.config.ts"
import { defineConfig } from '@playwright/test';

export default defineConfig({
globalTimeout: 60 * 60 * 1000,
globalTimeout: 3_600_000,
});
```

Expand All @@ -146,21 +157,11 @@ These are the low-level timeouts that are pre-configured by the test runner, you

|Timeout |Default |Description |
|:----------|:----------------|:--------------------------------|
|Action timeout| no timeout |Timeout for each action:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set default</span><br/><code>{`config = { use: { actionTimeout: 10000 } }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override</span><br/>`locator.click({ timeout: 10000 })` |
|Navigation timeout| no timeout |Timeout for each navigation action:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set default</span><br/><code>{`config = { use: { navigationTimeout: 30000 } }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override</span><br/>`page.goto('/', { timeout: 30000 })` |
|Global timeout|no timeout |Global timeout for the whole test run:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in config</span><br/>`config = { globalTimeout: 60*60*1000 }`<br/> |
|`beforeAll`/`afterAll` timeout|30000 ms|Timeout for the hook:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in hook</span><br/>`test.setTimeout(60000)`<br/> |
|Fixture timeout|no timeout |Timeout for an individual fixture:<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in fixture</span><br/>`{ scope: 'test', timeout: 30000 }`<br/> |

### Set timeout for a single assertion

```js
import { test, expect } from '@playwright/test';

test('basic test', async ({ page }) => {
await expect(page.getByRole('button')).toHaveText('Sign in', { timeout: 10000 });
});
```
|Action timeout| no timeout |Timeout for each action<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in config</span><br/><code>{`{ use: { actionTimeout: 10_000 } }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override in test</span><br/>`locator.click({ timeout: 10_000 })` |
|Navigation timeout| no timeout |Timeout for each navigation action<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in config</span><br/><code>{`{ use: { navigationTimeout: 30_000 } }`}</code><br/><span style={{textTransform: 'uppercase',fontSize: 'smaller', fontWeight: 'bold', opacity: '0.7'}}>Override in test</span><br/>`page.goto('/', { timeout: 30_000 })` |
|Global timeout|no timeout |Global timeout for the whole test run<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in config</span><br/>`{ globalTimeout: 3_600_000 }`<br/> |
|`beforeAll`/`afterAll` timeout|30_000 ms|Timeout for the hook<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in hook</span><br/>`test.setTimeout(60_000)`<br/> |
|Fixture timeout|no timeout |Timeout for an individual fixture<br/><span style={{textTransform:'uppercase',fontSize:'smaller',fontWeight:'bold',opacity:'0.7'}}>Set in fixture</span><br/>`{ scope: 'test', timeout: 30_000 }`<br/> |

### Set action and navigation timeouts in the config

Expand All @@ -179,7 +180,7 @@ API reference: [testOptions.actionTimeout](/api/class-testoptions.mdx#test-optio

### Set timeout for a single action

```js
```js title="example.spec.ts"
import { test, expect } from '@playwright/test';

test('basic test', async ({ page }) => {
Expand All @@ -192,52 +193,21 @@ test('basic test', async ({ page }) => {

By default, [fixture](./test-fixtures) shares timeout with the test. However, for slow fixtures, especially [worker-scoped](./test-fixtures#worker-scoped-fixtures) ones, it is convenient to have a separate timeout. This way you can keep the overall test timeout small, and give the slow fixture more time.

<Tabs
groupId="js-flavor"
defaultValue="ts"
values={[
{label: 'TypeScript', value: 'ts'},
{label: 'JavaScript', value: 'js'}
]
}>
<TabItem value="ts">

```js
```js title="example.spec.ts"
import { test as base, expect } from '@playwright/test';

const test = base.extend<{ slowFixture: string }>({
slowFixture: [async ({}, use) => {
// ... perform a slow operation ...
await use('hello');
}, { timeout: 60000 }]
}, { timeout: 60_000 }]
});

test('example test', async ({ slowFixture }) => {
// ...
});
```

</TabItem>
<TabItem value="js">

```js
const { test: base, expect } = require('@playwright/test');

const test = base.extend({
slowFixture: [async ({}, use) => {
// ... perform a slow operation ...
await use('hello');
}, { timeout: 60000 }]
});

test('example test', async ({ slowFixture }) => {
// ...
});
```

</TabItem>
</Tabs>

API reference: [test.extend()](/api/class-test.mdx#test-extend).


Expand Down
10 changes: 10 additions & 0 deletions nodejs/docs/test-typescript.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,16 @@ Alternatively, you can specify a single tsconfig file to use in the command line
npx playwright test --tsconfig=tsconfig.test.json
```

You can specify a single tsconfig file in the config file, that will be used for loading test files, reporters, etc. However, it will not be used while loading the playwright config itself or any files imported from it.

```js title="playwright.config.ts"
import { defineConfig } from '@playwright/test';
export default defineConfig({
tsconfig: './tsconfig.test.json',
});
```

## Manually compile tests with TypeScript

Sometimes, Playwright Test will not be able to transform your TypeScript code correctly, for example when you are using experimental or very recent features of TypeScript, usually configured in `tsconfig.json`.
Expand Down

0 comments on commit fff4a1d

Please sign in to comment.