-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SWC-6536: handle timing issues related to using larger runner #5184
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -71,10 +71,26 @@ export async function loginTestUser( | |
|
||
await page.getByRole('link', { name: 'Log in to Synapse' }).first().click() | ||
await page.getByRole('button', { name: 'Sign in with your email' }).click() | ||
await page.getByLabel('Username or Email Address').fill(testUserName) | ||
await page.getByLabel('Password').fill(testUserPassword) | ||
|
||
const usernameInput = page.getByLabel('Username or Email Address') | ||
await usernameInput.fill(testUserName) | ||
await expect(usernameInput).toHaveValue(testUserName) | ||
|
||
const passwordInput = page.getByLabel('Password') | ||
await expect(passwordInput).toBeEmpty() | ||
await passwordInput.fill(testUserPassword) | ||
await expect(passwordInput).not.toBeEmpty() | ||
Comment on lines
+75
to
+82
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ensure that the username/password values have been set before trying to sign in |
||
|
||
await page.getByRole('button', { name: 'Sign in' }).click() | ||
|
||
// Ensure that correct username/password were received | ||
const loadingButton = page.getByRole('button', { name: 'Logging you in' }) | ||
await expect(loadingButton).toBeVisible() | ||
await expect(loadingButton).not.toBeVisible() | ||
await expect( | ||
page.getByText('The provided username/password combination is incorrect'), | ||
).not.toBeVisible() | ||
Comment on lines
+86
to
+92
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Confirm that the correct username/password were received, hopefully will help us troubleshoot if login fails to redirect as in this run |
||
|
||
// Wait for redirect | ||
await expect(async () => { | ||
expect(page.url()).not.toContain('LoginPlace') | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,7 +21,7 @@ dotenv.config() | |
export default defineConfig({ | ||
testDir: './e2e', | ||
/* Timeout to allow portal enough time to compile when running locally */ | ||
timeout: 5 * 60 * 1000, | ||
timeout: 2 * 60 * 1000, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reduce overall test timeout, so flaky tests fail faster -- currently, the longest test ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like the idea of having these fail faster if making these longer doesn't substantially make these less flaky. Optimizing for resilience is good, but I wouldn't worry much about shaving seconds or minutes off to save money. |
||
/* Increase expectation timeout on CI */ | ||
expect: { timeout: process.env.CI ? 30 * 1000 : 5 * 1000 }, | ||
/* Run tests in files in parallel */ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Successful
auth.setup
tests are taking 2-4 seconds on the larger Linux runner, 4-6 seconds locally, and 7-11 seconds on the macOS runner. Generally, this test is much faster than the other e2e tests, so stop marking this test as "slow" -- allow the test to fail faster when flaky, so we spend spend less money on larger runner in CI.