Skip to content

New quiz types to production #2799

New quiz types to production

New quiz types to production #2799

Triggered via pull request August 15, 2023 06:58
Status Failure
Total duration 1h 4m 42s
Artifacts 4

system-tests-and-deploy.yml

on: pull_request
Matrix: system-tests
Fit to window
Zoom out
Zoom in

Annotations

15 errors, 44 warnings, and 2 notices
[chromium] › model-solution.spec.ts:61:7 › Model solutions › model-solutions are not displayed in the exercises: system-tests/src/utils/screenshot.ts#L339
1) [chromium] › model-solution.spec.ts:61:7 › Model solutions › model-solutions are not displayed in the exercises › Expect screenshots to match snapshots "model-solutions-in-exercises" Error: Screenshot comparison failed: 3343 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(model-solutions-in-exercises-mobile-tall.png) with timeout 5000ms - verifying given screenshot expectation - taking page screenshot - disabled all CSS animations - 3343 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - taking page screenshot - disabled all CSS animations - captured a stable screenshot - 3343 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/model-solution-Model-solutions-model-solutions-are-not-displayed-in-the-exercises-chromium/model-solutions-in-exercises-mobile-tall-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/model-solution-Model-solutions-model-solutions-are-not-displayed-in-the-exercises-chromium/model-solutions-in-exercises-mobile-tall-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/model-solution-Model-solutions-model-solutions-are-not-displayed-in-the-exercises-chromium/model-solutions-in-exercises-mobile-tall-1-diff.png at ../utils/screenshot.ts:339 337 | await page.waitForTimeout(600) 338 | if (isPage(screenshotTarget)) { > 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 340 | } else { 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 342 | } at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:339:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:149:9 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/model-solution.spec.ts:86:5
[chromium] › multiple-tasks-exercise-tests/enable-resubmitting.spec.ts:11:5 › quizzes: system-tests/src/utils/screenshot.ts#L341
2) [chromium] › multiple-tasks-exercise-tests/enable-resubmitting.spec.ts:11:5 › quizzes, after wrong answer modify only the incorrect choice and resubmit › Expect screenshots to match snapshots "before-answering-resubmitting-test" Error: Screenshot comparison failed: Expected an image 1920px by 829px, received 1920px by 1223px. 17329 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(before-answering-resubmitting-test-desktop-regular.png) with timeout 5000ms - verifying given screenshot expectation - waiting for locator('id=be169e81-f217-5bee-8475-2d94a5c67045') - locator resolved to <section class="css-bkcw7t" aria-labelledby=":r2:" id="be…>…</section> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - Expected an image 1920px by 829px, received 1920px by 1223px. 17329 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('id=be169e81-f217-5bee-8475-2d94a5c67045') - locator resolved to <section class="css-bkcw7t" aria-labelledby=":r2:" id="be…>…</section> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - Expected an image 1920px by 829px, received 1920px by 1223px. 17329 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/multiple-tasks-exercise-tests-enable-resubmitt-a56e0-r-modify-only-the-incorrect-choice-and-resubmit-chromium/before-answering-resubmitting-test-desktop-regular-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/multiple-tasks-exercise-tests-enable-resubmitt-a56e0-r-modify-only-the-incorrect-choice-and-resubmit-chromium/before-answering-resubmitting-test-desktop-regular-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/multiple-tasks-exercise-tests-enable-resubmitt-a56e0-r-modify-only-the-incorrect-choice-and-resubmit-chromium/before-answering-resubmitting-test-desktop-regular-1-diff.png at ../utils/screenshot.ts:341 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 340 | } else { > 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 342 | } 343 | } finally { 344 | testInfo.config.updateSnapshots = originalUpdateSnapshotsSetting at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:341:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:165:7 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/multiple-tasks-exercise-tests/enable-resubmitting.spec.ts:23:3
[chromium] › multiple-tasks-exercise-tests/multiple-task-exercise.spec.ts:16:5 › Exercise score updates gradually: system-tests/src/utils/screenshot.ts#L341
3) [chromium] › multiple-tasks-exercise-tests/multiple-task-exercise.spec.ts:16:5 › Exercise score updates gradually › Expect screenshots to match snapshots "exercise-before-answering" Error: Screenshot comparison failed: 3343 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(exercise-before-answering-mobile-tall.png) with timeout 5000ms - verifying given screenshot expectation - waiting for locator('id=c1d545d7-c46b-5076-8f34-32374dd03310') - locator resolved to <section class="css-bkcw7t" aria-labelledby=":r2:" id="c1…>…</section> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - 3343 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('id=c1d545d7-c46b-5076-8f34-32374dd03310') - locator resolved to <section class="css-bkcw7t" aria-labelledby=":r2:" id="c1…>…</section> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - 3343 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/multiple-tasks-exercise-tests-multiple-task-exercise-Exercise-score-updates-gradually-chromium/exercise-before-answering-mobile-tall-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/multiple-tasks-exercise-tests-multiple-task-exercise-Exercise-score-updates-gradually-chromium/exercise-before-answering-mobile-tall-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/multiple-tasks-exercise-tests-multiple-task-exercise-Exercise-score-updates-gradually-chromium/exercise-before-answering-mobile-tall-1-diff.png at ../utils/screenshot.ts:341 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 340 | } else { > 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 342 | } 343 | } finally { 344 | testInfo.config.updateSnapshots = originalUpdateSnapshotsSetting at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:341:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:149:9 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/multiple-tasks-exercise-tests/multiple-task-exercise.spec.ts:38:3
[chromium] › playground-views.test.ts:5:5 › Playground views works: system-tests/src/tests/playground-views.test.ts#L51
4) [chromium] › playground-views.test.ts:5:5 › Playground views works ──────────────────────────── TimeoutError: locator.waitFor: Timeout 15000ms exceeded. =========================== logs =========================== waiting for getByText('"name": "b"\n }\n]') to be visible ============================================================ 49 | ]`, 50 | ) > 51 | .waitFor() | ^ 52 | await page 53 | .getByText( 54 | `{ at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/playground-views.test.ts:51:6
[chromium] › proposed-edits/proposed-edits.spec.ts:11:5 › test: system-tests/src/utils/screenshot.ts#L339
5) [chromium] › proposed-edits/proposed-edits.spec.ts:11:5 › test › Expect screenshots to match snapshots "no-edits-yet" Error: Screenshot comparison failed: 3343 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(no-edits-yet-mobile-tall.png) with timeout 5000ms - verifying given screenshot expectation - taking page screenshot - disabled all CSS animations - 3343 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - taking page screenshot - disabled all CSS animations - captured a stable screenshot - 3343 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/proposed-edits-proposed-edits-test-chromium/no-edits-yet-mobile-tall-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/proposed-edits-proposed-edits-test-chromium/no-edits-yet-mobile-tall-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/proposed-edits-proposed-edits-test-chromium/no-edits-yet-mobile-tall-1-diff.png at ../utils/screenshot.ts:339 337 | await page.waitForTimeout(600) 338 | if (isPage(screenshotTarget)) { > 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 340 | } else { 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 342 | } at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:339:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:149:9 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/proposed-edits/proposed-edits.spec.ts:37:3
[chromium] › quizzes/create-and-save-quizzes.spec.ts:11:5 › create quizzes test: system-tests/src/tests/quizzes/create-and-save-quizzes.spec.ts#L126
6) [chromium] › quizzes/create-and-save-quizzes.spec.ts:11:5 › create quizzes test ─────────────── TimeoutError: locator.click: Timeout 15000ms exceeded. =========================== logs =========================== waiting for frameLocator(':nth-match(iframe[src*="http://project-331.local/quizzes/iframe"], 1)').locator('body').locator('#quiz-option-card-open') ============================================================ 124 | await scrollToFrame(page, frame3) 125 | > 126 | await frame3.locator(`#quiz-option-card-open`).click() | ^ 127 | await frame3.locator(`label:has-text("Validity regular expression") input`).fill(`1\\/2`) 128 | await frame3.locator(`label:has-text("Format regular expression") input`).fill(`\\d+\\/\\d+`) 129 | await frame3 at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/quizzes/create-and-save-quizzes.spec.ts:126:50
[chromium] › quizzes/feedback/clickable-multiple-choice.spec.ts:11:5 › test quizzes clickable multiple-choice feedback: system-tests/src/tests/quizzes/feedback/clickable-multiple-choice.spec.ts#L40
7) [chromium] › quizzes/feedback/clickable-multiple-choice.spec.ts:11:5 › test quizzes clickable multiple-choice feedback TimeoutError: locator.click: Timeout 15000ms exceeded. =========================== logs =========================== waiting for locator('text=Submit') locator resolved to <button disabled class="css-1fytp2i-PrimaryButton e4cjxi…>Submit</button> attempting click action waiting for element to be visible, enabled and stable element is not enabled - waiting... ============================================================ 38 | await frame.locator(`button:text("Jupiter")`).click() 39 | > 40 | await page.locator("text=Submit").click() | ^ 41 | 42 | await expectScreenshotsToMatchSnapshots({ 43 | screenshotTarget: page, at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/quizzes/feedback/clickable-multiple-choice.spec.ts:40:37
[chromium] › quizzes/feedback/timeline.spec.ts:10:5 › test quizzes timeline feedback: system-tests/src/tests/quizzes/feedback/timeline.spec.ts#L49
8) [chromium] › quizzes/feedback/timeline.spec.ts:10:5 › test quizzes timeline feedback ────────── TimeoutError: locator.click: Timeout 15000ms exceeded. =========================== logs =========================== waiting for frameLocator('iframe').locator('[aria-label="Remove"]:right-of(:text("2002"))').first() ============================================================ 47 | .locator(`[aria-label="Remove"]:right-of(:text("2002"))`) 48 | .nth(0) > 49 | .click() | ^ 50 | 51 | await page 52 | .frameLocator("iframe") at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/quizzes/feedback/timeline.spec.ts:49:6
[chromium] › quizzes/feedback/vector.spec.ts:10:5 › test quizzes vector feedback: system-tests/src/utils/screenshot.ts#L339
9) [chromium] › quizzes/feedback/vector.spec.ts:10:5 › test quizzes vector feedback › Expect screenshots to match snapshots "vector-initial" Error: Screenshot comparison failed: 133310 pixels (ratio 0.07 of all image pixels) are different. Call log: - expect.toHaveScreenshot(vector-initial-desktop-regular.png) with timeout 5000ms - verifying given screenshot expectation - taking page screenshot - disabled all CSS animations - 133310 pixels (ratio 0.07 of all image pixels) are different. - waiting 100ms before taking screenshot - taking page screenshot - disabled all CSS animations - captured a stable screenshot - 133310 pixels (ratio 0.07 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/quizzes-feedback-vector-test-quizzes-vector-feedback-chromium/vector-initial-desktop-regular-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/quizzes-feedback-vector-test-quizzes-vector-feedback-chromium/vector-initial-desktop-regular-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/quizzes-feedback-vector-test-quizzes-vector-feedback-chromium/vector-initial-desktop-regular-1-diff.png at ../utils/screenshot.ts:339 337 | await page.waitForTimeout(600) 338 | if (isPage(screenshotTarget)) { > 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 340 | } else { 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 342 | } at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:339:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:165:7 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/quizzes/feedback/vector.spec.ts:17:3
[chromium] › quizzes/widget/multiple-choice-course-material-widget-column.spec.ts:10:5 › multiple-choice course material column test: system-tests/src/utils/screenshot.ts#L341
10) [chromium] › quizzes/widget/multiple-choice-course-material-widget-column.spec.ts:10:5 › multiple-choice course material column test › Expect screenshots to match snapshots "course-material-multiple-choice-before-success-click-column-single" Error: Screenshot comparison failed: Expected an image 1920px by 151px, received 1920px by 127px. Call log: - expect.toHaveScreenshot(course-material-multiple-choice-before-success-click-column-single-desktop-regular.png) with timeout 5000ms - verifying given screenshot expectation - waiting for frameLocator(':nth-match(iframe[src*="http://project-331.local/quizzes/iframe"], 1)').locator('body') - locator resolved to <body>…</body> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - Expected an image 1920px by 151px, received 1920px by 127px. - waiting 100ms before taking screenshot - waiting for frameLocator(':nth-match(iframe[src*="http://project-331.local/quizzes/iframe"], 1)').locator('body') - locator resolved to <body>…</body> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - Expected an image 1920px by 151px, received 1920px by 127px. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/quizzes-widget-multiple-choice-course-material-42dce-umn-multiple-choice-course-material-column-test-chromium/course-material-multiple-choice-before-success-click-column-single-desktop-regular-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/quizzes-widget-multiple-choice-course-material-42dce-umn-multiple-choice-course-material-column-test-chromium/course-material-multiple-choice-before-success-click-column-single-desktop-regular-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/quizzes-widget-multiple-choice-course-material-42dce-umn-multiple-choice-course-material-column-test-chromium/course-material-multiple-choice-before-success-click-column-single-desktop-regular-1-diff.png at ../utils/screenshot.ts:341 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 340 | } else { > 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 342 | } 343 | } finally { 344 | testInfo.config.updateSnapshots = originalUpdateSnapshotsSetting at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:341:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:165:7 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/quizzes/widget/multiple-choice-course-material-widget-column.spec.ts:41:3
[chromium] › creating-course-and-returning-an-exercise.spec.ts:14:5 › test: system-tests/src/tests/creating-course-and-returning-an-exercise.spec.ts#L147
1) [chromium] › creating-course-and-returning-an-exercise.spec.ts:14:5 › test ──────────────────── TimeoutError: page.waitForSelector: Timeout 15000ms exceeded. =========================== logs =========================== waiting for locator('text="Operation successful!"') to be visible ============================================================ 145 | 146 | await page.click('button:text-is("Save") >> visible=true') > 147 | await page.waitForSelector(`text="Operation successful!"`) | ^ 148 | 149 | // Check that the assignment still displays after saving 150 | await page.click('[aria-label="Block: ExerciseTask"] [aria-label="Edit"]') at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/creating-course-and-returning-an-exercise.spec.ts:147:14
[chromium] › feedback/feedback.spec.ts:15:5 › feedback test: system-tests/src/utils/screenshot.ts#L339
2) [chromium] › feedback/feedback.spec.ts:15:5 › feedback test › Expect screenshots to match snapshots "feedback-tooltip" Error: Screenshot comparison failed: 3245 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(feedback-tooltip-mobile-tall.png) with timeout 5000ms - verifying given screenshot expectation - taking page screenshot - disabled all CSS animations - 3245 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - taking page screenshot - disabled all CSS animations - captured a stable screenshot - 3245 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/feedback-feedback-feedback-test-chromium/feedback-tooltip-mobile-tall-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/feedback-feedback-feedback-test-chromium/feedback-tooltip-mobile-tall-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/feedback-feedback-feedback-test-chromium/feedback-tooltip-mobile-tall-1-diff.png at ../utils/screenshot.ts:339 337 | await page.waitForTimeout(600) 338 | if (isPage(screenshotTarget)) { > 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 340 | } else { 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 342 | } at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:339:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:149:9 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/feedback/feedback.spec.ts:47:3
[chromium] › history/history.spec.ts:16:5 › history test: system-tests/src/utils/screenshot.ts#L339
3) [chromium] › history/history.spec.ts:16:5 › history test › Expect screenshots to match snapshots "initial-page" Error: Screenshot comparison failed: 3343 pixels (ratio 0.01 of all image pixels) are different. Call log: - expect.toHaveScreenshot(initial-page-mobile-tall.png) with timeout 5000ms - verifying given screenshot expectation - taking page screenshot - disabled all CSS animations - 3343 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - taking page screenshot - disabled all CSS animations - captured a stable screenshot - 3343 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/history-history-history-test-chromium/initial-page-mobile-tall-expected.png Received: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/history-history-history-test-chromium/initial-page-mobile-tall-1-actual.png Diff: /home/runner/work/secret-project-331/secret-project-331/system-tests/test-results/history-history-history-test-chromium/initial-page-mobile-tall-1-diff.png at ../utils/screenshot.ts:339 337 | await page.waitForTimeout(600) 338 | if (isPage(screenshotTarget)) { > 339 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) | ^ 340 | } else { 341 | await expect(screenshotTarget).toHaveScreenshot(screenshotName, screenshotOptions) 342 | } at takeScreenshotAndComparetoSnapshot (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:339:38) at snapshotWithViewPort (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:273:5) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:149:9 at expectScreenshotsToMatchSnapshots (/home/runner/work/secret-project-331/secret-project-331/system-tests/src/utils/screenshot.ts:88:3) at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/history/history.spec.ts:34:3
[chromium] › limited-tries.spec.ts:9:5 › Limited tries work: system-tests/src/tests/limited-tries.spec.ts#L66
4) [chromium] › limited-tries.spec.ts:9:5 › Limited tries work ─────────────────────────────────── TimeoutError: locator.click: Timeout 15000ms exceeded. =========================== logs =========================== waiting for locator('text=Submit') locator resolved to <button disabled class="css-1fytp2i-PrimaryButton e4cjxi…>Submit</button> attempting click action waiting for element to be visible, enabled and stable element is not enabled - waiting... ============================================================ 64 | await page.frameLocator("iframe").locator("text=AC").click() 65 | > 66 | await page.locator("text=Submit").click() | ^ 67 | 68 | await page.locator("text=Tries remaining: 1").click() 69 | at /home/runner/work/secret-project-331/secret-project-331/system-tests/src/tests/limited-tries.spec.ts:66:37
system-tests (1)
The operation was canceled.
typecheck
The following actions uses node12 which is deprecated and will be forced to run on node16: actions/checkout@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
build
The following actions uses node12 which is deprecated and will be forced to run on node16: actions/checkout@v2, yokawasa/[email protected], actions/upload-artifact@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
build
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The following actions uses node12 which is deprecated and will be forced to run on node16: actions/checkout@v2, yokawasa/[email protected], actions/download-artifact@v2, actions/upload-artifact@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
Slow Test: system-tests/[chromium] › peer-reviews/manual-review-everything.spec.ts#L1
system-tests/[chromium] › peer-reviews/manual-review-everything.spec.ts took 1.9m
Slow Test: system-tests/[chromium] › material-reference.spec.ts#L1
system-tests/[chromium] › material-reference.spec.ts took 58.0s
Slow Test: system-tests/[chromium] › peer-reviews/automatically-accept-or-reject.spec.ts#L1
system-tests/[chromium] › peer-reviews/automatically-accept-or-reject.spec.ts took 46.7s
Slow Test: system-tests/[chromium] › peer-reviews/automatically-accept-or-manual-review-by-average.spec.ts#L1
system-tests/[chromium] › peer-reviews/automatically-accept-or-manual-review-by-average.spec.ts took 44.5s
Slow Test: system-tests/[chromium] › quizzes/widget/multiple-choice-course-material-widget-row.spec.ts#L1
system-tests/[chromium] › quizzes/widget/multiple-choice-course-material-widget-row.spec.ts took 39.6s
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (2)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The following actions uses node12 which is deprecated and will be forced to run on node16: actions/checkout@v2, yokawasa/[email protected], actions/download-artifact@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
system-tests (1)
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Slow Test: system-tests/[chromium] › course-modules/course-modules.spec.ts#L1
system-tests/[chromium] › course-modules/course-modules.spec.ts took 57.2s
Slow Test: system-tests/[chromium] › draft.spec.ts#L1
system-tests/[chromium] › draft.spec.ts took 46.1s
Slow Test: system-tests/[chromium] › automatic-completion.spec.ts#L1
system-tests/[chromium] › automatic-completion.spec.ts took 42.6s
Slow Test: system-tests/[chromium] › glossary.spec.ts#L1
system-tests/[chromium] › glossary.spec.ts took 42.2s
Slow Test: system-tests/[chromium] › creating-course-and-returning-an-exercise.spec.ts#L1
system-tests/[chromium] › creating-course-and-returning-an-exercise.spec.ts took 39.4s
🎭 Playwright Run Summary
13 failed [chromium] › model-solution.spec.ts:61:7 › Model solutions › model-solutions are not displayed in the exercises [chromium] › multiple-tasks-exercise-tests/enable-resubmitting.spec.ts:11:5 › quizzes, after wrong answer modify only the incorrect choice and resubmit [chromium] › multiple-tasks-exercise-tests/multiple-task-exercise.spec.ts:16:5 › Exercise score updates gradually [chromium] › playground-views.test.ts:5:5 › Playground views works ───────────────────────────── [chromium] › proposed-edits/proposed-edits.spec.ts:11:5 › test ───────────────────────────────── [chromium] › quizzes/create-and-save-quizzes.spec.ts:11:5 › create quizzes test ──────────────── [chromium] › quizzes/feedback/clickable-multiple-choice.spec.ts:11:5 › test quizzes clickable multiple-choice feedback [chromium] › quizzes/feedback/timeline.spec.ts:10:5 › test quizzes timeline feedback ─────────── [chromium] › quizzes/feedback/vector.spec.ts:10:5 › test quizzes vector feedback ─────────────── [chromium] › quizzes/widget/multiple-choice-course-material-widget-column.spec.ts:10:5 › multiple-choice course material column test [chromium] › quizzes/widget/multiple-choice-multi-widget.spec.ts:13:5 › widget, multiple-choice multi screenshot test [chromium] › quizzes/widget/open-widget.spec.ts:10:5 › widget, open ──────────────────────────── [chromium] › quizzes/widget/scale-widget.spec.ts:10:5 › widget, scale ────────────────────────── 35 passed (14.7m)
🎭 Playwright Run Summary
4 failed [chromium] › creating-course-and-returning-an-exercise.spec.ts:14:5 › test ───────────────────── [chromium] › feedback/feedback.spec.ts:15:5 › feedback test ──────────────────────────────────── [chromium] › history/history.spec.ts:16:5 › history test ─────────────────────────────────────── [chromium] › limited-tries.spec.ts:9:5 › Limited tries work ──────────────────────────────────── 44 passed (13.7m)

Artifacts

Produced during runtime
Name Size
playwright-html-report Expired
787 KB
skaffold-build-output Expired
3.13 KB
skaffold-images Expired
344 MB
test-results Expired
38.2 MB