Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a nice message when reftest screenshots differ in size #49837

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Add a nice message when reftest screenshots differ in size

e2a2e32
Select commit
Loading
Failed to load commit list.
Open

Add a nice message when reftest screenshots differ in size #49837

Add a nice message when reftest screenshots differ in size
e2a2e32
Select commit
Loading
Failed to load commit list.
Community-TC Integration / infrastructure/ tests succeeded Dec 24, 2024 in 19m 24s

Community-TC (pull_request)

Smoketests for wptrunner

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2024-12-24T06:06:27.876Z
Resolved: 2024-12-24T06:22:54.763Z
Task Execution Time: 16 minutes, 26 seconds, 887 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log
- public/results/checkrun.md
- public/results/emulator.log
- public/results/logcat-emulator-5554.log
- public/results/wptreport-chrome.json.gz
- public/results/wptreport-firefox_android.json.gz
- public/results/wptreport-firefox.json.gz

WPT Command: python3 ./wpt run --log-tbpl=- --log-wptreport=../artifacts/wptreport-chrome.json --logcat-dir=../artifacts/ --metadata=infrastructure/metadata/ --include=infrastructure/ --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --no-headless --verify-log-full --enable-swiftshader --install-browser --install-webdriver chrome

WPT Command: python3 ./wpt run --log-tbpl=- --log-wptreport=../artifacts/wptreport-firefox.json --logcat-dir=../artifacts/ --metadata=infrastructure/metadata/ --include=infrastructure/ --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --no-headless --verify-log-full --binary=/home/test/build/firefox/firefox firefox

WPT Command: python3 ./wpt run --log-tbpl=- --log-wptreport=../artifacts/wptreport-firefox_android.json --logcat-dir=../artifacts/ --metadata=infrastructure/metadata/ --include=infrastructure/ --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --no-headless --verify-log-full --install-browser --install-webdriver --logcat-dir /home/test/artifacts/ firefox_android

[taskcluster 2024-12-24 06:06:27.966Z] Task ID: EbpkyfwoSW6HXoTmd8dK6A
[taskcluster 2024-12-24 06:06:27.966Z] Worker ID: 6382543209968955863
[taskcluster 2024-12-24 06:06:27.966Z] Worker Group: us-east1-b
[taskcluster 2024-12-24 06:06:27.966Z] Worker Node Type: projects/757942385826/machineTypes/n2-standard-4
[taskcluster 2024-12-24 06:06:27.966Z] Worker Pool: proj-wpt/ci
[taskcluster 2024-12-24 06:06:27.967Z] Worker Version: 46.1.0
[taskcluster 2024-12-24 06:06:27.967Z] Public IP: 35.237.97.191
[taskcluster 2024-12-24 06:06:27.967Z] Hostname: proj-wpt-ci-begc2iwqs0-juz2enb-sxq

ff2b96c8f8ff - Started downloading
ff2b96c8f8ff - Downloaded in 0.011 seconds
72ebe8ecf66d - Started downloading
560c024910be - Started downloading
430011f338a2 - Started downloading
430011f338a2 - Downloaded in 0.043 seconds
560c024910be - Downloaded in 0.191 seconds
79df10d18997 - Started downloading
4f4fb700ef54 - Started downloading
4f4fb700ef54 - Downloaded in 0.001 seconds
7e274023cae9 - Started downloading

...(7315 lines hidden)...

 8:10.89 TEST_START: /infrastructure/reftest/reftest.https.html
TEST-START | /infrastructure/reftest/reftest.https.html
 8:11.57 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest.https.html | took 679ms
 8:11.57 TEST_START: /infrastructure/reftest/reftest.www.sub.html
TEST-START | /infrastructure/reftest/reftest.www.sub.html
 8:12.01 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest.www.sub.html | took 440ms
 8:12.01 TEST_START: /infrastructure/reftest/reftest_fuzzy_1.html
TEST-START | /infrastructure/reftest/reftest_fuzzy_1.html
 8:12.85 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_fuzzy_1.html | took 838ms
 8:12.85 TEST_START: /infrastructure/reftest/reftest_fuzzy_ini_full.html
TEST-START | /infrastructure/reftest/reftest_fuzzy_ini_full.html
 8:13.34 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_fuzzy_ini_full.html | took 491ms
 8:13.34 TEST_START: /infrastructure/reftest/reftest_fuzzy_ini_ref_only.html
TEST-START | /infrastructure/reftest/reftest_fuzzy_ini_ref_only.html
 8:13.76 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_fuzzy_ini_ref_only.html | took 419ms
 8:13.76 TEST_START: /infrastructure/reftest/reftest_fuzzy_ini_short.html
TEST-START | /infrastructure/reftest/reftest_fuzzy_ini_short.html
 8:14.18 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_fuzzy_ini_short.html | took 424ms
 8:14.18 TEST_START: /infrastructure/reftest/reftest_fuzzy_no_differences.html
TEST-START | /infrastructure/reftest/reftest_fuzzy_no_differences.html
 8:14.62 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_fuzzy_no_differences.html | took 433ms
 8:14.62 TEST_START: /infrastructure/reftest/reftest_fuzzy_no_differences_1.html
TEST-START | /infrastructure/reftest/reftest_fuzzy_no_differences_1.html
 8:15.06 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_fuzzy_no_differences_1.html | took 442ms
 8:15.06 TEST_START: /infrastructure/reftest/reftest_match.html
TEST-START | /infrastructure/reftest/reftest_match.html
 8:15.53 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_match.html | took 472ms
 8:15.53 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-0.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-0.html
 8:16.42 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_and_mismatch-0.html | took 886ms
 8:16.42 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-1.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-1.html
 8:16.88 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_and_mismatch-1.html | took 463ms
 8:16.88 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-2.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-2.html
 8:17.58 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_match_and_mismatch-2.html | took 699ms
 8:17.58 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-3.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-3.html
 8:17.99 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_match_and_mismatch-3.html | took 406ms
 8:17.99 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-4.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-4.html
 8:18.70 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_and_mismatch-4.html | took 715ms
 8:18.71 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-5.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-5.html
 8:19.36 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_and_mismatch-5.html | took 651ms
 8:19.36 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-6.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-6.html
 8:20.82 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_and_mismatch-6.html | took 1463ms
 8:20.82 TEST_START: /infrastructure/reftest/reftest_match_and_mismatch-7.html
TEST-START | /infrastructure/reftest/reftest_match_and_mismatch-7.html
 8:21.24 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_and_mismatch-7.html | took 416ms
 8:21.24 TEST_START: /infrastructure/reftest/reftest_match_fail.html
TEST-START | /infrastructure/reftest/reftest_match_fail.html
 8:21.75 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_match_fail.html | took 513ms
 8:21.75 TEST_START: /infrastructure/reftest/reftest_mismatch.html
TEST-START | /infrastructure/reftest/reftest_mismatch.html
 8:22.19 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_mismatch.html | took 437ms
 8:22.19 TEST_START: /infrastructure/reftest/reftest_mismatch_fail.html
TEST-START | /infrastructure/reftest/reftest_mismatch_fail.html
 8:22.65 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_mismatch_fail.html | took 457ms
 8:22.65 TEST_START: /infrastructure/reftest/reftest_multiple_match-0.html
TEST-START | /infrastructure/reftest/reftest_multiple_match-0.html
 8:23.67 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_multiple_match-0.html | took 1022ms
 8:23.67 TEST_START: /infrastructure/reftest/reftest_multiple_match-1.html
TEST-START | /infrastructure/reftest/reftest_multiple_match-1.html
 8:26.08 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_multiple_match-1.html | took 2412ms
 8:26.08 TEST_START: /infrastructure/reftest/reftest_multiple_mismatch-0.html
TEST-START | /infrastructure/reftest/reftest_multiple_mismatch-0.html
 8:27.54 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_multiple_mismatch-0.html | took 1453ms
 8:27.54 TEST_START: /infrastructure/reftest/reftest_multiple_mismatch-1.html
TEST-START | /infrastructure/reftest/reftest_multiple_mismatch-1.html
 8:28.33 TEST_END: FAIL
TEST-FAIL | /infrastructure/reftest/reftest_multiple_mismatch-1.html | took 793ms
 8:28.33 TEST_START: /infrastructure/reftest/reftest_ref_timeout.html
TEST-START | /infrastructure/reftest/reftest_ref_timeout.html
 8:48.42 TEST_END: TIMEOUT
TEST-TIMEOUT | /infrastructure/reftest/reftest_ref_timeout.html | took 20095ms
 8:48.42 TEST_START: /infrastructure/reftest/reftest_timeout.html
TEST-START | /infrastructure/reftest/reftest_timeout.html
mem avail: 11937 of 15806 MiB (75 %), swap free:    0 of    0 MiB ( 0 %)
 9:08.52 TEST_END: TIMEOUT
TEST-TIMEOUT | /infrastructure/reftest/reftest_timeout.html | took 20096ms
 9:08.52 TEST_START: /infrastructure/reftest/reftest_wait_0.html
TEST-START | /infrastructure/reftest/reftest_wait_0.html
 9:11.39 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_wait_0.html | took 2868ms
 9:11.39 TEST_START: /infrastructure/reftest/reftest_wait_TestRendered.html
TEST-START | /infrastructure/reftest/reftest_wait_TestRendered.html
 9:12.37 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_wait_TestRendered.html | took 979ms
 9:12.37 TEST_START: /infrastructure/reftest/reftest_window_load.html
TEST-START | /infrastructure/reftest/reftest_window_load.html
 9:13.32 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/reftest_window_load.html | took 946ms
 9:13.32 TEST_START: /infrastructure/reftest/size.html
TEST-START | /infrastructure/reftest/size.html
 9:14.19 TEST_END: PASS
TEST-PASS | /infrastructure/reftest/size.html | took 872ms
 9:14.19 TEST_START: /infrastructure/reftest-wait.html
TEST-START | /infrastructure/reftest-wait.html
 9:15.94 TEST_END: PASS
TEST-PASS | /infrastructure/reftest-wait.html | took 1755ms
 9:15.95 INFO Restarting browser for new test group
Restarting browser for new test group
 9:16.15 INFO Closing logging queue
Closing logging queue
 9:16.15 INFO queue closed
queue closed
 9:16.62 INFO STDOUT: cleanup aborted: Unable to remount device
STDOUT: cleanup aborted: Unable to remount device
 9:16.78 INFO STDOUT: cleanup aborted: Unable to remount device
STDOUT: cleanup aborted: Unable to remount device
 9:16.80 INFO --certutil-binary not supplied; Firefox will not check certificates
--certutil-binary not supplied; Firefox will not check certificates
 9:17.61 web-platform-tests INFO Granting important runtime permissions to org.mozilla.geckoview.test_runner
web-platform-tests Granting important runtime permissions to org.mozilla.geckoview.test_runner
 9:18.76 web-platform-tests INFO launch_application: am start -W -n org.mozilla.geckoview.test_runner/org.mozilla.geckoview.test_runner.TestRunnerActivity -a android.intent.action.MAIN --es env0 MOZ_CRASHREPORTER=1 --es env1 MOZ_CRASHREPORTER_NO_REPORT=1 --es env2 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env3 MOZ_HIDE_RESULTS_TABLE=1 --es env4 MOZ_IN_AUTOMATION=1 --es env5 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --es env6 R_LOG_LEVEL=6 --es env7 R_LOG_DESTINATION=stderr --es env8 R_LOG_VERBOSE=1 --es env9 MOZ_PROCESS_LOG=/tmp/tmphd3zte3dpidlog --es env10 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es arg0 -no-remote --es arg1 -profile --es arg2 /data/local/tmp/test_root/profile --es arg3 --marionette --es arg4 about:blank
web-platform-tests launch_application: am start -W -n org.mozilla.geckoview.test_runner/org.mozilla.geckoview.test_runner.TestRunnerActivity -a android.intent.action.MAIN --es env0 MOZ_CRASHREPORTER=1 --es env1 MOZ_CRASHREPORTER_NO_REPORT=1 --es env2 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env3 MOZ_HIDE_RESULTS_TABLE=1 --es env4 MOZ_IN_AUTOMATION=1 --es env5 MOZ_LOG=signaling:3,mtransport:4,DataChannel:4,jsep:4 --es env6 R_LOG_LEVEL=6 --es env7 R_LOG_DESTINATION=stderr --es env8 R_LOG_VERBOSE=1 --es env9 MOZ_PROCESS_LOG=/tmp/tmphd3zte3dpidlog --es env10 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es arg0 -no-remote --es arg1 -profile --es arg2 /data/local/tmp/test_root/profile --es arg3 --marionette --es arg4 about:blank
 9:20.64 INFO Starting runner
Starting runner
 9:21.78 TEST_START: /infrastructure/crashtests/example.html
TEST-START | /infrastructure/crashtests/example.html
 9:24.31 TEST_END: PASS
TEST-PASS | /infrastructure/crashtests/example.html | took 2524ms
 9:24.31 INFO No more tests
No more tests
 9:24.45 INFO Closing logging queue
Closing logging queue
 9:24.92 INFO STDOUT: cleanup aborted: Unable to remount device
STDOUT: cleanup aborted: Unable to remount device
 9:24.45 INFO queue closed
queue closed
 9:25.13 INFO STDOUT: cleanup aborted: Unable to remount device
STDOUT: cleanup aborted: Unable to remount device
 9:25.13 SUITE_END

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 381 checks (219 subtests, 162 tests)
Expected results: 371
Skipped: 10 tests
Unexpected results: 0
OK
SUITE-END | took 427s
 9:25.13 INFO Got 0 unexpected results, with 0 unexpected passes
Got 0 unexpected results, with 0 unexpected passes
 9:25.14 wptserve INFO Stopped http server on 127.0.0.1:8446
wptserve Stopped http server on 127.0.0.1:8446
 9:25.21 wptserve INFO Stopped http server on 127.0.0.1:8003
wptserve Stopped http server on 127.0.0.1:8003
 9:25.22 wptserve INFO Stopped http server on 127.0.0.1:8001
wptserve Stopped http server on 127.0.0.1:8001
 9:25.25 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
wptserve Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8888))
 9:25.27 wptserve INFO Stopped http server on 127.0.0.1:9000
wptserve Stopped http server on 127.0.0.1:9000
 9:25.33 wptserve INFO Stopped http server on 127.0.0.1:8000
wptserve Stopped http server on 127.0.0.1:8000
 9:25.50 wptserve INFO Stopped http server on 127.0.0.1:8443
wptserve Stopped http server on 127.0.0.1:8443
 9:25.52 wptserve INFO Stopped http server on 127.0.0.1:8444
wptserve Stopped http server on 127.0.0.1:8444
 9:25.58 wptserve INFO Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
wptserve Close on: (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 8889))
 9:25.58 wptserve INFO Stopped http server on 127.0.0.1:8445
wptserve Stopped http server on 127.0.0.1:8445
 9:25.62 wptserve INFO Stopped http server on 127.0.0.1:8002
wptserve Stopped http server on 127.0.0.1:8002
 9:25.65 INFO Removed font: Ahem.ttf
Removed font: Ahem.ttf
 9:25.68 INFO Closing logging queue
Closing logging queue
 9:25.68 INFO queue closed
queue closed
[taskcluster 2024-12-24 06:22:52.197Z] === Task Finished ===
[taskcluster 2024-12-24 06:22:53.953Z] Successful task run with exit code: 0 completed in 985.989 seconds
View more details on Community-TC Integration