Skip to content

Nightly fast dispatch tests #5685

Nightly fast dispatch tests

Nightly fast dispatch tests #5685

Triggered via schedule November 10, 2024 10:00
Status Failure
Total duration 1h 27m 33s
Artifacts 23
build-artifact  /  ...  /  build-docker-image
34s
build-artifact / build-docker-image / build-docker-image
Matrix: build-artifact / build-artifact
Matrix: fd-nightly / fd-nightly
Matrix: fd-nightly / nightly-wh-models
Fit to window
Zoom out
Zoom in

Annotations

7 errors, 3 warnings, and 140 notices
fd-nightly / Nightly N300 common_models
Process completed with exit code 1.
fd-nightly / Nightly N150 mamba
The action 'Run frequent reg tests scripts' has timed out after 50 minutes.
pcie-cards-are-being-used-cleanup
Tenstorrent cards seem to be in use. Killing PIDs and exiting unsuccessfully. This can happen if a test hung and is normally an issue with the test, rather than infra.
fd-nightly / Nightly N150 mamba
Process completed with exit code 1.
fd-nightly / Nightly N300 mamba
The self-hosted runner: tt-metal-ci-vm-87 lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.
pcie-cards-are-being-used-cleanup
Tenstorrent cards seem to be in use. Killing PIDs and exiting unsuccessfully. This can happen if a test hung and is normally an issue with the test, rather than infra.
fd-nightly / Nightly N300 mamba
The action 'Run frequent reg tests scripts' has timed out after 50 minutes.
fd-nightly / FD WH N300 pgm dispatch nightly wormhole_b0
No files were found with the provided path: generated/test_reports/. No artifacts will be uploaded.
fd-nightly / Nightly N150 mamba
No files were found with the provided path: generated/test_reports/. No artifacts will be uploaded.
unsuccessful-reset-attempt-cleanup
Unsuccessful board reset, trying again in 1 minute ...
disk-usage-before-startup
Disk usage is 84 %
disk-usage-after-startup
Disk usage is 84 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 86 %
disk-usage-after-startup
Disk usage is 86 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 83 %
disk-usage-after-startup
Disk usage is 83 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 83 %
disk-usage-after-startup
Disk usage is 83 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 83 %
disk-usage-after-startup
Disk usage is 83 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 78 %
disk-usage-after-startup
Disk usage is 78 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 70 %
disk-usage-after-startup
Disk usage is 70 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 85 %
disk-usage-after-startup
Disk usage is 85 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 89 %
disk-usage-after-startup
Disk usage is 89 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 85 %
disk-usage-after-startup
Disk usage is 85 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 78 %
disk-usage-after-startup
Disk usage is 78 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 83 %
disk-usage-after-startup
Disk usage is 83 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 73 %
disk-usage-after-startup
Disk usage is 73 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 87 %
disk-usage-after-startup
Disk usage is 87 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 68 %
disk-usage-after-startup
Disk usage is 68 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 68 %
disk-usage-after-startup
Disk usage is 68 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 79 %
disk-usage-after-startup
Disk usage is 79 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 84 %
disk-usage-after-startup
Disk usage is 84 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
disk-usage-before-startup
Disk usage is 83 %
disk-usage-after-startup
Disk usage is 83 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
attempting-reset-cleanup
Attempting to reset card(s).
successful-reset-cleanup
tt-smi reset was successful
reset-successful-cleanup
tt-smi reset was successful
printing-out-smi-info-cleanup
Touching and printing out SMI info
attempting-reset-cleanup
Attempting to reset card(s).
disk-usage-before-startup
Disk usage is 90 %
disk-usage-after-startup
Disk usage is 90 %
printing-smi-info-startup
Touching and printing out SMI info
reset-successful-startup
tt-smi reset was successful

Artifacts

Produced during runtime
Name Size
TTMetal_build_grayskull
51.1 MB
TTMetal_build_wormhole_b0
51.4 MB
test_reports_060b250c-9fe9-4320-bc15-14f8dbc8eee2
618 Bytes
test_reports_1135c3c5-f989-4eb4-8aa1-8428c82402e0
561 Bytes
test_reports_2491d73c-4dfd-4be2-9862-b50e2cb9dc0a
606 Bytes
test_reports_2701fa2d-c063-4a4b-a37d-e949ab139e68
623 Bytes
test_reports_4052170f-5e29-4d00-963c-698e7c148987
6.39 KB
test_reports_5ff9a205-1f09-4317-956a-176da03de39b
479 Bytes
test_reports_76d3b61d-5157-4da0-b18d-4b3b7df931eb
431 Bytes
test_reports_7a5b82b7-29c2-45ff-9072-2719fdd21d3b
497 Bytes
test_reports_7c353f81-26b9-4f9a-a503-d647a1c9059a
1.26 KB
test_reports_90bcb14b-b8ba-4781-a2ca-7e2a26b24c93
5.89 KB
test_reports_9461e149-39a4-4c5d-a8b2-ccc054440f56
4.6 KB
test_reports_98f68b06-9e02-4399-bae5-76f3d1deaca8
1.32 KB
test_reports_9c4b84bc-2bd9-44ea-81d4-19772aa10738
611 Bytes
test_reports_aa405b4e-9393-426c-81ce-634abdf8d31b
552 Bytes
test_reports_ab131473-8b71-46f5-a0f7-c2b6763ef1c0
1.13 KB
test_reports_b6c565f8-1af6-4e7a-abf0-28da87134bca
480 Bytes
test_reports_c7e86169-5e97-4bbb-8d88-1567b6c82a05
430 Bytes
test_reports_d8241c3f-f01f-41b4-b9e1-bc634466bfea
4.66 KB
test_reports_e2129a72-c729-4848-acd6-041db1f2cb08
5.87 KB
test_reports_f741458a-ea18-4552-bd30-55d04a1309ae
1.36 KB
test_reports_f93bcfff-7793-41ad-b39c-a634500a4c0d
1.04 KB