Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add proof runtime to GITHUB_STEP_SUMMARY #221

Open
hanno-becker opened this issue Nov 6, 2024 · 0 comments
Open

Add proof runtime to GITHUB_STEP_SUMMARY #221

hanno-becker opened this issue Nov 6, 2024 · 0 comments

Comments

@hanno-becker
Copy link

run-cbmc-proofs.py supports adding a summary table to GITHUB_STEP_SUMMARY, which is convenient for Github CI's.

Currently, this table has only two columns: Proof name and proof status.

To help identify long-running proofs, it would be useful to log also the runtime of the proofs in that table, either unconditionally or via another parameter to run-cbmc-proofs.py.

Here is a sketch that adds a column with the cumulative runtime of all jobs run for a proof:

--- a/cbmc/proofs/lib/summarize.py
+++ b/cbmc/proofs/lib/summarize.py
@@ -84,7 +84,7 @@ def _get_status_and_proof_summaries(run_dict):
     The second sub-list maps each proof to its status.
     """
     count_statuses = {}
-    proofs = [["Proof", "Status"]]
+    proofs = [["Proof", "Status", "Duration (in s)"]]
     for proof_pipeline in run_dict["pipelines"]:
         status_pretty_name = proof_pipeline["status"].title().replace("_", " ")
         try:
@@ -93,7 +93,11 @@ def _get_status_and_proof_summaries(run_dict):
             count_statuses[status_pretty_name] = 1
         if proof_pipeline["name"] == "print_tool_versions":
             continue
-        proofs.append([proof_pipeline["name"], status_pretty_name])
+        duration = 0
+        for stage in proof_pipeline["ci_stages"]:
+            for job in stage["jobs"]:
+                duration += int(job["duration"])
+        proofs.append([proof_pipeline["name"], status_pretty_name, str(duration)])
     statuses = [["Status", "Count"]]
     for status, count in count_statuses.items():
         statuses.append([status, str(count)])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant