Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functionality to diff GitHub Actions job logs and update related documentation and tool definitions #748

Merged
merged 5 commits into from
Oct 2, 2024

Conversation

pelikhan
Copy link
Member

@pelikhan pelikhan commented Oct 2, 2024

This pull request adds functionality to diff GitHub Actions job logs. It includes updates to the documentation and tool definitions to support this new feature. The changes include adding a new method diffWorkflowJobLogs to the GitHub interface, as well as updating the defTool definitions for github_actions_job_log_get and github_actions_job_log_diff. This new functionality allows users to compare the logs of two different workflow jobs and see the differences between them.


  • The github_actions_job_log function in the system.github_actions tool is now renamed to github_actions_job_log_get and its associated description is updated accordingly. The name change makes the function intention clearer. 👌

  • Introduced a new function (github_actions_job_log_diff) in the system.github_actions tool. This function does the heavy lifting of diff'ing 2 workflow job logs. Given two job ids, this function downloads the corresponding logs and generates the diff. 🔍

  • Added respective function descriptions, tool updates, and parameters in the tools' markdown files (components/BuiltinTools.mdx and reference/scripts/system.mdx). 💼📚

  • Added the newly introduced function diffWorkflowJobLogs() in the GitHubClient class in github.ts. This function internally calls private function downladJob() to download individual job logs, processes them accordingly and generates the diff. 📂

  • Updated the query constraints in the agent_github.genai.mjs file. It now advises consumers to prefer diffing job logs rather than downloading entire logs which can be very large. This promotes efficient usage of resources. 🛠️

  • Lastly, updated the function signature of diffWorkflowJobLogs() in prompt_template.d.ts, formally introducing this new function in the public API. 🌐

generated by pr-describe

@@ -20,7 +20,8 @@ import { LinkCard } from '@astrojs/starlight/components';
<LinkCard title="github_actions_workflows_list" description="List all workflows as a list of 'id: name' pair." href="/genaiscript/reference/scripts/system#systemgithub_actions" />
<LinkCard title="github_actions_runs_list" description="List all runs for a workflow. Use 'git_actions_list_workflows' to list workflows." href="/genaiscript/reference/scripts/system#systemgithub_actions" />
<LinkCard title="github_actions_jobs_list" description="List all jobs for a run." href="/genaiscript/reference/scripts/system#systemgithub_actions" />
<LinkCard title="github_actions_job_log" description="Download job log." href="/genaiscript/reference/scripts/system#systemgithub_actions" />
<LinkCard title="github_actions_job_log_get" description="Download workflow job log." href="/genaiscript/reference/scripts/system#systemgithub_actions" />
<LinkCard title="github_actions_job_log_diff" description="Diffs two workflow job logs." href="/genaiscript/reference/scripts/system#systemgithub_actions" />
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tool name 'github_actions_job_log_diff' should be 'github_actions_job_logs_diff' to maintain consistency in naming convention.

generated by pr-docs-review-commit incorrect_tool_name

@@ -796,7 +803,8 @@ Queries results from workflows in GitHub actions.
- tool `github_actions_workflows_list`: List all workflows as a list of 'id: name' pair.
- tool `github_actions_runs_list`: List all runs for a workflow. Use 'git_actions_list_workflows' to list workflows.
- tool `github_actions_jobs_list`: List all jobs for a run.
- tool `github_actions_job_log`: Download job log.
- tool `github_actions_job_log_get`: Download workflow job log.
- tool `github_actions_job_log_diff`: Diffs two workflow job logs.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tool name 'github_actions_job_log_diff' should be 'github_actions_job_logs_diff' to maintain consistency in naming convention.

generated by pr-docs-review-commit incorrect_tool_name

Copy link

github-actions bot commented Oct 2, 2024

The changes in GIT_DIFF introduce a new functionality to the GitHubClient class. The functionality is to compare the logs of two different GitHub Action workflow jobs.

Here are the added functions and their descriptions:

  • downladJob(job_id: number): Downloads the job logs of a given job ID. The function first prepares the client, owner, and repo information, then uses the GitHub REST API method downloadJobLogsForWorkflowRun to get the URL of the job logs, and finally fetches the content of the job logs from the URL.

  • diffWorkflowJobLogs(job_id: number, other_job_id: number): Compares the logs of two jobs with the given job IDs. The function first downloads the job logs of the two jobs, parses the logs, creates a diff of the two logs, and returns a diff in llmified format.

The changes also update prompt_template.d.ts to add the type definition of the new diffWorkflowJobLogs method.

The changes should not introduce any breaking changes as they only add new functions.

However, there's a typo in the method name downladJob which should be downloadJob. Additionally, the method diffWorkflowJobLogs doesn't seem to handle error cases, e.g., if the job logs can't be downloaded, parsed, or diffed. You might want to add error handling to improve reliability.

Suggested changes:

-    private async downladJob(job_id: number) {
+    private async downloadJob(job_id: number) {
        ...
    }

    async diffWorkflowJobLogs(job_id: number, other_job_id: number) {
+       try {
            ...
+       } catch (error) {
+           console.error('Error diffing job logs:', error);
+           throw error;
+       }
    }

generated by pr-review

Copy link

github-actions bot commented Oct 2, 2024

Investigator report

Root Cause Analysis

The failure in the second run is caused by a TypeScript compilation error. The error message indicates:

##[error]genaisrc/poem.genai.mts(2,1): error TS1434: Unexpected keyword or identifier.
##[error]genaisrc/poem.genai.mts(2,3): error TS1434: Unexpected keyword or identifier.

This suggests there are syntax errors or unexpected tokens in the genaisrc/poem.genai.mts file starting at line 2.

Suggested Fix

  1. Review the genaisrc/poem.genai.mts file for syntax errors around line 2.
  2. Ensure TypeScript syntax rules are being followed correctly.

Without access to the actual file's content, it's difficult to specify the exact fix, but the focus should be on correcting the unexpected keywords or identifiers at the specified location.

generated by gai

Copy link

github-actions bot commented Oct 2, 2024

No changes were detected in the genaisrc/poem.genai.mts file between the latest successful run and the failed run based on the git diff.

The issue might be related to an environment change or an external dependency update. Further investigation would require checking those aspects.

generated by github-agent

@pelikhan pelikhan merged commit 78ef570 into main Oct 2, 2024
11 checks passed
@pelikhan pelikhan deleted the diff-job-logs branch October 2, 2024 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant