Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

isolated python code interpreter #619

Merged
merged 5 commits into from
Aug 14, 2024
Merged

isolated python code interpreter #619

merged 5 commits into from
Aug 14, 2024

Conversation

pelikhan
Copy link
Member

@pelikhan pelikhan commented Aug 14, 2024

  • The python_interpreter tool has been replaced with the python_code_interpreter tool specifically for data analysis tasks. 🛠️
  • The new python_code_interpreter comes with pre-installed packages - numpy, pandas, scipy - for data analysis tasks. 📊
  • It limits the functionality of the old tool, as it no longer supports network connectivity or installation of other packages. 🚧
  • The python code is now directly executed, without installing requirements from script parameters. 🏃‍♀️
  • Tool descriptions and tool reference documentation have been updated to reflect these changes. 📜
  • On a lower level, changes were made in the DockerManager class to provide the functionality to disconnect from the network. ⛔️
  • There is also an update to user-facing APIs to expose the new disconnect() method. 👥
  • The data-analyst.genai.mjs file was updated to use new python_code_interpreter instead of the old python_interpreter. 🔄
  • The container.genai.mjs file was updated to test the new disconnection functionality. 🔍

generated by pr-describe

@@ -10,7 +10,7 @@ import { LinkCard } from '@astrojs/starlight/components';
<LinkCard title="fs_read_file" description="Reads a file as text from the file system." href="/genaiscript/reference/scripts/system#systemfs_read_file" />
<LinkCard title="fs_read_summary" description="Reads a summary of a file from the file system." href="/genaiscript/reference/scripts/system#systemfs_read_summary" />
<LinkCard title="math_eval" description="Evaluates a math expression" href="/genaiscript/reference/scripts/system#systemmath" />

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link for the python_code_interpreter tool needs to be updated to reflect the new tool name.

generated by pr-docs-review-commit link_update

@@ -590,64 +590,53 @@ Emit type information compatible with PyLance.`
`````


### `system.python_interpreter`
### `system.python_code_interpreter`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header should be updated to system.python_code_interpreter to match the new tool name.

generated by pr-docs-review-commit header_update


Python Dockerized code execution
Python Dockerized code execution for data analysis



Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description of the tool should be updated to "Executes python 3.12 code for Data Analysis tasks in a docker container."

generated by pr-docs-review-commit description_update

Copy link

The changes in the pull request introduce a new disconnect function in the DockerManager class. This function iterates over all networks and disconnects the container from the "bridge" network.

Additionally, the disconnect method has been added to the ContainerHost interface in the prompt_template.d.ts file.

Overall, the changes look good. However, there are a few potential functional issues:

  1. The disconnect method disconnects the container from the "bridge" network regardless of whether the container should actually be disconnected from it. It might be better to have a check or condition to determine whether the container should be disconnected.

  2. Also, error handling seems to be missing in the disconnect function. What happens if n.disconnect({ Container: container.id }) throws an error? It would be prudent to wrap this in a try-catch block to handle potential errors gracefully.

  3. There is no check if the network is already disconnected. It would be more efficient to check if the network is connected before attempting to disconnect it.

Unfortunately, I can't provide a diff as these are logical considerations and would need further information to propose code changes.

For these reasons, I would suggest revising the changes before merging. 🚀

generated by pr-review

})

const image = env.vars.pythonImage ?? "python:3.12"
const packages = ["numpy", "pandas", "scipy"]

let container = null

defTool(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tool name in the definition should be updated to python_code_interpreter.

generated by pr-docs-review-commit tool_name_update

<package1>
<package2>
`,
},
main: {
type: "string",
description: "python 3.12 source code to execute",
},
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requirements property has been removed and should be updated in the documentation.

generated by pr-docs-review-commit property_removal

main: {
type: "string",
description: "python 3.12 source code to execute",
},
},
required: ["requirements", "main"],
required: ["main"],

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The required array should be updated to only include main since requirements has been removed.

generated by pr-docs-review-commit required_update

])
if (res.failed) throw new Error(`Failed to install requirements`)
await container.disconnect()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section handling the installation of requirements has been removed and should be updated in the documentation.

generated by pr-docs-review-commit installation_removal

])
if (res.failed) throw new Error(`Failed to install requirements`)
await container.disconnect()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The addition of await container.disconnect() should be documented.

generated by pr-docs-review-commit disconnect_addition

})

const image = env.vars.pythonImage ?? "python:3.12"
const packages = ["numpy", "pandas", "scipy"]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The declaration of the packages constant should be documented, explaining that only numpy, pandas, and scipy are available and no additional packages can be installed.

generated by pr-docs-review-commit packages_declaration

"python_interpreter",
"Executes python 3.12 code in a docker container. The process output is returned. Use 'print' to output data.",
"python_code_interpreter",
"Executes python 3.12 code for Data Analysis tasks in a docker container. The process output is returned. Do not generate visualizations. The only packages available are numpy, pandas, scipy. There is NO network connectivity. Do not attempt to install other packages or make web requests.",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tool description should be updated to match the new functionality and limitations of the python_code_interpreter.

generated by pr-docs-review-commit tool_description_update

<package1>
<package2>
`,
},
main: {
type: "string",
description: "python 3.12 source code to execute",
},
},

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requirements field is no longer required and should be removed from the required fields list.

generated by pr-docs-review-commit required_fields_update

main: {
type: "string",
description: "python 3.12 source code to execute",
},
},
required: ["requirements", "main"],
required: ["main"],
},
async (args) => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The handling of the requirements field should be removed as it is no longer part of the tool's functionality.

generated by pr-docs-review-commit remove_requirements_handling

])
if (res.failed) throw new Error(`Failed to install requirements`)
await container.disconnect()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The call to container.disconnect() should be removed as it may interfere with the execution of the Python code in the container. Containers are typically disconnected after the execution is complete, not before.

generated by pr-docs-review-commit disconnect_removal

@@ -309,6 +325,7 @@ export class DockerManager {
writeText,
readText,
copyTo,
disconnect,
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disconnect function is declared but not used within the DockerManager class. If this function is not needed, consider removing it to avoid confusion. If it is needed in the future, it can be added back. 😊

generated by pr-review-commit disconnect_unused

/**
* Force disconnect network
*/
disconnect(): Promise<void>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The disconnect function in the ContainerHost interface is missing a detailed description. Consider adding more information about what the function does, what parameters it takes (if any), and what it returns. This will help other developers understand its purpose more easily. 😊

generated by pr-review-commit missing_function_description

@pelikhan pelikhan merged commit 4489368 into main Aug 14, 2024
8 of 10 checks passed
@pelikhan pelikhan deleted the code-interpreter branch August 14, 2024 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant