Guard as action #810
Guard as action #810
Conversation
|
@Pouyanpi, this is one half of the upcoming integration with Guardrails AI -- they've got the other half on their side. |
|
|
||
| def register_guardrails_guard_actions(rails: LLMRails, guard: Guard, guard_name: str): | ||
| def fix_action(text, metadata={}): | ||
| return guard.validate(llm_output=text, metadata=metadata).validated_output |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
Hi @zsimjee, is your PR ready for review? If it is, please mark it as ready for review. Thanks 👍 |
There was a problem hiding this comment.
Please make necessary adjustments and move this to https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/docs/user_guides/community
And you can add a brief intro to it similar to the other community integerations at https://github.com/NVIDIA/NeMo-Guardrails/blob/develop/docs/user_guides/guardrails-library.md
There was a problem hiding this comment.
Notebooks should be placed at https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/examples/notebooks
There was a problem hiding this comment.
I need to review it more closely, but this is not consistent with other community integrations. Please have a look at https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/nemoguardrails/library
There was a problem hiding this comment.
This was built in the model of the langchain integration, not for a single guardrail. It's a framework level integration that gives nemoguardrails users access to 60+ validators available on the hub and the features , not a singular type of guardrail.
pyproject.toml
Outdated
There was a problem hiding this comment.
pyproject.toml will only include the core dependencies. Other integerations have either instructed the user in docs on what are the requirements or have included a
requirements.txt at https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/nemoguardrails/library
I can also suggest you to use a lazy import and raise an error to install the package with the desired constraint. Deffered imports delay ImportError or ModuleNotFoundError exceptions for optional dependencies until the specific functionality is invoked, and you can instruct the user to install the package, look how it is used in the package 
Signed-off-by: Zayd <[email protected]>
| ``` | ||
|
|
||
| Now we can define our Guard. | ||
| This Guard will use the DetectPII validator to safeguard against leaking personally identifiable information such as names, email addresses, etc.. |
There was a problem hiding this comment.
| This Guard will use the DetectPII validator to safeguard against leaking personally identifiable information such as names, email addresses, etc.. | |
| This Guard will use the `DetectPII` validator to safeguard against leaking personally identifiable information such as names, email addresses, etc.. |
| Next configure the guardrails cli so we can install the validator we want to use from the Guardrails Hub. | ||
|
|
||
| ```bash | ||
| guardrails configure |
There was a problem hiding this comment.
once guardrails cli is used the user is prompted:
guardrails configure
Enable anonymous metrics reporting? [Y/n]:
Do you wish to use remote inferencing? [Y/n]:
Enter API Key below 👉 You can find your API Key at https://hub.guardrailsai.com/keys
API Key:
|
|
||
| ```colang | ||
| define flow detect_pii | ||
| $output = execute pii_guard_validate(text=$bot_message) | ||
|
|
||
| if not $output | ||
| bot refuse to respond | ||
| stop | ||
| ``` | ||
|
|
||
| ```yaml | ||
| models: | ||
| - type: main | ||
| engine: openai | ||
| model: gpt-3.5-turbo-instruct | ||
|
|
||
| rails: | ||
| output: | ||
| flows: | ||
| - detect_pii | ||
| ``` | ||
|
|
There was a problem hiding this comment.
we need to create a config directory named guardrailsai and decide where to place it. It can be in the user-guides similar to directory at examples/configs and include a config.yml and flows.co file.
examples/configs/guardrailsai
├── config.yml
└── flows.cothen adapt the instructions to use the correct path. See other docs at user-guides for reference (https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/docs/user-guides/llm/nvidia-ai-endpoints)
|
Hi @zsimjee, thank you for making further improvements. I think there are still gaps that I will comment later. I believe the way that the integration is being done is not consistent, and we should avoid breaking consistency. I still think my previous comments are valid and I think the best way to integrate Guardrails AI is similar to I will provide suggestions later, but for example following incompatibality needs to be addressed with current implementation: nemoguardrails chat --config=./examples/configs/guardrailsai
Starting the chat (Press Ctrl + C twice to quit) ...
Fetching 5 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 18724.57it/s]
> Who is the current president of the United States, and what was their email address?
Action 'pii_guard_validate' not found.In the meantime, please rebase the branch to develop. Note that the |
Pouyanpi
added
status: in review
status: in progress
Description
This PR contains helper functions and docs that show nemoguardrails users how to use Guardrails AI-style validation in their nemoguardrails code flow.
Related Issue(s)
N/A
Checklist