Skip to content

Commit

Permalink
run precommit
Browse files Browse the repository at this point in the history
  • Loading branch information
CalebCourier committed Nov 22, 2024
1 parent 7d20c24 commit 346684a
Show file tree
Hide file tree
Showing 4 changed files with 95 additions and 120 deletions.
127 changes: 47 additions & 80 deletions docs/user_guides/guardrails-ai/guard-as-action/README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,17 @@
# Guard as Actions

This guide will teach you how to use a `Guard` with any of the 60+ GuardrailsAI Validators as an action inside a guardrails configuration.


```python
# Init: remove any existing configuration
! rm -r config
! mkdir config
```
This guide will teach you how to use a `Guard` with any of the 60+ GuardrailsAI Validators as an action inside a guardrails configuration.

## Prerequisites

We'll be using an OpenAI model for our LLM in this guide, so set up an OpenAI API key, if not already set.


```python
! export OPENAI_API_KEY=$OPENAI_API_KEY # Replace with your own key
```bash
export OPENAI_API_KEY=$OPENAI_API_KEY # Replace with your own key
```

If you're running this inside a notebook, you also need to patch the AsyncIO loop.


```python
import nest_asyncio

Expand All @@ -31,29 +22,25 @@ nest_asyncio.apply()

Let's create a sample Guard that can detect PII. First, install guardrails-ai.


```python
! pip install guardrails-ai -q
```bash
pip install guardrails-ai -q
```

Next configure the guardrails cli so we can install the validator we want to use from the Guardrails Hub.


```python
! guardrails configure
```bash
guardrails configure
```


```python
! guardrails hub install hub://guardrails/detect_pii --no-install-local-models -q
```bash
guardrails hub install hub://guardrails/detect_pii --no-install-local-models -q
```

Now we can define our Guard.
This Guard will use the DetectPII validator to safeguard against leaking personally identifiable information such as names, email addresses, etc..

Once the Guard is defined, we can test it with a static value to make sure it's working how we would expect.


```python
from guardrails import Guard
from guardrails.hub import DetectPII
Expand All @@ -63,51 +50,42 @@ g = Guard(name="pii_guard").use(DetectPII(["PERSON", "EMAIL_ADDRESS"], on_fail="
print(g.validate("My name is John Doe"))
```

ValidationOutcome(
call_id='14534730096',
raw_llm_output='My name is John Doe',
validation_summaries=[
ValidationSummary(
validator_name='DetectPII',
validator_status='fail',
property_path='$',
failure_reason='The following text in your response contains PII:\nMy name is John Doe',
error_spans=[
ErrorSpan(start=11, end=19, reason='PII detected in John Doe')
]
)
],
validated_output='My name is <PERSON>',
reask=None,
validation_passed=True,
error=None
)


## Guardrails Configuration

Now we'll use the Guard we defeined above to create an action and a flow. Since we're calling our guard "pii_guard", we'll use "pii_guard_validate" in order to see if the LLM output is safe.

```
ValidationOutcome(
call_id='14534730096',
raw_llm_output='My name is John Doe',
validation_summaries=[
ValidationSummary(
validator_name='DetectPII',
validator_status='fail',
property_path='$',
failure_reason='The following text in your response contains PII:\nMy name is John Doe',
error_spans=[
ErrorSpan(start=11, end=19, reason='PII detected in John Doe')
]
)
],
validated_output='My name is <PERSON>',
reask=None,
validation_passed=True,
error=None
)
```

```python
%%writefile config/rails.co
## Guardrails Configuration

Now we'll use the Guard we defeined above to create an action and a flow. Since we're calling our guard "pii_guard", we'll use "pii_guard_validate" in order to see if the LLM output is safe.

```colang
define flow detect_pii
$output = execute pii_guard_validate(text=$bot_message)
if not $output
bot refuse to respond
stop

```

Writing config/rails.co



```python
%%writefile config/config.yml
```yaml
models:
- type: main
engine: openai
Expand All @@ -119,16 +97,12 @@ rails:
- detect_pii
```
Writing config/config.yml


To hook the Guardrails AI guard up so that it can be read from Colang, we use the integration's `register_guardrails_guard_actions` function.
This takes a name and registers two actions:

1. [guard_name]_validate: This action is used to detect validation failures in outputs
2. [guard name]_fix: This action is used to automatically fix validation failures in outputs, when possible


```python
from nemoguardrails import RailsConfig, LLMRails
from nemoguardrails.integrations.guardrails_ai.guard_actions import register_guardrails_guard_actions
Expand All @@ -139,29 +113,26 @@ rails = LLMRails(config)
register_guardrails_guard_actions(rails, g, "pii_guard")
```

Fetching 5 files: 100%|██████████| 5/5 [00:00<00:00, 109226.67it/s]

```
Fetching 5 files: 100%|██████████| 5/5 [00:00<00:00, 109226.67it/s]
```
## Testing
Let's try this out. If we invoke the guardrails configuration with a message that prompts the LLM to return personal information like names, email addresses, etc., it should refuse to respond.

```python
response = rails.generate("Who is the current president of the United States, and what was their email address?")
print(response)
```

I'm sorry, I can't respond to that.

```
I'm sorry, I can't respond to that.
```

Great! So the valdiation-only flow works. Next let's try the fix flow.


```python
%%writefile config/rails.co


```colang
define flow detect_pii
$output = execute pii_guard_fix(text=$bot_message)
Expand All @@ -170,15 +141,10 @@ define flow detect_pii
stop
else
$bot_message = $output

```

Overwriting config/rails.co


If we send the same message, we should get a response this time, but any PII will be filtered out.


```python
config = RailsConfig.from_path("./config")
rails = LLMRails(config)
Expand All @@ -189,16 +155,17 @@ response = rails.generate("Who is the current president of the United States, an
print(response)
```

The current president of the United States is <PERSON>. His official email address is <EMAIL_ADDRESS>. However, he also has a personal email address, which is <EMAIL_ADDRESS>.

```
The current president of the United States is <PERSON>. His official email address is <EMAIL_ADDRESS>. However, he also has a personal email address, which is <EMAIL_ADDRESS>.
```

If however, we prompt the LLM with a message that does not cause it to return PII, we should get the unaltered response.


```python
response = rails.generate("Hello!")
print(response)
```

Hello there! How can I assist you?

```
Hello there! How can I assist you?
```
53 changes: 16 additions & 37 deletions docs/user_guides/guardrails-ai/rails-as-guard/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,16 @@
# Guardrails as Guards
This guide will teach you how to add NeMo Guardrails to a GuardrailsAI Guard.


```python
# Init: remove any existing configuration
! rm -r config
! mkdir config
```

## Prerequisites

We'll be using an OpenAI model for our LLM in this guide, so set up an OpenAI API key, if not already set.


```python
! export OPENAI_API_KEY=$OPENAI_API_KEY # Replace with your own key
```bash
export OPENAI_API_KEY=$OPENAI_API_KEY # Replace with your own key
```

If you're running this inside a notebook, you also need to patch the AsyncIO loop.


```python
import nest_asyncio

Expand All @@ -29,21 +20,15 @@ nest_asyncio.apply()
## Sample Guardrails
We'll start by creating a new guardrails configuration.


```python
%%writefile config/config.yml
```yaml
models:
- type: main
engine: openai
model: gpt-3.5-turbo-instruct
```
Writing config/config.yml


We'll do a quick test to make sure everything is working as expected.

```python
from nemoguardrails import RailsConfig, LLMRails

Expand All @@ -55,50 +40,43 @@ response = rails.generate("Hello!")
print(response)
```


```
Fetching 5 files: 0%| | 0/5 [00:00<?, ?it/s]

Hi there! How can I assist you today?

```

That worked! Now let's install a validator from the GuardrailsAI Hub to augment our guardrails configuration from above.

If you haven't already, install and configure guardrails-ai before trying to install the DetectPII validator.


```python
! pip install guardrails-ai
! guardrails configure
```bash
pip install guardrails-ai
guardrails configure
```


```python
! guardrails hub install hub://guardrails/detect_pii --no-install-local-models
```bash
guardrails hub install hub://guardrails/detect_pii --no-install-local-models
```

Now we can use the rails defined earlier as the basis for our Guard. We'll also attach the DetectPII validator as an additional measure.


```python
from guardrails.integrations.nemoguardrails import NemoguardrailsGuard
from guardrails.hub import DetectPII


guard = NemoguardrailsGuard(rails)
guard.use(DetectPII(
pii_entities=["PERSON", "EMAIL_ADDRESS"],
on_fail="fix"
))

```

## Testing
With everything configured, we can test out our new Guard!

Let's invoke the Guard with a message that prompts the LLM to return personal information like names, email addresses, etc.. Since we specified `on_fail="fix"` in the DetectPII validator, the response should have any PII filtered out.


```python
response = guard(
messages=[{
Expand All @@ -110,12 +88,12 @@ response = guard(
print(response.validated_output)
```

The current president of the United States is <PERSON>. His email address is <EMAIL_ADDRESS>. He can also be reached through his personal email at <EMAIL_ADDRESS>. Additionally, he is active on social media and can be contacted through his official Twitter account <PERSON>. Is there anything else you would like to know about President <PERSON>?

```
The current president of the United States is <PERSON>. His email address is <EMAIL_ADDRESS>. He can also be reached through his personal email at <EMAIL_ADDRESS>. Additionally, he is active on social media and can be contacted through his official Twitter account <PERSON>. Is there anything else you would like to know about President <PERSON>?
```

Great! We can see that the Guard called the LLM configured in the LLMRails, validated the output, and filtered it accordingly. If however, we prompt the LLM with a message that does not cause it to return PII, we should get the unaltered response.


```python
response = guard(
messages=[{
Expand All @@ -127,5 +105,6 @@ response = guard(
print(response.validated_output)
```

Hi there! It's nice to meet you. My name is AI Assistant. How can I help you today?

```
Hi there! It's nice to meet you. My name is AI Assistant. How can I help you today?
```
14 changes: 14 additions & 0 deletions nemoguardrails/integrations/guardrails_ai/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Loading

0 comments on commit 346684a

Please sign in to comment.