Skip to content

Commit

Permalink
Improve docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@dluc
dluc committed Aug 29, 2023
1 parent 90605de commit 9fc27ed
Show file tree
Hide file tree
Showing 2 changed files with 164 additions and 76 deletions.
100 changes: 63 additions & 37 deletions docs/FAQ.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,35 @@
There are two main modalities, **As a Service** and **Serverless**, plus
customizations you can apply.

Running Semantic Memory as a Service allows you to **interact with the memory
via HTTP, in any language**. The repo contains a Memory Web client written in
C# and some examples showing how to do the same from command line with `curl`.
We will provide soon Web clients written in other languages. Semantic Memory
Service is designed to run as an internal service, behind your backend,
similarly to a DB, so you should not expose the service to public traffic
without having your authentication in front, similar to how design a typical
backend integrated with a SQL server, Service Bus, etc.
One important benefit of the service, it's designed to scale horizontally and
with **durable queues for long-running operations**.
See the [service documentation](../dotnet/Service/README.md) for more details.
[Here](../examples/002-dotnet-WebClient/README.md) you can find an example
showing the web client.

Alternatively, you can **embed Semantic Memory directly into your C#
applications**, using the **Serverless Memory client**. This is limited to
.NET applications and doesn't allow mixing .NET pipelines with other languages,
e.g. pipeline handlers written in Python or TypeScript. The serverless approach
can be very useful for console applications, tests and demos. **The API is the
same** API offered by the service, so it's possible to switch from Service to
Serverless changing only the memory client instance.
[Here](../examples/001-dotnet-Serverless/README.md) you can find an example
showing the serverless client.
1. Running Semantic Memory as a Service allows you to **interact with the memory
via HTTP, in any language**. The repo contains a Memory Web client for .NET
and some examples showing how to do the same from command line with `curl`.
We will provide soon Web clients written in other languages.

Semantic Memory Service is designed to run as an internal service, behind
your backend, similarly to a DB, so you should not expose the service to
public traffic without authenticating your users first, similar to a typical
backend integrated with a SQL server, Service Bus, etc.

One important benefit of the service, the solution can scale horizontally
and can support long running operation reliably using durable queues.

For more details, see the [service documentation](../dotnet/Service/README.md).

[Here](../examples/002-dotnet-WebClient/README.md) you can find an example
showing the web client interacting with the service.

2. Alternatively, you can **embed Semantic Memory directly into your .NET
applications**, using the **Serverless Memory client**. This is limited to
.NET applications and doesn't allow mixing .NET pipelines with pipeline
handlers written in Python or TypeScript. The serverless approach can be
very useful to create console applications, run tests and demos.

**The serverless memory API is the same** offered by the service, so it's
possible to switch from Service to Serverless changing only few lines code.

[Here](../examples/001-dotnet-Serverless/README.md) you can find an example
showing the serverless client.

![image](https://github.com/microsoft/semantic-memory/assets/371009/83d6487f-75f2-42d9-9ab5-ea6aed65231b)

Expand All @@ -36,25 +42,35 @@ showing the serverless client.
In order to protect users data, you should follow these design principles:

* Use Semantic Memory as **a private backend component**, similar to a SQL
Server, without granting direct access.
* Authenticate your users in your backend using a secure solution like Azure
Server, without granting direct access. When using Semantic Memory as a
service, consider assigning the service a reserved IP, accessible only to
your IP, and using HTTPS only.
* Authenticate users in your backend using a secure solution like Azure
Active Directory, extract the user ID from the signed credentials like JWT
tokens or client certs.
* **Use Semantic Memory Tags as Security Filters**. See
[Security Filters](SECURITY_FILTERS.md) for more details.
tokens or client certs, and tag every interaction with Semantic Memory with
this User ID
* **Use Semantic Memory Tags as Security Filters**. Make sure every API call
to Semantic Memory uses a User tag, both when reading and writing to memory.
See [Security Filters](SECURITY_FILTERS.md) for more details.

![Network diagram](network.png)

![image](https://github.com/microsoft/semantic-memory/assets/371009/83d6487f-75f2-42d9-9ab5-ea6aed65231b)

### Is it possible to download web pages and turn the content into memory? Can I ask questions about the content of a web page?

Yes, the memory API includes a `ImportWebPageAsync` method that can be used
Yes, the memory API includes an `ImportWebPageAsync` method that can be used
to take a web page content, and process the text content like files. Once
the content is imported, asking questions is very simple:

```csharp
var docId = await memory.ImportWebPageAsync("https://raw.githubusercontent.com/microsoft/semantic-memory/main/README.md");
// Import memories from a web page
var docId = await memory.ImportWebPageAsync(
"https://raw.githubusercontent.com/microsoft/semantic-memory/main/README.md");

var answer = await memory.AskAsync("Where can I store my semantic memory records?", MemoryFilters.ByDocument(docId));
// Answer questions using the page content to ground the answer
var answer = await memory.AskAsync("Where can I store my semantic memory records?",
MemoryFilters.ByDocument(docId));
```

![image](https://github.com/microsoft/semantic-memory/assets/371009/83d6487f-75f2-42d9-9ab5-ea6aed65231b)
Expand All @@ -63,9 +79,12 @@ var answer = await memory.AskAsync("Where can I store my semantic memory records

When uploading a file (or multiple files), you can specify a document ID,
or you can let the service generate a document ID for you. You will see these
Document IDs also when getting answers. When sending a question, it's
possible to **include a filter**, so it's possible to filter by tags and
**by document ID**. Here's an example:
Document IDs also when getting answers.

When sending a question, it's possible to **include a filter**, so it's possible
to filter by tags and **by document ID**.

Here's an example:

```csharp
string docId = await memory.ImportDocumentAsync("manual.pdf");
Expand All @@ -80,12 +99,19 @@ can save and use for questions.
In the second example ("book.docx"), the document ID is fixed, chosen by the
client.

And this is the code showing how to ask a questions using only a specific document:
And this is the code showing how to ask a questions using only a specific
document:

```csharp
var answer1 = await memory.AskAsync("What's the produc name?", MemoryFilters.ByDocument(docId));
var answer1 = await memory.AskAsync("What's the product name?",
MemoryFilters.ByDocument(docId));

var answer2 = await memory.AskAsync("What's the total population?", MemoryFilters.ByDocument("europe001"));
var answer2 = await memory.AskAsync("What's the total population?",
MemoryFilters.ByDocument("europe001"));
```

![image](https://github.com/microsoft/semantic-memory/assets/371009/18ea98ee-1210-498d-8513-56abc795ce4d)

If you have any question, please do not hesitate to
[open a new issue](https://github.com/microsoft/semantic-memory/issues/new)
in the Semantic Memory repository. Thanks!
140 changes: 101 additions & 39 deletions docs/SECURITY_FILTERS.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Security Filters

This document provides some guidance about how to organize your documents in
order to secure your data, e.g. making sure users can access only data
meant to be accessible to them.

Semantic Memory allows to organize memories with two main approaches, which
can also be used together for maximum flexibility.

Expand Down Expand Up @@ -27,16 +31,22 @@ to each document a User ID tag that your application can filter by.

**Vector DBs like Azure Cognitive Search, Qdrant, Pinecone, etc. don't offer
document-level permissions** and search results can't vary by user.
Documents stored in Vector DBs though can be decorated with metadata and can
be filtered when searching, applying some filters.
Vector storages are optimized to store large quantity of documents indexed
using embedding vectors, and to quickly find similar documents.
Vector records stored in Vector DBs though can be decorated with metadata, and
can be filtered when searching, applying some logical filters.

Semantic Memory leverages this capability, and uses specific native filters
on all the supported Vector DBs (Azure Cognitive Search, Qdrant, etc), removing
the need to learn ad-hoc filtering syntax, allowing to **tag every memory
during the ingestion**, and allowing to **filter by tag when searching**,
during the retrieval process.

Semantic Memory makes this transparent regardless of the vector DB selected,
allowing to **tag every memory during the ingestion**, and allowing to **filter
by tag when searching**, during the retrieval process.
Tags are free and customizable. Multiple tags can be used and each tag can
have multiple values. Tags can be used to filter by user, by type, etc. and
in particular can be leveraged for your security scenarios.

Tags are completely free and customizable. Multiple tags can be used and each
tag can have multiple values, so you can create your custom security filters.
Here's some example scenarios:
Here's some examples:

* Use a "userID" tag to restrict records to one or multiple users.
* Use a "userEmail" tag to restrict records using the user email address.
Expand All @@ -52,10 +62,10 @@ cascade deletions, etc.
> so you should consider these two important points:
>
> 1. Memories stored without tags are visible only when searching without
> filters, e.g. they are visible to all users.
>2. Searching without filters searches the entire index. If you are using tags
> as security filters, **you should always filter by tags when retrieving**
> information.
> filters, e.g. they are visible to all users.
> 2. Searching without filters searches the entire index. If you are using tags
> as security filters, **you should always filter by tags when retrieving**
> information.
## Code examples

Expand All @@ -65,8 +75,10 @@ Simple file upload, without tags or explicit index name. The associated
memory records can't be filterable and are stored in the default index.

```csharp
// Upload a file into memory. This file has no tags.
var docId = await memory.ImportDocumentAsync("project.docx");


// Ask a question, without tags. This will search the entire index.
var answer = await memory.AskAsync("what's the project timeline?");
```

Expand All @@ -75,6 +87,7 @@ memory records can't be filtered by tags, but are isolated in a dedicated
index.

```csharp
// Upload a file in a specific index.
var docId = await memory.ImportDocumentAsync("project.docx", index: "index001");

// NO ANSWER: the data is not in the default index
Expand All @@ -84,6 +97,13 @@ var answer = await memory.AskAsync("what's the project timeline?");
var answer = await memory.AskAsync("what's the project timeline?", index: "index001");
```

### Security Filters

These examples use the `user` tag to secure data retrieval, making sure the
current user can see only data tagged by their user ID.

#### Example 1

File upload with a `user` tag. The associated memory records can be filtered
using the `user` tag.

Expand All @@ -92,67 +112,109 @@ a filter**.

```csharp
var docId = await memory.ImportDocumentAsync(new Document()
.AddFile("project.docx")
.AddTag("user", "USER-333"));
.AddFile("project.docx")
.AddTag("user", "USER-333"));

// OK
var answer = await memory.AskAsync("what's the project timeline?");

// OK
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-333"));
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-333"));

// NO ANSWER: memories are tagged with 'USER-333', so filter 'USER-444' will not match the information extracted from project.docs
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-444"));
// NO ANSWER: memories are tagged with 'USER-333', so filter 'USER-444'
// will not match the information extracted from project.docs
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-444"));
```

#### Example 2

Very similar to previous example, using a specific index.

```csharp
// Upload a document in specific user and tag with user ID.
var docId = await memory.ImportDocumentAsync(new Document()
.AddFile("project.docx")
.AddTag("user", "USER-333"),
index: "index002");
.AddFile("project.docx")
.AddTag("user", "USER-333"),
index: "index002");

// NO ANSWER: the data is not in the default index
var answer = await memory.AskAsync("what's the project timeline?");

// NO ANSWER: the data is not in the default index
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-333"));
// NO ANSWER: even if the filter is correct, the data is not in the default index
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-333"));

// OK
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-333"), index: "index002");
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-333"),
index: "index002");

// IMPORTANT: this command is missing the user tag and the service will return the data.
// This is equivalent to an admin having full access.
var answer = await memory.AskAsync("what's the project timeline?",
index: "index002");
```

#### Example 3

Example showing how to apply multiple tags, even for the same tag name.
E.g. in this case the document information is tagged with two user IDs,

In this case the document information is tagged with two user IDs,
so both users can ask for questions.

```csharp
// Upload file, allow two users to access
var docId = await memory.ImportDocumentAsync(new Document()
.AddFile("project.docx")
.AddTag("user", "USER-333")
.AddTag("user", "USER-444"));
.AddFile("project.docx")
.AddTag("user", "USER-333")
.AddTag("user", "USER-444"));

// OK
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-333"));
// OK: USER-333 tag matches
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-333"));

// OK
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-444"));
// OK: USER-444 tag matches
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-444"));
```

#### Example 4

Finally , tags can be used also for categorizing data:

```csharp
// Upload file, allow two users to access, and add a content type tag for extra filtering
var docId = await memory.ImportDocumentAsync(new Document()
.AddFile("project.docx")
.AddTag("user", "USER-333")
.AddTag("user", "USER-444")
.AddTag("type", "planning"));
.AddFile("project.docx")
.AddTag("user", "USER-333")
.AddTag("user", "USER-444")
.AddTag("type", "planning"));

// No information found, the type tag doesn't match
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-333").ByTag("type", "email"));
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-333")
.ByTag("type", "email"));

// OK
var answer = await memory.AskAsync("what's the project timeline?", MemoryFilters.ByTag("user", "USER-333").ByTag("type", "planning"));
var answer = await memory.AskAsync("what's the project timeline?",
MemoryFilters.ByTag("user", "USER-333")
.ByTag("type", "planning"));

```

# Security best practices

Summarizing, we recommend these best practices to secure Semantic Memory usage:

```
* Use Semantic Memory as **a private backend component**, similar to a SQL
Server, without granting direct access. When using Semantic Memory as a
service, consider assigning the service a reserved IP, accessible only to
your IP, and using HTTPS only.
* Authenticate users in your backend using a secure solution like Azure
Active Directory, extract the user ID from the signed credentials like JWT
tokens or client certs, and tag every interaction with Semantic Memory with
this User ID
* **Use Semantic Memory Tags as Security Filters**. Make sure every API call
to Semantic Memory uses a User tag, both when reading and writing to memory.

0 comments on commit 9fc27ed

Please sign in to comment.