Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding documentation for remote index use in AD #8191

Merged
merged 26 commits into from
Sep 13, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
4ca8528
adding documentation for remote index use in AD
amitgalitz Sep 6, 2024
aa16e9a
Update _observing-your-data/ad/index.md
vagimeli Sep 6, 2024
351c8f0
Update _observing-your-data/ad/index.md
vagimeli Sep 6, 2024
08bc9d6
Update _observing-your-data/ad/index.md
vagimeli Sep 6, 2024
addce3b
Update _observing-your-data/ad/index.md
vagimeli Sep 6, 2024
6eac47f
adding additional security information
amitgalitz Sep 10, 2024
39bebb1
fixing formatting issues
amitgalitz Sep 10, 2024
c923eec
Update _observing-your-data/ad/index.md
vagimeli Sep 11, 2024
1672240
Update _observing-your-data/ad/index.md
vagimeli Sep 11, 2024
1586c9e
Update _observing-your-data/ad/index.md
vagimeli Sep 11, 2024
b909911
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
8a04f39
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
85ea3c8
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
d0f494e
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
bdf6b03
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
563a610
doc review new content and address editorial review comments
vagimeli Sep 11, 2024
7f76d59
doc review new content and address editorial review comments
vagimeli Sep 11, 2024
b278b1d
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
ab43ebf
Update _observing-your-data/ad/security.md
vagimeli Sep 11, 2024
52382a5
doc review new content and address editorial review comments
vagimeli Sep 11, 2024
37bbc82
Merge branch 'ad-remote-cluster' of https://github.com/amitgalitz/doc…
vagimeli Sep 11, 2024
93dc948
Update _observing-your-data/ad/security.md
vagimeli Sep 13, 2024
f89eb55
Update _observing-your-data/ad/security.md
vagimeli Sep 13, 2024
c2cf1c0
Update _observing-your-data/ad/security.md
vagimeli Sep 13, 2024
60de517
Update _observing-your-data/ad/security.md
vagimeli Sep 13, 2024
a179f64
Merge branch 'main' into ad-remote-cluster
vagimeli Sep 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion _observing-your-data/ad/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,14 @@
1. Add in the detector details.
- Enter a name and brief description. Make sure the name is unique and descriptive enough to help you to identify the purpose of the detector.
1. Specify the data source.
- For **Data source**, choose the index you want to use as the data source. You can optionally use index patterns to choose multiple indexes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line below: Should "index pattern" be singular?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This singular usage sounds good to me on latest revision:
"- For Data source, choose one or more indexes to use as the data source. Alternatively, you can use an alias or index pattern to choose multiple indexes."

- For **Data source**, choose the index or indexes you want to use as the data source. Alternatively, you can use an alias or index patterns to choose multiple indexes.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- Detectors can use remote indexes. You can access them using the `cluster-name:index-name` pattern, as used in [cross-cluster search]({{site.url}}{{site.baseurl}}/search-plugins/cross-cluster-search/). Alternatively, you can select the clusters and indexes you want to use through OpenSearch Dashboards 2.17 or later. For more information about configuring remote indexes with the Security plugin enabled, you can view [here]({{site.url}}{{site.baseurl}}/observing-your-data/ad/security/#selecting-remote-indexes-with-fine-grained-access-control)

Check failure on line 33 in _observing-your-data/ad/index.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: selectin. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: selectin. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_observing-your-data/ad/index.md", "range": {"start": {"line": 33, "column": 493}}}, "severity": "ERROR"}
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
- (Optional) For **Data filter**, filter the index you chose as the data source. From the **Data filter** menu, choose **Add data filter**, and then design your filter query by selecting **Field**, **Operator**, and **Value**, or choose **Use query DSL** and add your own JSON filter query. Only [Boolean queries]({{site.url}}{{site.baseurl}}/query-dsl/compound/bool/) are supported for query domain-specific language (DSL).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line above: Instead of "here", please use the name of the page for the link text.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natebower This is my bad. The team added new content yesterday after the approved doc review. I needed to reread it and pull it from the editorial queue. I'll work on this and tag you one it's ready. I'm sorry.



To create a cross-cluster detector through OpenSearch Dashboards, the following [permissions]({{site.url}}{{site.baseurl}}/security/access-control/permissions/) are required: `indices:data/read/field_caps`, `indices:admin/resolve/index`, and `cluster:monitor/remote/info`.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
{: .note}

#### Example filter using query DSL
The query is designed to retrieve documents in which the `urlPath.keyword` field matches one of the following specified values:

Expand Down
35 changes: 35 additions & 0 deletions _observing-your-data/ad/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ As an admin user, you can use the Security plugin to assign specific permissions

The Security plugin has two built-in roles that cover most anomaly detection use cases: `anomaly_full_access` and `anomaly_read_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security/access-control/users-roles#predefined-roles).

If you are using OpenSearch Dashboards to create your anomaly detectors you might notice issues creating a detector even with `anomaly_full_access`, this has been mitigated in OpenSearch 2.17. In order to mitigate this issue in prior versions, you should add these additional permissions:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line above: What do we mean by "you may experience issues"? What "issue" may the user experience, specifically?

Copy link
Contributor

@vagimeli vagimeli Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitgalitz Please see the editor's comment above. We should give the users context about what issues they may experience. I think we're referring to permissions issues. Please confirm.

- `indices:data/read/search` - You need the search permission because the Anomaly Detection plugin needs to search the data source in order to validate if there is enough data to train the model.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `indices:data/read/search` - You need the search permission because the Anomaly Detection plugin needs to search the data source in order to validate if there is enough data to train the model.
- `indices:data/read/search` -- You need this permission because the Anomaly Detection plugin needs to search the data source in order to validate whether there is enough data to train the model.

- `indices:admin/mappings/fields/get` and `indices:admin/mappings/fields/get*` - You need field mapping permissions since as part of detector creation we validate if the given data source has a valid timestamp field and categorical field (in case of creating a high cardinality detector)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `indices:admin/mappings/fields/get` and `indices:admin/mappings/fields/get*` - You need field mapping permissions since as part of detector creation we validate if the given data source has a valid timestamp field and categorical field (in case of creating a high cardinality detector)
- `indices:admin/mappings/fields/get` and `indices:admin/mappings/fields/get*` -- You need these permissions to validate whether the given data source has a valid timestamp field and categorical field (in the case of creating a high-cardinality detector).


If these roles don't meet your needs, mix and match individual anomaly detection [permissions]({{site.url}}{{site.baseurl}}/security/access-control/permissions/) to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/ad/detector/delete` permission lets you delete detectors.

### A note on alerts and fine-grained access control
Expand All @@ -31,6 +36,36 @@ When a trigger generates an alert, the detector and monitor configurations, the

To reduce the chances of unintended users viewing metadata that could describe an index, we recommend that administrators enable role-based access control and keep these kinds of design elements in mind when assigning permissions to the intended group of users. See [Limit access by backend role](#advanced-limit-access-by-backend-role) for details.


### Selecting remote indexes with fine-grained access control

In order to specify a remote index as the data source for a detector you must follow similar instruction as mentioned in the cross-cluster search [documentation]({{site.url}}{{site.baseurl}}/search-plugins/cross-cluster-search/#authentication-flow). By utilizing a role that exists in the both the remote and local cluster.The remote cluster will need to have the mapping between the chosen role and the same user name as in the local cluster.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This para needs some revision for completeness and clarity. Also, we should just tell the user what instructions they need to follow rather than telling them to follow instructions similar to something else.


For example:

Create a new user on the local cluster from which you’ll create detectors from:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revise.


```
curl -XPUT -k -u 'admin:<custom-admin-password>' 'https://localhost:9200/_plugins/_security/api/internalusers/anomalyuser' -H 'Content-Type: application/json' -d '{"password":"password"}'
```

Map the new user to the anomaly_full_access role:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

```
curl -XPUT -k -u 'admin:<custom-admin-password>' -H 'Content-Type: application/json' 'https://localhost:9200/_plugins/_security/api/rolesmapping/anomaly_full_access' -d '{"users" : ["anomalyuser"]}'
```

On the remote cluster, create the same user, and map anomaly_full_access to that role:
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

```
curl -XPUT -k -u 'admin:<custom-admin-password>' 'https://localhost:9250/_plugins/_security/api/internalusers/anomalyuser' -H 'Content-Type: application/json' -d '{"password":"password"}'
curl -XPUT -k -u 'admin:<custom-admin-password>' -H 'Content-Type: application/json' 'https://localhost:9250/_plugins/_security/api/rolesmapping/anomaly_full_access' -d '{"users" : ["anomalyuser"]}'
```

### Custom result index
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

In order to use a custom result index users will need to have additional permissions that aren't part of the default roles that OpenSearch Security plugin offers. Users will have to add additional permissions as mentioned in the custom result index set up step in [define detector]({{site.url}}{{site.baseurl}}/observing-your-data/ad/index/#step-1-define-a-detector).
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line above: Last sentence needs some revision.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revised

## (Advanced) Limit access by backend role

Use backend roles to configure fine-grained access to individual detectors based on roles. For example, users of different departments in an organization can view detectors owned by their own department.
Expand Down
Loading