From c9bd6fea0a032347df3e35316b6e19b0e33c6f1f Mon Sep 17 00:00:00 2001 From: Amit Galitzky Date: Fri, 13 Sep 2024 12:29:45 -0700 Subject: [PATCH] Adding documentation for remote index use in AD (#8191) * adding documentation for remote index use in AD Signed-off-by: Amit Galitzky * Update _observing-your-data/ad/index.md Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/index.md Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/index.md Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/index.md Signed-off-by: Melissa Vagi * adding additional security information Signed-off-by: Amit Galitzky * fixing formatting issues Signed-off-by: Amit Galitzky * Update _observing-your-data/ad/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * doc review new content and address editorial review comments Signed-off-by: Melissa Vagi * doc review new content and address editorial review comments Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Signed-off-by: Melissa Vagi * doc review new content and address editorial review comments Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/security.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Amit Galitzky Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower --- _observing-your-data/ad/index.md | 9 +++++-- _observing-your-data/ad/security.md | 41 +++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/_observing-your-data/ad/index.md b/_observing-your-data/ad/index.md index 5dfa1b8f1a..f565ca6e31 100644 --- a/_observing-your-data/ad/index.md +++ b/_observing-your-data/ad/index.md @@ -10,7 +10,7 @@ redirect_from: # Anomaly detection -An anomaly in OpenSearch is any unusual behavior change in your time-series data. Anomalies can provide valuable insights into your data. For example, for IT infrastructure data, an anomaly in the memory usage metric might help you uncover early signs of a system failure. +An _anomaly_ in OpenSearch is any unusual behavior change in your time-series data. Anomalies can provide valuable insights into your data. For example, for IT infrastructure data, an anomaly in the memory usage metric might help you uncover early signs of a system failure. It can be challenging to discover anomalies using conventional methods such as creating visualizations and dashboards. You could configure an alert based on a static threshold, but this requires prior domain knowledge and isn't adaptive to data that exhibits organic growth or seasonal behavior. @@ -29,9 +29,14 @@ A detector is an individual anomaly detection task. You can define multiple dete 1. Add in the detector details. - Enter a name and brief description. Make sure the name is unique and descriptive enough to help you to identify the purpose of the detector. 1. Specify the data source. - - For **Data source**, choose the index you want to use as the data source. You can optionally use index patterns to choose multiple indexes. + - For **Data source**, choose one or more indexes to use as the data source. Alternatively, you can use an alias or index pattern to choose multiple indexes. + - Detectors can use remote indexes. You can access them using the `cluster-name:index-name` pattern. See [Cross-cluster search]({{site.url}}{{site.baseurl}}/search-plugins/cross-cluster-search/) for more information. Alternatively, you can select clusters and indexes in OpenSearch Dashboards 2.17 or later. To learn about configuring remote indexes with the Security plugin enabled, see [Selecting remote indexes with fine-grained access control]({{site.url}}{{site.baseurl}}/observing-your-data/ad/security/#selecting-remote-indexes-with-fine-grained-access-control) in the [Anomaly detection security](observing-your-data/ad/security/) documentation. - (Optional) For **Data filter**, filter the index you chose as the data source. From the **Data filter** menu, choose **Add data filter**, and then design your filter query by selecting **Field**, **Operator**, and **Value**, or choose **Use query DSL** and add your own JSON filter query. Only [Boolean queries]({{site.url}}{{site.baseurl}}/query-dsl/compound/bool/) are supported for query domain-specific language (DSL). + +To create a cross-cluster detector in OpenSearch Dashboards, the following [permissions]({{site.url}}{{site.baseurl}}/security/access-control/permissions/) are required: `indices:data/read/field_caps`, `indices:admin/resolve/index`, and `cluster:monitor/remote/info`. +{: .note} + #### Example filter using query DSL The query is designed to retrieve documents in which the `urlPath.keyword` field matches one of the following specified values: diff --git a/_observing-your-data/ad/security.md b/_observing-your-data/ad/security.md index 8eeaa3df41..e4816cec46 100644 --- a/_observing-your-data/ad/security.md +++ b/_observing-your-data/ad/security.md @@ -23,6 +23,11 @@ As an admin user, you can use the Security plugin to assign specific permissions The Security plugin has two built-in roles that cover most anomaly detection use cases: `anomaly_full_access` and `anomaly_read_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security/access-control/users-roles#predefined-roles). +If you use OpenSearch Dashboards to create your anomaly detectors, you may experience access issues even with `anomaly_full_access`. This issue has been resolved in OpenSearch 2.17, but for earlier versions, the following additional permissions need to be added: + +- `indices:data/read/search` -- You need this permission because the Anomaly Detection plugin needs to search the data source in order to validate whether there is enough data to train the model. +- `indices:admin/mappings/fields/get` and `indices:admin/mappings/fields/get*` -- You need these permissions to validate whether the given data source has a valid timestamp field and categorical field (in the case of creating a high-cardinality detector). + If these roles don't meet your needs, mix and match individual anomaly detection [permissions]({{site.url}}{{site.baseurl}}/security/access-control/permissions/) to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/ad/detector/delete` permission lets you delete detectors. ### A note on alerts and fine-grained access control @@ -31,6 +36,42 @@ When a trigger generates an alert, the detector and monitor configurations, the To reduce the chances of unintended users viewing metadata that could describe an index, we recommend that administrators enable role-based access control and keep these kinds of design elements in mind when assigning permissions to the intended group of users. See [Limit access by backend role](#advanced-limit-access-by-backend-role) for details. +### Selecting remote indexes with fine-grained access control + +To use a remote index as a data source for a detector, see the setup steps in [Authentication flow]({{site.url}}{{site.baseurl}}/search-plugins/cross-cluster-search/#authentication-flow) in [Cross-cluster search]({{site.url}}{{site.baseurl}}/search-plugins/cross-cluster-search/). You must use a role that exists in both the remote and local clusters. The remote cluster must map the chosen role to the same username as in the local cluster. + +--- + +#### Example: Create a new user on the local cluster + +1. Create a new user on the local cluster to use for detector creation: + +``` +curl -XPUT -k -u 'admin:' 'https://localhost:9200/_plugins/_security/api/internalusers/anomalyuser' -H 'Content-Type: application/json' -d '{"password":"password"}' +``` +{% include copy-curl.html %} + +2. Map the new user to the `anomaly_full_access` role: + +``` +curl -XPUT -k -u 'admin:' -H 'Content-Type: application/json' 'https://localhost:9200/_plugins/_security/api/rolesmapping/anomaly_full_access' -d '{"users" : ["anomalyuser"]}' +``` +{% include copy-curl.html %} + +3. On the remote cluster, create the same user and map `anomaly_full_access` to that role: + +``` +curl -XPUT -k -u 'admin:' 'https://localhost:9250/_plugins/_security/api/internalusers/anomalyuser' -H 'Content-Type: application/json' -d '{"password":"password"}' +curl -XPUT -k -u 'admin:' -H 'Content-Type: application/json' 'https://localhost:9250/_plugins/_security/api/rolesmapping/anomaly_full_access' -d '{"users" : ["anomalyuser"]}' +``` +{% include copy-curl.html %} + +--- + +### Custom results index + +To use a custom results index, you need additional permissions not included in the default roles provided by the OpenSearch Security plugin. To add these permissions, see [Step 1: Define a detector]({{site.url}}{{site.baseurl}}/observing-your-data/ad/index/#step-1-define-a-detector) in the [Anomaly detection]({{site.url}}{{site.baseurl}}/observing-your-data/ad/index/) documentation. + ## (Advanced) Limit access by backend role Use backend roles to configure fine-grained access to individual detectors based on roles. For example, users of different departments in an organization can view detectors owned by their own department.