Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding documentation for remote index use in AD #8191

Merged
merged 26 commits into from
Sep 13, 2024

Conversation

amitgalitz
Copy link
Member

@amitgalitz amitgalitz commented Sep 6, 2024

Description

In 2.17 we have added the capability to select a remote index as a data source for creating an anomaly detector

Issues Resolved

Closes #8169

Version

2.17

Frontend features

new addition is just the cluster drop down

Screenshot 2024-09-06 at 8 02 55 AM

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

github-actions bot commented Sep 6, 2024

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

@kolchfa-aws kolchfa-aws assigned vagimeli and unassigned kolchfa-aws Sep 6, 2024
@kolchfa-aws kolchfa-aws added release-notes PR: Include this PR in the automated release notes v2.17.0 labels Sep 6, 2024
Copy link
Contributor

@vagimeli vagimeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc review complete. Edits made for clarity and compliance with OpenSearch style.

@vagimeli vagimeli added 5 - Editorial review PR: Editorial review in progress 6 - Done but waiting to merge PR: The work is done and ready to merge labels Sep 6, 2024

### Selecting remote indices with fine-grained access control

In order to specify a remote index as the data source for a detector you must follow similar instruction as mentioned in the cross-cluster search documentation. By utilizing a role that exists in the both the remote and local cluster.The remote cluster will need to have the mapping between the chosen role and the same user name as in the local cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you link the instruction from cross-cluster search?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added


### Custom result index

In order to use a custom result index users will need to have additional permissions that aren't part of the default roles that opensearch security plugin offers. Users will have to add additional permissions as mentioned in the custom result index setup [step](https://opensearch.org/docs/latest/observing-your-data/ad/index/#example-filter-using-query-dsl:~:text=Specify%20custom%20results%20index.).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it ok to link in this way? I saw others link like ({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see both ways being used, but I changed to use the constants here to be consistent with any changes in baseurl

@kaituo
Copy link
Contributor

kaituo commented Sep 10, 2024

Jekyll is failing but links works locally fine for me, I can just delete links if its a blocker

Can you try {{site.url}}{{site.baseurl}}/monitoring-plugins/ad/... or change back to original link?

Signed-off-by: Amit Galitzky <[email protected]>
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitgalitz @vagimeli Please see my comments and changes and tag me when complete for a reread and approval. Thanks!

_observing-your-data/ad/index.md Outdated Show resolved Hide resolved
@@ -29,9 +29,14 @@ A detector is an individual anomaly detection task. You can define multiple dete
1. Add in the detector details.
- Enter a name and brief description. Make sure the name is unique and descriptive enough to help you to identify the purpose of the detector.
1. Specify the data source.
- For **Data source**, choose the index you want to use as the data source. You can optionally use index patterns to choose multiple indexes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line below: Should "index pattern" be singular?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This singular usage sounds good to me on latest revision:
"- For Data source, choose one or more indexes to use as the data source. Alternatively, you can use an alias or index pattern to choose multiple indexes."

_observing-your-data/ad/index.md Outdated Show resolved Hide resolved
@@ -29,9 +29,14 @@ A detector is an individual anomaly detection task. You can define multiple dete
1. Add in the detector details.
- Enter a name and brief description. Make sure the name is unique and descriptive enough to help you to identify the purpose of the detector.
1. Specify the data source.
- For **Data source**, choose the index you want to use as the data source. You can optionally use index patterns to choose multiple indexes.
- For **Data source**, choose the index or indexes you want to use as the data source. Alternatively, you can use an alias or index patterns to choose multiple indexes.
- Detectors can use remote indexes. You can access them using the `cluster-name:index-name` pattern, as used in [cross-cluster search]({{site.url}}{{site.baseurl}}/search-plugins/cross-cluster-search/). Alternatively, you can select the clusters and indexes you want to use through OpenSearch Dashboards 2.17 or later. For more information about configuring remote indexes with the Security plugin enabled, you can view [here]({{site.url}}{{site.baseurl}}/observing-your-data/ad/security/#selecting-remote-indexes-with-fine-grained-access-control)
- (Optional) For **Data filter**, filter the index you chose as the data source. From the **Data filter** menu, choose **Add data filter**, and then design your filter query by selecting **Field**, **Operator**, and **Value**, or choose **Use query DSL** and add your own JSON filter query. Only [Boolean queries]({{site.url}}{{site.baseurl}}/query-dsl/compound/bool/) are supported for query domain-specific language (DSL).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line above: Instead of "here", please use the name of the page for the link text.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@natebower This is my bad. The team added new content yesterday after the approved doc review. I needed to reread it and pull it from the editorial queue. I'll work on this and tag you one it's ready. I'm sorry.

_observing-your-data/ad/index.md Outdated Show resolved Hide resolved
_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
### Custom result index

In order to use a custom result index users will need to have additional permissions that aren't part of the default roles that OpenSearch Security plugin offers. Users will have to add additional permissions as mentioned in the custom result index set up step in [define detector]({{site.url}}{{site.baseurl}}/observing-your-data/ad/index/#step-1-define-a-detector).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line above: Last sentence needs some revision.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revised

vagimeli and others added 10 commits September 11, 2024 08:42
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amitgalitz @vagimeli Approved with one comment and a couple of changes. Thanks!

@@ -23,6 +23,11 @@ As an admin user, you can use the Security plugin to assign specific permissions

The Security plugin has two built-in roles that cover most anomaly detection use cases: `anomaly_full_access` and `anomaly_read_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security/access-control/users-roles#predefined-roles).

If you use OpenSearch Dashboards to create your anomaly detectors, you may experience permission issues even with `anomaly_full_access`. This issue has been resolved in OpenSearch 2.17, but for earlier versions, the following additional permissions need to be added:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"permissions issues"? Or do we actually mean "access issues"?

_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
_observing-your-data/ad/security.md Outdated Show resolved Hide resolved
vagimeli and others added 3 commits September 13, 2024 13:20
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
@vagimeli vagimeli merged commit c9bd6fe into opensearch-project:main Sep 13, 2024
5 checks passed
noahstaveley pushed a commit to noahstaveley/documentation-website that referenced this pull request Sep 23, 2024
)

* adding documentation for remote index use in AD

Signed-off-by: Amit Galitzky <[email protected]>

* Update _observing-your-data/ad/index.md

Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/index.md

Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/index.md

Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/index.md

Signed-off-by: Melissa Vagi <[email protected]>

* adding additional security information

Signed-off-by: Amit Galitzky <[email protected]>

* fixing formatting issues

Signed-off-by: Amit Galitzky <[email protected]>

* Update _observing-your-data/ad/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/index.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* doc review new content and address editorial review comments

Signed-off-by: Melissa Vagi <[email protected]>

* doc review new content and address editorial review comments

Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Signed-off-by: Melissa Vagi <[email protected]>

* doc review new content and address editorial review comments

Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>

* Update _observing-your-data/ad/security.md

Signed-off-by: Melissa Vagi <[email protected]>

---------

Signed-off-by: Amit Galitzky <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Noah Staveley <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Editorial review PR: Editorial review in progress 6 - Done but waiting to merge PR: The work is done and ready to merge release-notes PR: Include this PR in the automated release notes v2.17.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Rule-based AD, remote index, imputation
6 participants