Merge branch 'main' into trace-analytics-update

_api-reference/document-apis/reindex.md

@@ -57,7 +57,7 @@ Your request body must contain the names of the source index and destination ind
 
 Field | Description
 :--- | :---
-conflicts | Indicates to OpenSearch what should happen if the delete by query operation runs into a version conflict. Valid options are `abort` and `proceed`. Default is abort.
+conflicts | Indicates to OpenSearch what should happen if the Reindex operation runs into a version conflict. Valid options are `abort` and `proceed`. Default is `abort`.
 source | Information about the source index to include. Valid fields are `index`, `max_docs`, `query`, `remote`, `size`, `slice`, and `_source`.
 index | The name of the source index to copy data from.
 max_docs | The maximum number of documents to reindex.

_api-reference/security-apis.md

@@ -0,0 +1,20 @@
+---
+layout: default
+title: Security APIs
+nav_order: 84
+---
+
+# Security APIs
+
+Security APIs provide information that can be very useful in troubleshooting connection and configuration issues.
+
+API | Method | Description
+:--- | :--- | :---
+`/_plugins/_security/whoami` | GET/POST | Returns basic details about the logged-in user.
+`/_opendistro/_security/sslinfo` | GET | Returns details about the SSL connection when using certificate authentication.
+`/_plugins/_security/api/permissionsinfo` | GET | Returns permission details for the logged-in user.
+`/_plugins/_security/authinfo` | GET/POST | Returns the backend roles and OpenSearch roles mapped to the logged-in user.
+`/_plugins/_security/api/ssl/certs` | GET | Displays the details and expiration dates of the certificates used on the OpenSearch HTTP and transport communication layers. Can only be called by users with the `superadmin` certificate.
+`/_plugins/_security/api/ssl/transport/reloadcerts` | PUT | Reloads the certificates on the `transport` layer. For more information, see [Reload TLS certificates on the transport layer]({{site.url}}{{site.baseurl}}/security/configuration/tls/#reload-tls-certificates-on-the-transport-layer).
+`/_plugins/_security/api/ssl/http/reloadcerts` | PUT | Reloads the certificates on the `http` layer. For more information, see [Reload TLS certificates on the http layer]({{site.url}}{{site.baseurl}}/security/configuration/tls/#reload-tls-certificates-on-the-http-layer).
+

_automating-configurations/api/delete-workflow.md

@@ -7,9 +7,11 @@ nav_order: 80
 
 # Delete a workflow
 
-When you no longer need a workflow template, you can delete it by calling the Delete Workflow API. 
+When you no longer need a workflow template, you can delete it by calling the Delete Workflow API.
 
-Note that deleting a workflow only deletes the stored template but does not deprovision its resources.  
+Note that deleting a workflow only deletes the stored template---it does not deprovision its resources.
+
+When a workflow is deleted, its corresponding status (returned by the [Workflow State API]({{site.url}}{{site.baseurl}}/automating-configurations/api/get-workflow-status/)) is also deleted unless either the provisioning status is `IN_PROGRESS` or resources have been provisioned.
 
 ## Path and HTTP methods
 
@@ -25,13 +27,26 @@ The following table lists the available path parameters.
 | :--- | :--- | :--- |
 | `workflow_id` | String | The ID of the workflow to be retrieved. Required. |
 
+## Query parameters
+
+The following table lists the available query parameters. All query parameters are optional.
+
+| Parameter | Data type | Description |
+| :--- | :--- | :--- |
+| `clear_status` | Boolean | Determines whether to delete the workflow state (without deprovisioning resources) after deleting the template. OpenSearch deletes the workflow state only if the provisioning status is not `IN_PROGRESS`. Default is `false`. |
+
 #### Example request
 
-```
+```json
 DELETE /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50
 ```
 {% include copy-curl.html %}
 
+```json
+DELETE /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50?clear_status=true
+```
+{% include copy-curl.html %}
+
 #### Example response
 
 If the workflow exists, a delete response contains the status of the deletion, where the `result` field is set to `deleted` on success or `not_found` if the workflow does not exist (it may have already been deleted):
@@ -50,4 +65,4 @@ If the workflow exists, a delete response contains the status of the deletion, w
   "_seq_no": 2,
   "_primary_term": 1
 }
-```
+```

_automating-configurations/workflow-steps.md

@@ -42,6 +42,25 @@ The following table lists the workflow step types. The `user_inputs` fields for
 |`create_index`|[Create Index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/)     | Creates a new OpenSearch index. The inputs include `index_name`, which should be the name of the index to be created, and `configurations`, which contains the payload body of a regular REST request for creating an index.
 |`create_ingest_pipeline`|[Create Ingest Pipeline]({{site.url}}{{site.baseurl}}/ingest-pipelines/create-ingest/) | Creates or updates an ingest pipeline. The inputs include `pipeline_id`, which should be the ID of the pipeline, and `configurations`, which contains the payload body of a regular REST request for creating an ingest pipeline.
 |`create_search_pipeline`|[Create Search Pipeline]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/creating-search-pipeline/) | Creates or updates a search pipeline. The inputs include `pipeline_id`, which should be the ID of the pipeline, and `configurations`, which contains the payload body of a regular REST request for creating a search pipeline.
+|`reindex`|[Reindex]({{site.url}}{{site.baseurl}}/api-reference/document-apis/reindex/)  | The reindex document API operation lets you copy all or a subset of your data from a source index into a destination index. The input includes source_index, destination_index, and the following optional parameters from the document reindex API: `refresh`, `requests_per_second`, `require_alias`, `slices`, and `max_docs`. For more information, see [Reindexing considerations](#reindexing-considerations).
+
+## Reindexing considerations
+
+Reindexing can be a resource-intensive operation, and if not managed properly, it can potentially destabilize your cluster. 
+
+When using a `reindex` step, follow these best practices to ensure a smooth reindexing process and prevent cluster instability:
+
+- **Cluster scaling**: Before initiating a reindexing operation, ensure that your OpenSearch cluster is properly scaled to handle the additional workload. Increase the number of nodes and adjust resource allocation (CPU, memory, and disk) as needed to accommodate the reindexing process without impacting other operations.
+
+- **Request rate control**: Use the `requests_per_second` parameter to control the rate at which the reindexing requests are sent to the cluster. This helps to regulate the load on the cluster and prevent resource exhaustion. Start with a lower value and gradually increase it based on your cluster's capacity and performance.
+
+- **Slicing and parallelization**: The `slices` parameter allows you to divide the reindexing process into smaller, parallel tasks. This can help distribute the workload across multiple nodes and improve overall performance. However, be cautious when increasing the number of slices because adding slices can increase resource consumption.
+
+- **Monitoring and adjustments**: Closely monitor your cluster performance metrics (such as CPU, memory, disk usage, and thread pools) during the reindexing process. If you notice any signs of resource contention or performance degradation, adjust the reindexing parameters accordingly or consider pausing the operation until the cluster stabilizes.
+
+- **Prioritization and scheduling**: If possible, schedule reindexing operations during off-peak hours or periods of lower cluster utilization to minimize the impact on other operations and user traffic.
+
+By following these best practices and carefully managing the reindexing process, you can ensure that your OpenSearch cluster remains stable and performant while efficiently copying data between indexes.
 
 ## Additional fields
 

_automating-configurations/workflow-templates.md

@@ -138,5 +138,8 @@ The following table lists the supported workflow templates. To use a workflow te
 | `multimodal_search_with_bedrock_titan`    | Deploys an Amazon Bedrock multimodal model and configures an ingest pipeline with a `text_image_embedding` processor and a k-NN index for [multimodal search]({{site.url}}{{site.baseurl}}/search-plugins/multimodal-search/). You must provide your AWS credentials. | `create_connector.credential.access_key`, `create_connector.credential.secret_key`, `create_connector.credential.session_token` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/multimodal-search-bedrock-titan-defaults.json)                  |
 | `hybrid_search`  | Configures [hybrid search]({{site.url}}{{site.baseurl}}/search-plugins/hybrid-search/): <br> - Creates an ingest pipeline, a k-NN index, and a search pipeline with a `normalization_processor`. You must provide the model ID of the text embedding model to be used. | `create_ingest_pipeline.model_id`        |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/hybrid-search-defaults.json)                  |
 | `conversational_search_with_llm_deploy`  | Deploys a large language model (LLM) (by default, Cohere Chat) and configures a search pipeline with a `retrieval_augmented_generation` processor for [conversational search]({{site.url}}{{site.baseurl}}/search-plugins/conversational-search/).  | `create_connector.credential.key`  |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/conversational-search-defaults.json) |
+| `semantic_search_with_reindex`  | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/) with a newly deployed Cohere embedding model. The model is configured to reindex a source index into a newly configured k-NN index. You must provide the API key for the Cohere model along with the source index to be reindexed. | `create_connector.credential.key`, `reindex.source_index`|[Defaults](https://github.com/opensearch-project/flow-framework/blob/main/src/main/resources/defaults/semantic-search-with-reindex-defaults.json) |
+| `semantic_search_with_local_model`  | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/) and deploys a pretrained model (`msmarco-distilbert-base-tas-b`). Adds a [`query_enricher`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/neural-query-enricher/) search processor that sets a default model ID for neural queries and creates a linked k-NN index called `my-nlp-index`. You must provide the API key for the Cohere model.  | None  | [Defaults](https://github.com/opensearch-project/flow-framework/blob/main/src/main/resources/defaults/semantic-search-with-local-model-defaults.json) |
+| `hybrid_search_with_local_model`  | Configures [hybrid search]({{site.url}}{{site.baseurl}}/search-plugins/hybrid-search/) and deploys a pretrained model (`msmarco-distilbert-base-tas-b`). Creates an ingest pipeline, a k-NN index, and a search pipeline with a `normalization_processor`.  | None  | [Defaults](https://github.com/opensearch-project/flow-framework/blob/main/src/main/resources/defaults/hybrid-search-with-local-model-defaults.json) |