…search-project#12607)

Signed-off-by: Ashish Singh <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

opensearch-project#11699)

Signed-off-by: Sooraj Sinha <[email protected]>

…12724)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Fixing ehcache flaky test

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding a ehcache issue reference for thread leak issue

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Updating comment

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>

…e API (opensearch-project#12657)

* Update supported version for adding primary_only parameter to force-merge API

Signed-off-by: Gao Binlong <[email protected]>

* Modify change log

Signed-off-by: Gao Binlong <[email protected]>

* Remove some unused code

Signed-off-by: Gao Binlong <[email protected]>

* Remove change log

Signed-off-by: Gao Binlong <[email protected]>

* Fix test issue

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>

…arch-project#12783)

Signed-off-by: Shweta Thareja <[email protected]>
Co-authored-by: Shweta Thareja <[email protected]>

…ene writer (opensearch-project#12710)

Signed-off-by: Prabhat Sharma <[email protected]>
Co-authored-by: Prabhat Sharma <[email protected]>

* [FEATURE] Built-in secure transports support

Signed-off-by: Andriy Redko <[email protected]>

* Added more tests, addressing code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>

…pensearch-project#12753)

* Use cluster default remote store path type as fallback during snapshot restore

---------

Signed-off-by: Ashish Singh <[email protected]>

…n /plugins/repository-hdfs (opensearch-project#12721)

* Bump org.apache.commons:commons-configuration2

Bumps org.apache.commons:commons-configuration2 from 2.9.0 to 2.10.0.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-configuration2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Created new ReplicaShardBatchAllocator to be used instead of ReplicaShardAllocator for batch calls

Signed-off-by: Shivansh Arora <[email protected]>

…azure (opensearch-project#12723)

* Bump com.azure:azure-json in /plugins/repository-azure

Bumps [com.azure:azure-json](https://github.com/Azure/azure-sdk-for-java) from 1.0.1 to 1.1.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-json_1.0.1...v1.1.0)

---
updated-dependencies:
- dependency-name: com.azure:azure-json
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

…34.0 in /plugins/repository-gcs (opensearch-project#12365)

* Bump com.google.api.grpc:proto-google-common-protos

Bumps [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) from 2.33.0 to 2.34.0.
- [Release notes](https://github.com/googleapis/sdk-platform-java/releases)
- [Changelog](https://github.com/googleapis/sdk-platform-java/blob/main/CHANGELOG.md)
- [Commits](googleapis/sdk-platform-java@v2.33.0...v2.34.0)

---
updated-dependencies:
- dependency-name: com.google.api.grpc:proto-google-common-protos
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

…uster-state from remote (opensearch-project#12252)

Signed-off-by: Rajiv Kumar Vaidyanathan <[email protected]>

…pensearch-project#12494)

Changes for Primary Relocation during migration 

---------

Signed-off-by: Gaurav Bafna <[email protected]>

…s. (opensearch-project#12496)

* Integrate with CPU admission controller for cluster-manager Read API's.
The admission control is enforced at the transport layer.

Signed-off-by: Rajiv Kumar Vaidyanathan <[email protected]>

…n&clone&shrink&split APIs (opensearch-project#12819)

Signed-off-by: Gao Binlong <[email protected]>

* Fix Flaky SimpleQueryStringIT Tests

Signed-off-by: kkewwei <[email protected]>

* add the comment to unit test

Signed-off-by: kkewwei <[email protected]>

---------

Signed-off-by: kkewwei <[email protected]>

…ensearch-project#12830)

Signed-off-by: Rajiv Kumar Vaidyanathan <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: Varun Bansal <[email protected]>

* Always return a task description even when it cannot be serialized.

Signed-off-by: dblock <[email protected]>

* Expect tasks to fail.

Signed-off-by: dblock <[email protected]>

* Only catch exceptions when getting description.

Signed-off-by: dblock <[email protected]>

* Added <error: ...> to mark error more clearly.

Signed-off-by: dblock <[email protected]>

---------

Signed-off-by: dblock <[email protected]>

The 2.13 release notes have been created.

Signed-off-by: Andrew Ross <[email protected]>

Signed-off-by: vikasvb90 <[email protected]>

…#12851)

Signed-off-by: Varun Bansal <[email protected]>

…ch-project#12776)

Signed-off-by: Neetika Singhal <[email protected]>

…arch-project#12807)

* Add explicit dependency to PomValidation

Signed-off-by: Zelin Hao <[email protected]>

* Update CHANGELOG

Signed-off-by: Zelin Hao <[email protected]>

---------

Signed-off-by: Zelin Hao <[email protected]>

opensearch-project#12864)

Signed-off-by: Andriy Redko <[email protected]>

…e check at the build time (opensearch-project#12872)

* Added missed API visibility annotations for public APIs and enable the check at the build time

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>

* Make GetTermVersionAction internal

GetTermVersion action has been added to check if cluster state needs to
be updated as a performance improvement on all cluster state checks.
The Authorization systems in the Security Plugin check all actions that
do not start with `internal:` for permissions causing users without
`cluster:monitor/*` permissions to start getting 403 exceptions.

This change signals that this action does not require an authorization
check by any security systems since it cannot be called via REST APIs.

Signed-off-by: Peter Nied <[email protected]>

* Move TermVerson action outside of admin/cluster namespace

Signed-off-by: Peter Nied <[email protected]>

---------

Signed-off-by: Peter Nied <[email protected]>

…project#12876)

* Change from configureEach to all to resolve incompatibility

Signed-off-by: Zelin Hao <[email protected]>

* Update Changelog

Signed-off-by: Zelin Hao <[email protected]>

---------

Signed-off-by: Zelin Hao <[email protected]>

…#12895)

Signed-off-by: Craig Perkins <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

…oject#12877)

With this plugin enabled we see significant increase in memory usage.
This is due to a static map that collects Span entries per write/replication request.
Tests with higher counts of independent writes are at risk of running oom or experiencing
slow down / warnings related to this.

Disabled the plugin by default until this is resolved given it is still under FeatureFlag.

Signed-off-by: Marc Handalian <[email protected]>

…rch-project#12746)

First in a series of commits to support derived fields, a form of schema-on-read.

This commit adds:
1. DerivedFieldScript factory: This script factory will be used to execute scripts defined against 
   derived fields of any type.
2. DerivedFieldValueFetcher: The value fetcher contains logic to execute script and fetch the value 
   in form of List<Object>. It expects DerivedFieldScript.LeafFactory as an input and sets the contract 
   with consumer to call setNextReader() whenever a segment is switched.
3. DerivedFieldQuery: This query will be used by any of the derived fields. It expects an input query and
   DerivedFieldValueFetcher. It uses 2-phase iterator approach with approximation iterator set to match 
   all docs. On a match, it creates a lucene MemoryIndex for a given doc, fetches the value of the derived
   field from _source using DerivedFieldValueFetcher and executes the input query against.

---------

Signed-off-by: Rishabh Maurya <[email protected]>

…pensearch-project#12849)

* Fix issue with feature flags passed as system props where default value was not being honored

Signed-off-by: Craig Perkins <[email protected]>

* Add CHANGELOG entry

Signed-off-by: Craig Perkins <[email protected]>

* Add test for default value of false

Signed-off-by: Craig Perkins <[email protected]>

* Fix issue when empty settings passed in initialization

Signed-off-by: Craig Perkins <[email protected]>

* Get actual value from settings and default from ff setting

Signed-off-by: Craig Perkins <[email protected]>

* Add test with non-empty setting initialization

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>

Signed-off-by: Andriy Redko <[email protected]>

…in /plugins/repository-hdfs (opensearch-project#12896)

* Bump org.apache.commons:commons-configuration2

Bumps org.apache.commons:commons-configuration2 from 2.10.0 to 2.10.1.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-configuration2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Peter Nied <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Nied <[email protected]>

…s-fixture (opensearch-project#12893)

* Bump net.minidev:json-smart in /test/fixtures/hdfs-fixture

Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:json-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Peter Nied <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Nied <[email protected]>

…ct#12822)

Signed-off-by: Ashish Singh <[email protected]>

…ject#12920)

Signed-off-by: Ashish Singh <[email protected]>

* Fix flaky test CopyProcessorTests#testCopyWithRemoveSource

Signed-off-by: Gao Binlong <[email protected]>

* Remove an existing field to get a non-existing field name

Signed-off-by: Gao Binlong <[email protected]>

* Add a new randomFieldName method

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>

)

Signed-off-by: Andriy Redko <[email protected]>

Signed-off-by: kkewwei <[email protected]>

… from idle to non-idle (opensearch-project#12768)



---------

Signed-off-by: Ruirui Zhang <[email protected]>

* Fix Flaky SettingTests Tests

Signed-off-by: kkewwei <[email protected]>

* add the comment to unit test

Signed-off-by: kkewwei <[email protected]>

---------

Signed-off-by: kkewwei <[email protected]>
Co-authored-by: kkewwei <[email protected]>

…ons (opensearch-project#12697)

Signed-off-by: Jay Deng <[email protected]>

* Convert ingest processor supports ip type

Signed-off-by: Gao Binlong <[email protected]>

* Modify change log

Signed-off-by: Gao Binlong <[email protected]>

* Add comment

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>

…check for resize APIs (opensearch-project#12921)

Signed-off-by: Gao Binlong <[email protected]>

…ect#12865)

…vedFieldType (opensearch-project#12808)

Implementation for all supported types and DerivedFieldType. 

We support the following types so far:

boolean
date
geo_point
ip
keyword
long
double

---------

Signed-off-by: Rishabh Maurya <[email protected]>

…elated settings (opensearch-project#12838)

* Add RemoteStoreSettings class to handle remote store related settings
---------
Signed-off-by: Sachin Kale <[email protected]>
Co-authored-by: Sachin Kale <[email protected]>

…pensearch-project#12907)

Signed-off-by: Andriy Redko <[email protected]>

…arch requests (opensearch-project#12954)

Signed-off-by: Jay Deng <[email protected]>

…2.x (opensearch-project#12972)

…ry-hdfs (opensearch-project#12996)

* Bump commons-io:commons-io in /plugins/repository-hdfs

Bumps commons-io:commons-io from 2.15.1 to 2.16.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

Constant keyword fields behave similarly to regular keyword fields, except that they are 
defined only in the index mapping, and all documents in the index appear to have the same 
value for the constant keyword field.

---------

Signed-off-by: Mohammad Hasnain <[email protected]>

…eld (opensearch-project#12048)

The "exists" query on an object field would fail when a "search_as_you_type" field is nested under that object.

It would also fail for a text field with prefixes enabled nested under the object, or any other field with a 
"hidden" subfield.

---------

Signed-off-by: Mrudhul Guda <[email protected]>

We need to update our PR template to suggest that people add things to
"unreleased 2.x" unless they definitely want to leave things for 3.0.

Signed-off-by: Michael Froh <[email protected]>

…ensearch-project#12656)

Signed-off-by: Arpit-Bandejiya <[email protected]>

…search-project#12986)

Signed-off-by: Ashish Singh <[email protected]>

…arch-project#12994)

* Memoize isOnRemoteNode in index settings and rename it as well
---------
Signed-off-by: Gaurav Bafna <[email protected]>

…cation during Remote Store Migration (opensearch-project#12821)

Signed-off-by: Shourya Dutta Biswas <[email protected]>

…ilure (opensearch-project#13019)

Signed-off-by: Gaurav Bafna <[email protected]>

…project#12938)

* Add more identifiers to gradle-check job webhook payload

Signed-off-by: Rishabh Singh <[email protected]>

* Update .github/workflows/gradle-check.yml

Co-authored-by: Peter Nied <[email protected]>
Signed-off-by: Rishabh Singh <[email protected]>

* Add more identifiers to gradle-check job webhook payload

Signed-off-by: Rishabh Singh <[email protected]>

---------

Signed-off-by: Rishabh Singh <[email protected]>
Signed-off-by: Rishabh Singh <[email protected]>
Co-authored-by: Peter Nied <[email protected]>

…nsearch-project#12938)" (opensearch-project#13027)

This reverts commit a6bbc09.

Signed-off-by: dblock <[email protected]>

…oject#13030)

Signed-off-by: Andriy Redko <[email protected]>

…-project#12569)

Adds a DerivedFieldMapper to support the Derived Fields feature enhancement as well as updating the mapper parsing logic to recognize and currently parse derived fields in the mappings.

---------

Signed-off-by: Mohammad Qureshi <[email protected]>
Signed-off-by: Rishabh Maurya <[email protected]>
Co-authored-by: Rishabh Maurya <[email protected]>

…ttachment (opensearch-project#12998)

* Bump commons-io:commons-io in /plugins/ingest-attachment

Bumps commons-io:commons-io from 2.15.1 to 2.16.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

…y-azure-classic (opensearch-project#12999)

* Bump commons-io:commons-io in /plugins/discovery-azure-classic

Bumps commons-io:commons-io from 2.15.1 to 2.16.0.

---
updated-dependencies:
- dependency-name: commons-io:commons-io
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

…mic setting (opensearch-project#12704)

Signed-off-by: Sachin Kale <[email protected]>

Signed-off-by: Shivansh Arora <[email protected]>