Main repository azure managed identity support

[chengwushi-netapp]

Description

This PR added supported for managed identity in the repository-azure plugin.

Related Issues

Resolves #12423

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 9d0d565: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for a330e99: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

Compatibility status:

Checks if related components are compatible with change 3cb3c7e

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/sql.git]

[chengwushi-netapp]

This PR is expected to fail the Task :plugins:repository-azure:thirdPartyAudit because I am uncertain about the best approach to resolve this failure.

From my understanding, the thirdPartyAudit requires all dependencies to be explicitly declared. For instance, I have the following dependency tree from using com.azure:azure-identity:1.11.2.

com.azure:azure-identity:1.11.2
----net.java.dev.jna:jna-platform:5.14.0
----com.microsoft.azure:msal4j-persistence-extension:1.2.0
----com.microsoft.azure:msal4j:1.14.2
--------com.nimbusds:oauth2-oidc-sdk:11.10
------------com.nimbusds:nimbus-jose-jwt:9.37.3
------------com.nimbusds:content-type:2.3
------------com.nimbusds:lang-tag:1.7
----net.minidev:json-smart:2.5.0
--------net.minidev:accessors-smart:2.5.0
------------org.ow2.asm:asm:9.6

Note: For simplicity, I have not listed all the nested dependencies, as it would make the tree too large for demonstration purposes.

I believe to pass the Task :plugins:repository-azure:thirdPartyAudit check, I would need to declare all these nested dependencies, not just a subset of them.

Therefore, I have the following questions, and I would greatly appreciate it if anyone could provide answers:

1. Do i need to include all the nested dependencies?
   1.1 If the answer is yes, wouldn't the list of dependencies become too large? What happens if one of them contains vulnerabilities?
   1.2 If the answer is no, how do I determine which dependencies I can ignore?
2. Is there a way to automate the addition of nested dependencies to the build file? I am currently adding them manually by referencing mavenCentral.
3. Are there any best practices or recommended approaches when dealing with complex dependency trees when adding new dependencies in OpenSearch?
4. I am aware of the ignoreMissingClasses in the thirdPartyAudit check. How do we determine if we can ignore a missing class found in the thirdPartyAudit check?
5. When adding a new dependency, I was under the impression that Gradle would automatically fetch its nested dependencies. However, in my experience while testing this PR on an Azure Virtual Machine with an attached managed identity, I found that I had to manually install three nested dependencies, even though I had already included com.azure:azure-identity:1.11.2. The dependencies I had to manually add were com.microsoft.azure:msal4j:1.14.2, com.nimbusds:oauth2-oidc-sdk:11.10, and net.minidev:json-smart:2.5.0. Could anyone clarify why these dependencies weren't automatically fetched by Gradle?

[github-actions]

✅ Gradle check result for 280dd0a: SUCCESS

[chengwushi-netapp]

@andrross thank you for your review, i have responded to all your comments. However, this PR would need approvals from @andrross and @reta again, because I have rebased the branch to pass the tests. Cheers, chengwu