[DPE-5615] - Manage chain file for requests like that of a CA

[skourta]

Issue

We were not updating the chain.pem file used by the request module to send requests which caused communication cuts in a lot of cases.
Also fixes #448

Solution

Add the new chain to the chain.pem on CA rotation initiated and only delete it if all the nodes are done updating their CA and certs.

Summary of changes

This pull request introduces several changes to improve the handling of CA (Certificate Authority) rotation in the OpenSearch charm. The key changes include enhancements to the CA rotation process, ensuring proper cleanup of old CA certificates, and updating the request CA bundle. Additionally, a new method has been added to handle the completion of CA rotation.

Improvements to CA Rotation Process:

• lib/charms/opensearch/v0/opensearch_base_charm.py: Added checks to ensure CA rotation is complete in the cluster before removing old CAs and updating the request bundle (on_tls_conf_set, post_start_init, _on_update_status). [1] [2] [3]
• lib/charms/opensearch/v0/opensearch_tls.py: Introduced new methods ca_and_certs_rotation_complete_in_cluster and on_ca_certs_rotation_complete to handle the completion of CA rotations and update the request bundle accordingly. [1] [2]

Version Updates:

• lib/charms/opensearch/v0/opensearch_base_charm.py: Incremented LIBPATCH version from 2 to 3.
• lib/charms/opensearch/v0/opensearch_tls.py: Incremented LIBPATCH version from 1 to 2.

Unit Test Enhancements:

• tests/unit/lib/test_opensearch_base_charm.py: Updated unit tests to mock new methods and ensure they are called correctly during the CA rotation process. [1] [2]

Code Cleanup:

• lib/charms/opensearch/v0/opensearch_base_charm.py: Removed redundant calls to update_request_ca_bundle and remove_old_ca in _start_opensearch and _post_start_init. [1] [2]

Dependency Updates:

• lib/charms/opensearch/v0/opensearch_tls.py: Added import for Path from pathlib.