[New Feature]: Expand third party connectors

[chathuranga-jayanath-99]

Description

This feature involves implementing new connectors for key communication protocols. The identified connectors to be implemented are as follows:

• HTTP
• JMS
• MQTT
• WebSocket
• AMQP
• HL7

Describe your problem(s)

No response

Describe your solution(s)

No response

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

[chathuranga-jayanath-99]

Based on our initial discussion, we will begin by implementing the HTTP connector. In this phase, the connector will support a single operation for sending requests.

• Connection Creation: The URL path will be retrieved during the connection setup.
• Request Execution: The following parameters will be collected when sending a request: HTTP method type, headers, path parameters, query parameters, and the request body.

[chathuranga-jayanath-99]

The connection initiation and request sending operations have been implemented with the following features.

The connection is established using the following:

• url

In the send request operation, the following inputs are taken:

• method - can be: get, post, put, patch, delete, head, options
• headers - a comma separated list (h1: v1, h2:v2)
• path parameters - a comma separated list (p1: v1, p2:v2)
• query parameters - a comma separated list (q1: v1, q2:v2)
• request body type - can be: json/xml/text
• request body
• properties
  • POST_TO_URI
  • FORCE_SC_ACCEPTED
  • DISABLE_CHUNKING etc

Note: Currently, headers, path parameters, and query parameters accept a comma-separated list. However, once the feature from Issue #413 is available, we plan to switch to using that feature to handle these lists.

[chathuranga-jayanath-99]

As per the discussion we agreed to improve on following things

• introduce operations for GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS instead of sendRequest operation.
• handle authentication attributes.
• handle error handling attributes.
• handle payload in three ways: use payload from incoming request, get from a property, get inline.
• determine which HTTP transport properties should be included in the form and add only those.

[chathuranga-jayanath-99]

I implemented the following changes:

• Added new operations: Introduced support for GET, POST, and PUT HTTP methods.
• Handled authentication attributes: For OAuth attributes, there was a list called requestParameters to be maintained. In order to handle that as agreed the parameters were first written into the message context with a unique prefix, and then they were added to the requestParameters list while creating the endpoint.
• Introduced error handling attributes: Started adding attributes related to error handling. We need to introduce expression support for these attributes.

[chathuranga-jayanath-99]

Last week summary.

• Introduce expression support for all attributes and implement the corresponding test cases.
• Implement test cases for the functionalities developed for the HTTP connector.
• Discussed handling certificates required for HTTPS connections, and it was agreed to add the certificates to the existing keystore in WSO2 MI. This effort will include the following implementations:
  • Allow certificates to be stored in a directory within the project.
  • Update the vscode-car-plugin to package the certificates into the CAR file.
  • Develop a new deployer in to retrieve the certificates, add them to the keystore.

[chathuranga-jayanath-99]

Last week summary.

• Bundle the certificates into the C app once it's added to the project.
• Develop a deployer that adds the certificates to the client-truststore.jks.
• Load the updated client-truststore.jks with the newly added certificates into the runtime environment.
• After discussion, it was agreed that we should allow a way to configure the path to a directory containing the certificates as an environment variable.

[chathuranga-jayanath-99]

Last week summary.

• Agreed Approach for Handling Certificates: Certificates added through the connector will automatically create an entry in the config.properties file. So that it will considered as a config property when resolving a value.
• Looked into how we can achieve MTLS with MI and improvements required on the connector side to support MTLS.
• During deployment, values in the config.properties will be resolved, after which certificates will be loaded into the keystore. These certificates will then be reloaded into the keystore for use at deployment.
• Add the MI server-side changes with an improved version of reloading added certificates.

[chathuranga-jayanath-99]

Last week summary.

• Worked on UI updates for the HTTP connector, specifically focusing on the process of adding certificates during connection creation.
• Considered two methods for certificate addition: via a file path or as a configurable. Making an attribute configurable requires additional entries in supplementary files (config.properties file and .env file).
• After discussions, we decided to allow certificates to be uploaded to the project directly while also providing the configurable option.
• Additionally, we agreed to enhance the overall UX, will start working on them.

[chathuranga-jayanath-99]

Last week summary.

• Worked on improving the UX experience by addressing scenarios where:
  • Users change an added certificate.
  • Users attempt to reuse an existing certificate.
  • Unused certificates get deleted automatically when users change them.
• In the process, we identified a need to track the usage of each certificate. There a concern raised about the need of capturing the usage of all the resources in the project.
• Worked on implementing a basic way to capture the usages of the resources in the micro-integrator-language-server.
• During discussion on where to locate the added certificates, it was agreed that the having registry resources explorer to manage resources in the project is not intuitive. Therefore worked on planning how we can change registry resources explorer.