issue-465 Create a documentation section to use Grafana DataSource with SonataFlow Prometheus metrics #693

jianrongzhang89 · 2024-12-10T09:30:38Z

Fix apache/incubator-kie-kogito-serverless-operator#465

Update the document to include Prometheus and Grafana installation, and Grafana Data Source congfiguration and import the default dashboard.

You have read the contributions doc
Pull Request title is properly formatted: Issue-XYZ Subject
Pull Request title contains the target branch if not targeting main: [0.9.x] Issue-XYZ Subject
The nav.adoc file has a link to this guide in the proper category
The index.adoc file has a card to this guide in the proper category, with a meaningful description

ricardozanini · 2024-12-10T22:56:33Z

@jianrongzhang89 can you please take a look on CI?

github-actions · 2024-12-11T10:23:04Z

🎊 PR Preview f1a88df has been successfully built and deployed. See the documentation preview: https://sonataflow-docs-preview-pr-693.surge.sh

jianrongzhang89 · 2024-12-11T10:23:07Z

@jianrongzhang89 can you please take a look on CI?

@ricardozanini fixed CI errors.

wmedvede

I have followed all the document for the OpenShift installation and worked fine.
See image below with my workflows.

Guide is working.
LGTM

serverlessworkflow/modules/ROOT/pages/cloud/operator/monitoring-workflows.adoc

serverlessworkflow/modules/ROOT/pages/cloud/operator/sonataflow-metrics.adoc

wmedvede · 2024-12-11T14:39:25Z

Would you mind check the procedure for regular Kubernetes clusters? @domhanak

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

serverlessworkflow/modules/ROOT/pages/cloud/operator/monitoring-workflows.adoc

ricardozanini

Many thanks, @jianrongzhang89. This documentation seems good. Thanks, @wmedvede, for verifying the steps in the cluster!

ricardozanini · 2024-12-12T17:50:37Z

@kaldesai mind taking a look too?

wmedvede

Hi @jianrongzhang89 , I couldn't evict adding some more nitpicks when re-reading 😄

wmedvede · 2024-12-13T11:28:28Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+
+In {product_name}, you can check the following metrics:
+
+* `kogito_process_instance_started_total`: Number of started workflows (a workflow that has started might be running or completed)


Suggested change

* `kogito_process_instance_started_total`: Number of started workflows (a workflow that has started might be running or completed)

* `kogito_process_instance_started_total`: Number of started workflows.

wmedvede · 2024-12-13T11:29:14Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+
+* `kogito_process_instance_started_total`: Number of started workflows (a workflow that has started might be running or completed)
+* `kogito_process_instance_running_total`: Number of running workflows
+* `kogito_process_instance_completed_total`: Number of completed workflows


Suggested change

* `kogito_process_instance_completed_total`: Number of completed workflows

* `kogito_process_instance_completed_total`: Number of completed workflows.

wmedvede · 2024-12-13T11:29:37Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+* `kogito_process_instance_started_total`: Number of started workflows (a workflow that has started might be running or completed)
+* `kogito_process_instance_running_total`: Number of running workflows
+* `kogito_process_instance_completed_total`: Number of completed workflows
+* `kogito_process_instance_error`: Number of workflows that report an error ( a workflow with an error might be still running or have been completed) 


Suggested change

* `kogito_process_instance_error`: Number of workflows that report an error ( a workflow with an error might be still running or have been completed)

* `kogito_process_instance_error`: Number of workflows that report an error.

wmedvede · 2024-12-13T11:30:43Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+* `kogito_process_instance_running_total`: Number of running workflows
+* `kogito_process_instance_completed_total`: Number of completed workflows
+* `kogito_process_instance_error`: Number of workflows that report an error ( a workflow with an error might be still running or have been completed) 
+* `kogito_process_instance_duration_seconds`: Duration of a process instance in seconds


Suggested change

* `kogito_process_instance_duration_seconds`: Duration of a process instance in seconds

* `kogito_process_instance_duration_seconds`: Duration of a workflow instance in seconds.

wmedvede · 2024-12-13T11:31:34Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+* `kogito_process_instance_completed_total`: Number of completed workflows
+* `kogito_process_instance_error`: Number of workflows that report an error ( a workflow with an error might be still running or have been completed) 
+* `kogito_process_instance_duration_seconds`: Duration of a process instance in seconds
+* `kogito_node_instance_duration_milliseconds`: Duration of relevant nodes in milliseconds (a workflow is composed by nodes, user might be interested on the time consumed by an specific node type) 


Suggested change

* `kogito_node_instance_duration_milliseconds`: Duration of relevant nodes in milliseconds (a workflow is composed by nodes, user might be interested on the time consumed by an specific node type)

* `kogito_node_instance_duration_milliseconds`: Duration of relevant nodes in milliseconds.

wmedvede · 2024-12-13T11:32:51Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+
+[NOTE]
+====
+Internally, workflows are referred as processes. Therefore, the `processId` and `processName` is workflow ID and name respectively.


Suggested change

Internally, workflows are referred as processes. Therefore, the `processId` and `processName` is workflow ID and name respectively.

Internally, workflows are referred as processes. Therefore, the `processId` and `processName` are workflow id and name respectively.

wmedvede · 2024-12-13T11:33:26Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+Internally, workflows are referred as processes. Therefore, the `processId` and `processName` is workflow ID and name respectively.
+====
+
+Each of the metrics mentioned previously contains a label for a specific workflow ID. For example, the `kogito_process_instance_completed_total` metric below contains the labels for `callbackstatetimeouts` workflow:


Suggested change

Each of the metrics mentioned previously contains a label for a specific workflow ID. For example, the `kogito_process_instance_completed_total` metric below contains the labels for `callbackstatetimeouts` workflow:

Each of the metrics mentioned previously contains a label for a specific workflow id. For example, the `kogito_process_instance_completed_total` metric below contains the labels for `callbackstatetimeouts` workflow:

wmedvede · 2024-12-13T11:41:14Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+----
+
+=== kogito_process_instance_duration_seconds
+Calculates duration of a workflow instance that has reached a terminal state,, i.e. `Aborted` or `Completed`. This metric is registered when the process reaches the terminal state.


Suggested change

Calculates duration of a workflow instance that has reached a terminal state,, i.e. `Aborted` or `Completed`. This metric is registered when the process reaches the terminal state.

Calculates duration of a workflow instance that has reached a terminal state, i.e. `Aborted` or `Completed`. This metric is registered when the process reaches the terminal state.

wmedvede · 2024-12-13T11:43:25Z

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

+* `kogito_process_instance_error`: Number of workflows that report an error ( a workflow with an error might be still running or have been completed) 
+* `kogito_process_instance_duration_seconds`: Duration of a process instance in seconds
+* `kogito_node_instance_duration_milliseconds`: Duration of relevant nodes in milliseconds (a workflow is composed by nodes, user might be interested on the time consumed by an specific node type) 
+* `sonataflow_input_parameters_counter`: Records input parameters, the occurrences of <"param_name","param_value"> per `processId`.


Suggested change

* `sonataflow_input_parameters_counter`: Records input parameters, the occurrences of <"param_name","param_value"> per `processId`.

* `sonataflow_input_parameters_counter_total`: Records input parameters, the occurrences of <"param_name","param_value"> per `processId`.

wmedvede

Hi @RichardW98 a few nitpicks, and I have also re-installed the grafana dahsboard after these last modification.
Is working good, great work!

Just an observation regarding the dashboard, see screenshots please:

In my tests, I have these workflows: callbackstatetimeouts and callbackstatetimeouts-gitops.

The dashboard works fine:

However, in the filters a "greeting" value is shown.

serverlessworkflow/modules/ROOT/pages/_common-content/sonataflow-metrics.adoc

serverlessworkflow/modules/ROOT/pages/cloud/operator/monitoring-workflows.adoc

jianrongzhang89 · 2024-12-21T01:48:31Z

@wmedvede I updated PR based on your above comments. Thanks.

wmedvede

LGTM, thanks!

ricardozanini · 2025-01-09T15:45:21Z

@domhanak mind taking a look so we can close this one?

…ith SonataFlow Prometheus metrics

…ith SonataFlow Prometheus metrics: address review comments

…ow-metrics.adoc Co-authored-by: Walter Medvedeo <[email protected]>

…ith SonataFlow Prometheus metrics: address review comments

jianrongzhang89 requested review from domhanak, ricardozanini and krisv as code owners December 10, 2024 09:30

jianrongzhang89 force-pushed the monitoring branch from dd9e4a9 to 1537dee Compare December 10, 2024 09:47

jianrongzhang89 force-pushed the monitoring branch 2 times, most recently from 3250a02 to d95b765 Compare December 11, 2024 10:22

jianrongzhang89 force-pushed the monitoring branch from d95b765 to 79a31d8 Compare December 11, 2024 10:32

wmedvede approved these changes Dec 11, 2024

View reviewed changes

jianrongzhang89 force-pushed the monitoring branch 2 times, most recently from eb1bac5 to 4892c25 Compare December 11, 2024 21:10

wmedvede requested changes Dec 12, 2024

View reviewed changes

ricardozanini approved these changes Dec 12, 2024

View reviewed changes

jianrongzhang89 force-pushed the monitoring branch from 4892c25 to c406023 Compare December 12, 2024 19:25

wmedvede requested changes Dec 13, 2024

View reviewed changes

jianrongzhang89 force-pushed the monitoring branch 2 times, most recently from 1a39e31 to 80de021 Compare December 20, 2024 01:46

wmedvede requested changes Dec 20, 2024

View reviewed changes

jianrongzhang89 force-pushed the monitoring branch 2 times, most recently from c58e684 to 681b553 Compare December 21, 2024 23:14

wmedvede approved these changes Jan 9, 2025

View reviewed changes

jianrongzhang89 mentioned this pull request Jan 17, 2025

EPIC - Add the possibility to integrate the Operator with Prometheus and Grafana apache/incubator-kie-kogito-serverless-operator#461

Open

3 tasks

jianrongzhang89 added 2 commits January 17, 2025 11:19

[issue 465]Create a documentation section to use Grafana DataSource w…

8c1251b

…ith SonataFlow Prometheus metrics

[issue 465]Create a documentation section to use Grafana DataSource w…

4af7346

…ith SonataFlow Prometheus metrics: address review comments

jianrongzhang89 and others added 2 commits January 17, 2025 11:21

Update serverlessworkflow/modules/ROOT/pages/_common-content/sonatafl…

91c4544

…ow-metrics.adoc Co-authored-by: Walter Medvedeo <[email protected]>

Update serverlessworkflow/modules/ROOT/pages/_common-content/sonatafl…

4d8c636

…ow-metrics.adoc Co-authored-by: Walter Medvedeo <[email protected]>

jianrongzhang89 force-pushed the monitoring branch from 681b553 to e948b21 Compare January 17, 2025 16:22

[issue 465]Create a documentation section to use Grafana DataSource w…

90cf9c5

…ith SonataFlow Prometheus metrics: address review comments

jianrongzhang89 force-pushed the monitoring branch from e948b21 to 90cf9c5 Compare January 17, 2025 16:36

ricardozanini merged commit 4f4c9e6 into apache:main Jan 17, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

issue-465 Create a documentation section to use Grafana DataSource with SonataFlow Prometheus metrics #693

issue-465 Create a documentation section to use Grafana DataSource with SonataFlow Prometheus metrics #693

jianrongzhang89 commented Dec 10, 2024 •

edited by ricardozanini

Loading

ricardozanini commented Dec 10, 2024

github-actions bot commented Dec 11, 2024 •

edited

Loading

jianrongzhang89 commented Dec 11, 2024

wmedvede left a comment

wmedvede commented Dec 11, 2024

ricardozanini left a comment

ricardozanini commented Dec 12, 2024

wmedvede left a comment

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede Dec 13, 2024

jianrongzhang89 Dec 13, 2024

wmedvede left a comment

jianrongzhang89 commented Dec 21, 2024

wmedvede left a comment

ricardozanini commented Jan 9, 2025


		In {product_name}, you can check the following metrics:

		* `kogito_process_instance_started_total`: Number of started workflows (a workflow that has started might be running or completed)

	* `kogito_process_instance_completed_total`: Number of completed workflows
	* `kogito_process_instance_completed_total`: Number of completed workflows.

	* `kogito_process_instance_error`: Number of workflows that report an error ( a workflow with an error might be still running or have been completed)
	* `kogito_process_instance_error`: Number of workflows that report an error.

	* `kogito_process_instance_duration_seconds`: Duration of a process instance in seconds
	* `kogito_process_instance_duration_seconds`: Duration of a workflow instance in seconds.

	* `kogito_node_instance_duration_milliseconds`: Duration of relevant nodes in milliseconds (a workflow is composed by nodes, user might be interested on the time consumed by an specific node type)
	* `kogito_node_instance_duration_milliseconds`: Duration of relevant nodes in milliseconds.

	Internally, workflows are referred as processes. Therefore, the `processId` and `processName` is workflow ID and name respectively.
	Internally, workflows are referred as processes. Therefore, the `processId` and `processName` are workflow id and name respectively.

	Each of the metrics mentioned previously contains a label for a specific workflow ID. For example, the `kogito_process_instance_completed_total` metric below contains the labels for `callbackstatetimeouts` workflow:
	Each of the metrics mentioned previously contains a label for a specific workflow id. For example, the `kogito_process_instance_completed_total` metric below contains the labels for `callbackstatetimeouts` workflow:

	Calculates duration of a workflow instance that has reached a terminal state,, i.e. `Aborted` or `Completed`. This metric is registered when the process reaches the terminal state.
	Calculates duration of a workflow instance that has reached a terminal state, i.e. `Aborted` or `Completed`. This metric is registered when the process reaches the terminal state.

	* `sonataflow_input_parameters_counter`: Records input parameters, the occurrences of <"param_name","param_value"> per `processId`.
	* `sonataflow_input_parameters_counter_total`: Records input parameters, the occurrences of <"param_name","param_value"> per `processId`.

issue-465 Create a documentation section to use Grafana DataSource with SonataFlow Prometheus metrics #693

issue-465 Create a documentation section to use Grafana DataSource with SonataFlow Prometheus metrics #693

Conversation

jianrongzhang89 commented Dec 10, 2024 • edited by ricardozanini Loading

ricardozanini commented Dec 10, 2024

github-actions bot commented Dec 11, 2024 • edited Loading

jianrongzhang89 commented Dec 11, 2024

wmedvede left a comment

Choose a reason for hiding this comment

wmedvede commented Dec 11, 2024

ricardozanini left a comment

Choose a reason for hiding this comment

ricardozanini commented Dec 12, 2024

wmedvede left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wmedvede left a comment

Choose a reason for hiding this comment

jianrongzhang89 commented Dec 21, 2024

wmedvede left a comment

Choose a reason for hiding this comment

ricardozanini commented Jan 9, 2025

jianrongzhang89 commented Dec 10, 2024 •

edited by ricardozanini

Loading

github-actions bot commented Dec 11, 2024 •

edited

Loading