You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a sensor is frequently triggered, it results in over X requests per second (rq/s) to the Schema Registry. This level of traffic is unnecessary and causes performance issues in a shared Schema Registry, leading to degraded functionality. While the library used for schema management is capable of caching, the cache is not utilized effectively due to individual triggers for each event.
To Reproduce
Steps to reproduce the behavior:
Create a kafka sensor
Trigger the kafka sensor
Monitoring schema registry requests
Expected behavior
The sensor should instantiate and retrieve the schema from the registry only once per sensor activation. This approach reduces the total number of requests made to the Schema Registry, improving overall performance and efficiency.
Details
The instante SR in kafka.go already doing 1 call and 1 call for get schema, after them seeing it on the riferrei/srclient lib used, cache already implemented by default. The problem here is about many insantiate of trigger called by sensorCtx.triggerActions
Screenshots
Requests per second to SR
Environment (please complete the following information):
Kubernetes: 1.27.13
Argo Events: 1.7.5
Message from the maintainers:
If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.
The text was updated successfully, but these errors were encountered:
Thomasr-02
changed the title
Cache Schema Registry Schema - Kafka Trigger Sensor
Cache Schema Registry Client - Kafka Trigger Sensor
Dec 7, 2024
Thomasr-02
changed the title
Cache Schema Registry Client - Kafka Trigger Sensor
Instantiate SR only once per sensor - Kafka Trigger Sensor
Dec 10, 2024
Describe the bug
When a sensor is frequently triggered, it results in over X requests per second (rq/s) to the Schema Registry. This level of traffic is unnecessary and causes performance issues in a shared Schema Registry, leading to degraded functionality. While the library used for schema management is capable of caching, the cache is not utilized effectively due to individual triggers for each event.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The sensor should instantiate and retrieve the schema from the registry only once per sensor activation. This approach reduces the total number of requests made to the Schema Registry, improving overall performance and efficiency.
Details
The instante SR in kafka.go already doing 1 call and 1 call for get schema, after them seeing it on the
riferrei/srclient
lib used, cache already implemented by default. The problem here is about many insantiate of trigger called by sensorCtx.triggerActionsScreenshots
Requests per second to SR
Environment (please complete the following information):
Message from the maintainers:
If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.
The text was updated successfully, but these errors were encountered: