Queue plugin discussion

[richthegeek]

Hi, I wanted to open a discussion on the mertis, drawbacks, and technical implications of adding a Queue system on a route level.

As an example, the route has some additional config on API creation:

...,
"queue": {
  "concurrency": 20,
  "retries": 3, // 0 is try only once, -1 is try indefinitely
  "retryDelay": 60000, // delay between each attempt in milliseconds,
}
...

And then when a request is verified and routed, instead of immediately being passed to the registered endpoint is put into a Queue to be attempted and retried.

Retry behavior can be based on either the status of the request (non-2xx indicates a retry) or on a response header (x-annon-retry = 1 or 0)

It would require a single Queue provider such as RabbitMQ (may require 'delayed-message' exchange plugin) or Redis - perhaps something like Honeydew would be a satisfactory dependency.

Benefits of queueing requests:

1. Routes have an inherent rate limit (at least per-worker) determined by the queue concurrency
2. Route implemntors can simplify their own logic - rather than implemnting their own queues or retry behavior they can "just fail" in the knowledge that the request will be retried.
3. Route load should be more stable in bursty traffic

I'm wondering if an intermediary like this is supported/supportable by the existing plugin system. If so, I'm happy to work on it, as it's the only thing stopping me from using Annon currently!

[AndrewDryga]

Hello @richthegeek,

Can you tell more about your use case for this feature?

I've considered building it in the past but then decided that retries should be pushed closer to the client (implemented in client API library). This way you implement them once and they cover the most unreliable part of connection (between the client and your hosting environment).

Other cases are when a client does not care about the execution of a sent request, it wants to fire it and forget. In those cases, it looks like you want API gateway to run background jobs for you, which is not a concern that should be part of API gateway. I think this service can be built stand-alone and even drafted some ideas for myself a while ago: Nebo15/webhooks_service#1.