feat: return # of messages prioritized in the queue when calling message retry endpoint

[kamiyaa]

Description

Add a mpsc channel between relayer and message retry handler, so message retry handler knows the retry results from the relayer.

POST /message_retry endpoint now returns a JSON with details on the retry request:

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct MessageRetryResponse {
    /// ID of the retry request
    pub uuid: String,
    /// how many pending operations were processed
    pub processed: usize,
    /// how many of the pending operations matched the retry request pattern
    pub matched: u64,
}

New changes

• use less memory by dropping uuid for some responses
• changed channel size to 3 * number of destination chains
  • before it was just number of origin chains
  • this should prevent Senders from being blocked when calling send()
• added unittests to test process_retry_requests to ensure messages are being matched properly against MatchingList
• use Arc for message retry requests so we don't need to clone the entire struct
• add constant SUBMITTER_QUEUE_COUNT = 3

Drive-by changes

Related issues

Fixes #4059

Backward compatibility

Testing

Updated unit tests to return a response so the handler is not blocked

[changeset-bot]

⚠️ No Changeset found

Latest commit: f74aca4

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.53%. Comparing base (28becff) to head (f74aca4).
Report is 12 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5132   +/-   ##
=======================================
  Coverage   77.53%   77.53%           
=======================================
  Files         103      103           
  Lines        2110     2110           
  Branches      190      190           
=======================================
  Hits         1636     1636           
  Misses        453      453           
  Partials       21       21           

Components	Coverage Δ
core	87.80% <ø> (ø)
hooks	79.39% <ø> (ø)
isms	83.68% <ø> (ø)
token	91.27% <ø> (ø)
middlewares	79.80% <ø> (ø)

[tkporter]

haven't looked at any logic just a driveby

[tkporter]

can we add a comment as to why this is important? is there any way we can make sure that e.g. tests fail if this ever becomes stale?

[kamiyaa]

Yep, can add comment. Technically there is a way to test this with capacity() and permits
https://docs.rs/tokio/latest/tokio/sync/mpsc/struct.Sender.html#method.capacity

But I think that would require us to refactor OpSubmitter in a way that is easily testable via unittests