About resources consumption :)

[thibpoullain]

Thx A LOT for Ruby Async, I'm starting to use it for my clients !

I was wondering about resource consumption for a lot of external API call ;) We already have memory trouble and I'm looking for a way to handle it.

I know it's a very large (and may be silly) question but I would need to know the resource behavior of Async in case of high volume async call.

And as a side question, the same question about using jobs and async in parallel :)

Thx a lot for your help :)

[ioquatix]

This is a great question and the solution is as you can imagine highly context specific.

My general approach to this problem is to build naturally scalable systems, and the most important technical concept is "back pressure". If you schedule jobs in a single event loop, then event loop will eventually become saturated. As the event loop becomes saturated, it naturally slows down. This is one way to achieve back-pressure with an event loop. However, that does not mean you can't implement pathological cases, e.g. "fork bombs" and other such cases.

In any system like this, external resources may not provide sufficient back pressure. Especially as Async is designed to scale up without limitation by default. You may risk over-saturating other systems because they do not respond well to usage at scale. There are also some natural limits in Ruby or your hardware which you may run up against, e.g. garbage collection, memory, processor, disk space, etc. Building your system to respond to these limitations naturally will ensure the most robust scalability.

An example of natural back pressure would be as you use more space on your hard drive, allocating new sectors gets slower. An example of an artificial back pressure would be failing a request if there is not more than a certain amount of disk space available. An example of where this might impact an application would be logging. Eventually you have to decide what the failure case should look like, if the disk is full. Does your system grind to a halt, fail outright, or something else?

Async::Semaphore provides a natural mechanism for implementing an artificial resource limit. It requires you explicitly decide how many fibers can user a particular resource. But like any kind of mutual exclusion, it can cause deadlock. They also limit scalability, so they must be used with caution.

Some Async components naturally scale according to the public limits, for example Async::HTTP respects HTTP/1's connection count and HTTP/2's connection multiplexing. But this might not be sufficient for your use case as Async::HTTP might not enforce these limits if you connect to multiple hosts, or the host doesn't publish limits, or you simply bypass them explicitly.

Therefore, it would be useful to understand in more detail the memory trouble you are having. Can you explain in more detail exactly the challenge you are facing and then we can assess what kind of scalability approach makes the most sense. In addition, while at the moment I'm focusing my energy on core Ruby interfaces, going forward we will have more options building on Async for background job processing, batch processing, etc. So the solution space might not be ready yet but at least it helps me understand the specific problems people are facing and informs my decisions going forward.

[thibpoullain]

Thx for your answer !!

I'll investigate about our memory consumption and I'll try to implement Async as an alternative and benchmark it. It'll be a good study case I think ! Any advice about how I should benchmark things ?

Thx again !

[ioquatix]

Are you specifically interested in memory usage?