How scaling by requests works?

[coopsmoss]

Hi there, I'm just looking for a little bit of clarification regarding scaling based on requests as seen here

The docs say "Scale up or down based on the request count handled per tasks."
I interpret that to mean simultaneous requests per instance, so I was planning to set it to something quite low like 3,4,5 (because our requests take somewhat long to process) but the example docs has a huge number:

count:
  range: 1-10
  cpu_percentage: 70
  memory_percentage: 80
  requests: 10000
  response_time: 2s

It has 10,000 which makes me wary that I'm misunderstanding how this operates. I just don't want to set it too low and then have my application scale to the max right away. Is it 10,000 per minute/hour/etc?

So just looking for some clarification on that. Thanks a lot in advance.

[efekarakus]

I interpret that to mean simultaneous requests per instance

Apologies for the confusing verbiage, it represents the average number of requests received by each task. So for example, if you specify requests: 5 that means you expect every minute for an ECS task to handle on average 5 requests.

I just don't want to set it too low and then have my application scale to the max right away. Is it 10,000 per minute/hour/etc?

It is per minute! So the load balancer will track on average how many requests per minute an ECS task processes and ECS will autoscale to maintain your specified number. Setting it to a low number like 5 is totally reasonable if your responses take on average 12s for example.

[coopsmoss]

Thanks for the response, that makes a lot of sense. Cheers.

[tdsticks]

Can you also help explain further on cpu_percentage and memory_percentage please?

"Scale up or down based on the average CPU/memory your service should maintain". If these are set to a higher percentage, does the service in ECS attempt to keep more tasks running to match the average?

I've been looking for further documentation from AWS and I found this document https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-scaling-target-tracking.html, but I it's still a little unclear.

[efekarakus]

If these are set to a higher percentage, does the service in ECS attempt to keep more tasks running to match the average?

Your understanding is correct!
So for example, let's say you specified:

count:
   range: 1-100
   cpu_percentage: 70

Every minute, ECS will compare the average CPU utilization across all your tasks in your service. If your tasks are at a higher CPU utilization, then it will add new tasks in order to maintain the desired 70% cpu utilization up to a maximum of 100 tasks.
Similarly, if your tasks are at a lower CPU utilization, after 2 minutes, then it will slowly remove tasks in order to maintain 70% cpu with a lower bound of 1 task.

Hope this helps!

[tdsticks]

That makes sense. Thank you for help clarifying!