Clarity on Scaling Policy

[probably-not]

On line 30 of the lambda, the desired count of the fallback service is set to the desired count of the primary service.

Wouldn't this over-provision the fallback service and the amount of tasks?
For example, if I have my SPOT service with 3 tasks running, and the desired count is 4, the fallback service doesn't need to raise 4 tasks, it would just need to raise 1, so that the overall count is 4, as opposed to 7 (4 on the fallback, 3 on the SPOT).

[misterjoshua]

Yes, indeed it will. The current design calls up as many on-demand tasks as possible when the spot service fails to place a task. The idea is that if there's no spot capacity available, the other tasks are at higher risk of being terminated and also failing to place when recreated by the spot service. When this kind of thing happens, there may be a run on on-demand capacity and risk increases for the on-demand tasks failing to place. The policy mitigates these risks by spinning up all on-demand capacity as soon as possible.

However, if you'd like to make this behaviour configurable, I welcome a PR.

[probably-not]

Got it, that makes sense to me. Out of curiosity, how does this play with auto-scaling policies on each of the services?

[misterjoshua]

We're not using it for an auto-scaled service, but it should set the fallback service to the desired count of the primary service at the time of any placement failure. So, when your autoscale increases/decreases your primary service's desired count, that desired count will be carried over to the fallback service at the time of a task placement failure.

This may not be desirable because the autoscaling policy on the primary service isn't monitoring the fallback service but it's controlling the desired count on the fallback service based on the status of the primary service. To support this case, I think the fallback policy would need to activate/deactivate an autoscaling policy on the fallback service.