Automatic geo-failover in case of service outage in one region

[gmantri]

Currently we have models deployed in East US (2), East US and South Central US but our application connects to the models deployed in East US (2) region only.

Recently we have seen many short outages in East US (2) regions (one was earlier today) and because of that our entire application went down.

I am wondering if there is something in Semantic Kernel, that would allow us to specify alternate regions and have SK does automatic failover and connect to another region in case the primary region goes down.

To give you an analogy, Azure Storage SDK had this feature where you can simply enable reading from secondary for RA-GRS accounts and the SDK will automatically switch from primary to secondary in case it fails to connect to primary under certain circumstances.

If automatic geo-replication is not natively possible (and there is no plan to bring this feature in), I am wondering what would be the best way to implement such feature inside our application code. What error codes we should look out for that would tell us that the region is down. We have seen service return us different error codes back (like 401, 403, 500 etc.) when the service is unavailable.

Any insights into this will be highly appreciated.

[dmytrostruk]

Hi @gmantri , thanks for your question!
We don't have automatic geo-failover in Semantic Kernel, but we have a couple of examples how to implement such feature on your side:

• https://github.com/microsoft/semantic-kernel/blob/f330f809eb2da98b3f22e0131530fbede7332de2/dotnet/samples/Concepts/Filtering/RetryWithFilters.cs
• https://github.com/microsoft/semantic-kernel/tree/main/dotnet/samples/Demos/AIModelRouter

Please let me know if these examples are helpful, thanks!