Support concurrent invocations of different docker replicas

[sterliakov]

Hi! Thank you for this great solution. I needed to support at least two backend replicas, and noticed that it's really easy to support inside local-api-gateway container. Would you be interested in this contribution? A POC version is in the main branch of my fork, but I'm ready to clean up code there and sync only relevant changes - please let me know if it looks useful enough.

Building on top of your docs example, my suggested changes allow the following:

services:
    # This container runs API Gateway locally
    web:
        image: sterliakov/local-api-gateway
        ports: ['8000:8000']
        environment:
            # <host>:<port> -> the host here is "php" because that's the name of the second container
            TARGET: 'php:8080'
            TARGET_REPLICAS_COUNT: 3

    # Example of container running AWS Lambda locally
    php:
        image: bref/php-80-fpm
        # The command should contain the Lambda handler
        command: public/index.php
        volumes:
            - .:/var/task:ro
        deploy:
            mode: replicated
            replicas: 3

Due to aws/aws-lambda-runtime-interface-emulator#97, it needs to monitor the server allocation, but the code is quite trivial. TypeScript is not my native backend language, so sorry, it may need some cleanup afterwards.

I'm opening this ticket to clarify whether this idea aligns with your project goals to avoid spending time on PR cleanup which may render unnecessary.

[mnapoli]

Could you detail the use case? I.e. what are some real-world examples of how this can be useful?

[sterliakov]

Sure, sorry. I had to develop this in the following scenario: frontend calls a backend endpoint to set up an external service (MS Graph) webhook subscription. Creating a subscription involves endpoint validation, performed in a blocking fashion: MS creates a subscription and returns only when the endpoint responds with some appropriate content. So, the first replica is already running, serving the frontend request, and the validation request is stuck in the queue until first replica fails to set everything up due to lack of response. This requires an ability to serve more than one request at a time. (Backend is proxied via ngrok to the public HTTPS url which is used as a webhook destination)

As a bonus, this greatly improved localhost performance of my frontend on pages where several heavy requests are performed.

(sorry, nodejs/TS is not my mother tongue for backend, so the code I linked must be suboptimal - I'll clean it up if you want to incorporate these changes)