Add support for external authentication programs

[a-ilin]

Hi! Could you please take a look?

This PR adds a feature of delegation of the authentication process to an external program of user choice. This functionality improves the script usability when then EMail proxy is running on a headless server.

The PR includes a working example of a script implementing the authentication process for Device Authorisation Grant flow by sending the parameters to a gotify server specified by user. The example script can be configured via environment variables or by editing the included config file.

For the communication mechanism between the proxy and the script, the standard input and output streams are being used with the data encoded in JSON format. The stdin/stdout were selected instead of direct program arguments to mitigate possible size limit of the command line argument length for authentication URLs. According to POSIX, the command line argument minimum size is 4096 bytes (xargs --show-limits).

[simonrob]

Thanks for the contribution.

I've taken a look, and if I'm honest, my gut reaction is that this is currently a little outside of the intended scope of the proxy. That hasn't always stopped new features being merged (e.g., see the external secrets manager integration), but as things stand this feels both quite niche and also too specific to a single OAuth flow (DAG).

Could you explain a bit more about how you see this working for other OAuth 2.0 flows? For example, with the standard interactive authorization code flow, is the user expected to develop a program/script to handle enclosing the URL into the correct JSON format? (as an aside, why bother with JSON here at all since the result is just a single URL string?)

Could this be broader and much simpler – for example, just sending a web request to your server URL with a JSON payload and long timeout. Your server would simply return the authorisation string once you've handled everything on the remote side. That could all be done within the proxy script itself, and seems to me like it would abstract away any/all of the complexity.

[a-ilin]

Hi! Thanks for the feedback!

Could you explain a bit more about how you see this working for other OAuth 2.0 flows?

My proposal within this PR is to use an executable program or script to exchange the authentication data. The idea is to offload the interactive authentication procedure to a separate program, whether it is a simple script, or something more complex. For example, for non-DAG flows a user may implement a solution based on headless Chromium to intercept the redirect URI.

My primary use case includes the DAG flow, however, I attempted to design the interactions suitable for other flows as well, therefore the program output includes optional response_url parameter.

For the external program the authentication call sequence is as follows:

1. The client connects to the email proxy.
2. The proxy calls external program providing the needed authentication data, as well as timeout, and blocks until the external program returns.
3. The external program conducts the authentication in some way, and returns the result to the email proxy.
4. The email proxy resumes operation.

For different OAuth 2.0 flows the step (3) will be different, of course. For example, for the DAG flow I configured, it sends the request to the external notification service (gotifier instance in my case), which in it's own course sends a message to a user device, where the user completes the authentication procedure.

is the user expected to develop a program/script to handle enclosing the URL into the correct JSON format?
why bother with JSON here at all since the result is just a single URL string?

The external script accepts JSON, as there are several parameters required for authentication. I decided to use JSON for the return parameters as well, to keep the approach uniform. Using JSON may be beneficial for future extensions, in case of some authentication mechanisms will require more than one return parameter.

Besides, in Python the dictionaries can be converted in JSON quite easily.

Could this be broader and much simpler – for example, just sending a web request to your server URL with a JSON payload and long timeout.

The exact implementation and the request format depend on the service conducting the actual authentication. I decided to avoid direct HTTP call to the remote endpoint from the proxy script, as there are different servers user may decide to use. In my example it is gotify, however there are other notification services suitable for the DAG flow. Here is a couple of others:

• ntfy
• PushBits

The removal of this abstraction may lead further extensions of the email proxy script be more complex, when another notification mechanism will be required.

Another solution, involving deploying an additional intermediate HTTP proxy between the email proxy and the actual notification service should work as well, however this solution leads to higher amount of implementation and maintenance in comparison with the provided example script. In particular, the function of the HTTP proxy will be re-sending the data further to another service, which instead can be implemented with a direct curl call from a script.

[simonrob]

Thanks for the extra detail and justification.

I'm afraid I'm still not convinced that this wouldn't be simpler if the curl call in your external script was changed into a request within the proxy itself. I don't really want to take on the maintenance of additional external tools/services I don't use myself, though am happy to add extra command line options to have the proxy request an external URL (with a specific header for authentication where needed). If I understand correctly, this would achieve the same thing, but in a less-complicated manner. We could point to this PR as an example of how gotify or alternatives might be used here, but wouldn't need these scripts as part of the proxy itself.

I only had a quick look at ntfy, so may have missed something, but it looks like this is a one-way notification, so wouldn't work here outside of the DAG flow. And actually, if I'm not mistaken, the same seems to be the case in your own example and that of PushBits. Certainly the diagram on the gotify documentation homepage makes it seem like this: applications can only send messages and clients can only receive. If that's really the case, it wouldn't be any use for the interactive flow (which requires the code to be returned), so I'd certainly want to simplify as much as possible given it's DAG-specific.

You've made changes in the proxy to handle returned authorisation codes, so perhaps I'm misunderstanding here, or missing where/how this actually happens?

[a-ilin]

You are right, gotify, and other services I mentioned are send-and-forget services for notifications delivery. This perfectly suits the DAG flow. For the flows requiring a return value a further development is needed with a more complex solution than a Bash script sending a notification. Such kind of solution may be based on a program invoking headless Chromium and intercepting the redirect URI.

From my point of view the architecture involving an additional HTTP-based proxy service with an additional authentication mechanism is a more complicated one in comparison to the curl execution. Unlike a script calling curl an additional service requires development efforts, monitoring runtime and security updates (assuming curl is being updated with the OS).

Currently an e-mail forwarder within my home lab is based on the DAG flow with the script included into this PR. Therefore other flows than the DAG have lesser interest from my side.

We could point to this PR as an example of how gotify or alternatives might be used here

Certainly, please feel free to add the link to the documentation!

[simonrob]

Thanks for confirming. I think given how specific this change is to the DAG flow, which is already supported (thanks to you! :-) and how much extra complexity it adds, I won't merge as-is. But it's great to have as an example of how the proxy can be extended to further customise its behaviour in specific setups. I will point to this PR from the readme to help others with that. I'll also leave it open for a while in case others reading this see a way to better support other flows whilst keeping the additional complexity low.