feat: use files to manage secrets

[FoseFx]

Description

As a user, I want to use files to manage secrets, so that I can load SSL certificates from disk.

As far as I can tell, currently, only vault kv, and the environment are supported secret stores.

For large secrets, which may change on occasion, like SSL certificates and keys, I'd like to see file:// secrets.

Basic Proposal

Simply use the content of the (absolute?) path provided, instead of, for example, env.

ssls:
  - sni: localhost
-    cert: $env://SSL_CERT_LOCALHOST"
+    cert: $file:///etc/ssl/localhost.crt"
-    key: "$env://SSL_KEY_LOCALHOST"
+    cert: $file:///etc/ssl/localhost.key"

JSON

Additionally, it makes sense to use the established syntax for env secrets to access subkeys, if the file contains JSON.

file:///secret.json/apiKey

---

Possible Roadblocks

Permissions. Ideally, we want to ensure, that whoever is able to configure a key, must ensure they have the permissions to access the file.

---

Please let me know if there is interest for this to come to APISIX, and if this seems like something an external contributor (me) could reasonably implement.

[moonming]

Using local files to save secrets will bring security risks. How to solve this problem?

[FoseFx]

Can you elaborate which security risks you are worried about?

If APISIX runs as a lower privileged user, it already can not access privileged files, like /etc/passwd.

We have to make sure users, which do not have access to a file, can't abuse APISIX and then, for example, set the file contents as a response header, and thus stealing the file content.
Again, some of the protection come from the OS here, if APISIX does not have access to the file, this can't happen.
But there may be files APISIX should have access to, but someone else should not. In that case, they should not have access to the admin api / config file.
I'm unfamiliar with the security model of the admin api, but my intuition is, that someone with admin api access is able to do everything APISIX could do.

As a precaution, we could force APISIX to only load files owned by the same user the APISIX process is running as. This might be significantly limiting, as chown requires root permissions, and we might not want other users to su into the apisix user.
Another approach would be explicit whitelisting of files (e.e, using a glob) in the APSIX server config file.

[moonming]

Let me explain my understanding.

First, the administrators of secrets, API gateway, and server may be three people. If secrets are managed in vault, the administrators of API gateway and server cannot access secrets and have no permission to modify them.

However, if secrets are managed through local files, the boundary of permission management becomes blurred. The administrator of secrets is everyone who has file read and write permissions.

Second, we may be running hundreds or thousands of APISIX data planes, which means we need to protect thousands of servers from hacker attacks. If one of the servers has a system vulnerability that is exploited, the secrets will be leaked.

Software like Vault can solve the above two security issues, which is one of its values.

[FoseFx]

Sorry for my late response, I must have missed the email.

If secrets are managed in vault, the administrators of API gateway and server cannot access secrets and have no permission to modify them.

This is not entirely correct, though. The server admin is able to dump the secrets themselves, or the vault credentials, from the environment or from memory.
The APISIX admin might be able to access the secrets as well, for example by setting the secrets as basic auth, or other headers for a new upstream, which they control, and dump them there.

Second, we may be running hundreds or thousands of APISIX data planes, which means we need to protect thousands of servers from hacker attacks. If one of the servers has a system vulnerability that is exploited, the secrets will be leaked.

If the server is compromised, adversaries might have the same permissions as the server admin. And in the end, the server admin is able to access everything. I agree with you, that we want to prevent unauthorized access.

Software like Vault can solve the above two security issues, which is one of its values.

Vault is awesome! But often a too much of a burden to manage. Another key principle of security is simplicity, however.

Many other reverse proxies store their certificates on disk as well:

• traefik
• kong
• envoy