[FEATURE] Make it possible to restrict keys in scope.headers log entries

[LasseGravesen]

I had an issue pop up when I updated uvicorn to a newer version, specifically it seems that scope.headers got added to the log output, and there appears to be no way to filter what goes into that log.

Here is an example log:

{
	"message": "255.255.255.255:37326 - \"POST /api/v1/count HTTP/1.1\" 200",
	"timestamp": "2020-01-13T12:33:44.384008",
	"levelno": 20,
	"levelname": "INFO",
	"name": "uvicorn.access",
	"status_code": 200,
	"scope": {
		"type": "http",
		"http_version": "1.1",
		"server": ["0.0.0.0", 8080],
		"client": ["255.255.255.255", 37326],
		"scheme": "http",
		"method": "POST",
		"root_path": "",
		"path": "/api/v1/count",
		"raw_path": "b'/api/v1/count'",
		"query_string": "b''",
		"headers": [["b'host'", "b'lb.api.production.company.io:54776'"], ["b'sentry-trace'", "b'00-27422174e7f14c9aac3572a42d34006f-8f059e2531e39fb5-00'"], ["b'user-agent'", "b'test (version 2019.12.1)'"], ["b'accept-encoding'", "b'gzip, deflate'"], ["b'accept'", "b'*/*'"], ["b'content-type'", "b'application/json'"], ["b'authorization'", "b'Bearer fake-bearer-token'"], ["b'content-length'", "b'137'"], ["b'x-forwarded-port'", "b'54776'"], ["b'x-forwarded-for'", "b'255.255.255.255'"], ["b'connection'", "b'close'"]],
		"fastapi_astack": "<contextlib.AsyncExitStack object at 0x7f62ec631090>",
		"app": "<fastapi.applications.FastAPI object at 0x7f62eeda7710>",
		"state": {
		},
		"router": "<fastapi.routing.APIRouter object at 0x7f62eeda7bd0>",
		"endpoint": "<function count at 0x7f62ee6bc560>",
		"path_params": {}
	}
}

The problem are headers like ["b'authorization'", "b'Bearer fake-bearer-token'"] which can give away sensitive information like API Keys or valid Bearer tokens.

I would like a feature that gives me the option to exclude specific headers that may include sensitive information that I do not want to be logged.
The current way I do this is by creating a custom formatter that drops the scope.headers field entirely.

[tomchristie]

The sensible place to start here would be to look at other Python web servers and determine what header filtering they put in place. In particular what does Gunicorn do here? Do they output headers / WGI environ in logs at any point? Do they filter the headers in any particular way?

[LasseGravesen]

@tomchristie
Gunicorn defines the fields that it logs as follows:

Identifier	Description
h	remote address
l	'-'
u	user name
t	date of the request
r	status line (e.g. GET / HTTP/1.1)
m	request method
U	URL path without query string
q	query string
H	protocol
s	status
B	response length
b	response length or '-' (CLF format)
f	referer
a	user agent
T	request time in seconds
D	request time in microseconds
L	request time in decimal seconds
p	process ID
{header}i	request header
{header}o	response header
{variable}e	environment variable

Use lowercase for header and environment variable names, and put {...}x names inside %(...)s. For example: %({x-forwarded-for}i)s

In a log format, like %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s".

I would really caution against doing logging like Gunicorn does it, I'm quite happy with the way uvicorn handles it, with the exception of not being able to define headers that should be excluded.

[euri10]

I think this is fixed in #859