Add MITM support to smokescreen

.vscode/launch.json

@@ -0,0 +1,16 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "smokescreen",
+            "type": "go",
+            "request": "launch",
+            "mode": "auto",
+            "program": "./",
+            "args": ["--config-file", "config.yaml", "--egress-acl-file", "acl.yaml"]
+        }
+    ]
+}

Development.md

@@ -0,0 +1,308 @@
+
+# Development and Testing
+
+## Testing
+```bash
+go test ./...
+```
+
+## Running locally
+
+This section describes how to run Smokescreen locally with different scenarios and using `curl` as a client.
+
+- [HTTP Proxy](#http-proxy)
+- [HTTP CONNECT Proxy](#http-connect-proxy)
+- [Monitor metrics Smokescreen emits](#monitor-metrics-smokescreen-emits)
+- [HTTP CONNECT Proxy over TLS](#http-connect-proxy-over-tls)
+- [MITM (Man in the middle) Proxy](#mitm-man-in-the-middle-proxy)
+- [MITM (Man in the middle) Proxy over TLS](#mitm-man-in-the-middle-proxy-over-tls)
+
+### HTTP Proxy
+
+#### Configurations
+
+```yaml
+# config.yaml
+---
+allow_missing_role: true  # skip mTLS client validation (use default ACL)
+```
+
+```yaml
+# acl.yaml
+---
+version: v1
+services: []
+default:
+  name: default
+  project: security
+  action: enforce
+  allowed_domains: 
+    - example.com
+```
+
+#### Run
+
+```bash
+# Run smokescreen (in a different shell)
+go run . --config-file config.yaml --egress-acl-file acl.yaml
+
+# Curl
+curl -x localhost:4750 http://example.com
+# Curl with ALL_PROXY
+ALL_PROXY=localhost:4750 curl -v http://example.com
+```
+
+### HTTP CONNECT Proxy
+
+#### Configurations
+
+```yaml
+# config.yaml
+---
+allow_missing_role: true  # skip mTLS client validation (use default ACL)
+```
+
+```yaml
+# acl.yaml
+---
+version: v1
+services: []
+default:
+  name: default
+  project: security
+  action: enforce
+  allowed_domains: 
+    - api.github.com
+```
+
+#### Run
+
+```bash
+# Run smokescreen (in a different shell)
+go run . --config-file config.yaml --egress-acl-file acl.yaml
+
+# Curl
+curl --proxytunnel -x localhost:4750 https://api.github.com/zen
+# Curl with HTTPS_PROXY
+HTTPS_PROXY=localhost:4750 curl https://api.github.com/zen
+```
+
+### Monitor metrics Smokescreen emits
+
+#### Configurations
+
+```yaml
+# config.yaml
+---
+allow_missing_role: true  # skip mTLS client validation (use default ACL)
+statsd_address: 127.0.0.1:8200
+```
+
+```yaml
+# acl.yaml
+---
+version: v1
+services: []
+default:
+  name: default
+  project: security
+  action: enforce
+  allowed_domains: 
+    - api.github.com
+```
+
+#### Run
+
+```bash
+# Listen to a local port with nc (in a different shell)
+nc -uklv 127.0.0.1 8200
+
+# Run smokescreen (in a different shell)
+go run . --config-file config.yaml --egress-acl-file acl.yaml
+
+# Curl
+curl --proxytunnel -x localhost:4750 https://api.github.com/zen
+# Curl with HTTPS_PROXY
+HTTPS_PROXY=localhost:4750 curl https://api.github.com/zen
+```
+
+### HTTP CONNECT Proxy over TLS
+
+#### Set-up
+
+##### Generate certificates
+```bash
+mkdir -p mtls_setup
+# Private keys for CAs
+openssl genrsa -out mtls_setup/server-ca.key 2048
+openssl genrsa -out mtls_setup/client-ca.key 2048
+
+# Generate client and server CA certificates
+openssl req -new -x509 -nodes -days 1000 -key mtls_setup/server-ca.key -out mtls_setup/server-ca.crt \
+    -subj "/C=AQ/ST=Petrel Island/L=Dumont-d'Urville
+/O=Penguin/OU=Publishing house/CN=server CA"
+
+openssl req -new -x509 -nodes -days 1000 -key mtls_setup/client-ca.key -out mtls_setup/client-ca.crt \
+    -subj "/C=MA/ST=Tarfaya/L=Tarfaya/O=Fennec/OU=Aviator/CN=Client CA"
+
+# Generate a certificate signing request (client CN is localhost which is used by smokescreen as the service name by default)
+openssl req -newkey rsa:2048 -nodes -keyout mtls_setup/server.key -out mtls_setup/server.req \
+    -subj "/C=AQ/ST=Petrel Island/L=Dumont-d'Urville/O=Chionis/OU=Publishing house/CN=server req"
+openssl req -newkey rsa:2048 -nodes -keyout mtls_setup/client.key -out mtls_setup/client.req \
+    -subj "/C=MA/ST=Tarfaya/L=Tarfaya/O=Addax/OU=Writer/CN=localhost"
+
+# Have the CA sign the certificate requests and output the certificates.
+echo "authorityKeyIdentifier=keyid,issuer
+basicConstraints=CA:FALSE
+keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
+subjectAltName = @alt_names
+
+[alt_names]
+DNS.1 = localhost
+" > mtls_setup/localhost.ext
+
+openssl x509 -req -in mtls_setup/server.req -days 1000 -CA mtls_setup/server-ca.crt -CAkey mtls_setup/server-ca.key -set_serial 01 -out mtls_setup/server.crt -extfile mtls_setup/localhost.ext
+
+openssl x509 -req -in mtls_setup/client.req -days 1000 -CA mtls_setup/client-ca.crt -CAkey mtls_setup/client-ca.key -set_serial 01 -out mtls_setup/client.crt
+```
+
+##### Configurations
+
+```yaml
+# config.yaml
+---
+tls:
+  cert_file: "mtls_setup/server.crt"
+  key_file: "mtls_setup/server.key"
+  client_ca_files:
+    - "mtls_setup/client-ca.crt"
+```
+
+```yaml
+# acl.yaml
+---
+version: v1
+services:
+  - name: localhost
+    project: github
+    action: enforce
+    allowed_domains:
+      - api.github.com
+default:
+  name: default
+  project: security
+  action: enforce
+  allowed_domains: []
+```
+
+#### Run
+
+```bash
+# Run smokescreen (in a different shell)
+go run . --config-file config.yaml --egress-acl-file acl.yaml
+
+# Curl
+curl --proxytunnel -x https://localhost:4750 --proxy-cacert mtls_setup/server-ca.crt --proxy-cert mtls_setup/client.crt --proxy-key mtls_setup/client.key https://api.github.com/zen
+# Curl with HTTPS_PROXY
+HTTPS_PROXY=https://localhost:4750 curl --proxy-cacert mtls_setup/server-ca.crt --proxy-cert mtls_setup/client.crt --proxy-key mtls_setup/client.key https://api.github.com/zen
+```
+
+### MITM (Man in the middle) Proxy
+
+#### Set-up
+
+```yaml
+# config.yaml
+---
+allow_missing_role: true  # skip mTLS client validation (use default ACL)
+# Re-using goproxy library CA and key
+mitm_ca_cert_file: "vendor/github.com/stripe/goproxy/ca.pem"
+mitm_ca_key_file: "vendor/github.com/stripe/goproxy/key.pem"
+```
+
+```yaml
+# acl.yaml
+---
+version: v1
+services: []
+default:
+  name: default
+  project: security
+  action: enforce
+  allowed_domains:
+    - wttr.in
+  mitm_domains:
+  - domain: wttr.in
+    add_headers:
+      Accept-Language: el
+    detailed_http_logs: true
+    detailed_http_logs_full_headers:
+      - User-Agent
+```
+
+#### Run
+
+```bash
+# Run smokescreen (in a different shell)
+go run . --config-file config.yaml --egress-acl-file acl.yaml
+
+# Curl (weather should be in Greek since we set the Accept-Language header)
+curl --proxytunnel -x localhost:4750 --cacert vendor/github.com/stripe/goproxy/ca.pem https://wttr.in
+# Curl with HTTPS_PROXY
+HTTPS_PROXY=localhost:4750 curl --cacert vendor/github.com/stripe/goproxy/ca.pem https://wttr.in
+```
+
+### MITM (Man in the middle) Proxy over TLS
+
+#### Set-up
+
+Please generate the certificates from the TLS Generate certificates section.
+
+```yaml
+# config.yaml
+---
+tls:
+  cert_file: "mtls_setup/server.crt"
+  key_file: "mtls_setup/server.key"
+  client_ca_files:
+    - "mtls_setup/client-ca.crt"
+# Re-using goproxy library CA and key
+mitm_ca_cert_file: "vendor/github.com/stripe/goproxy/ca.pem"
+mitm_ca_key_file: "vendor/github.com/stripe/goproxy/key.pem"
+```
+
+```yaml
+# acl.yaml
+---
+version: v1
+services:
+  - name: localhost
+    project: github
+    action: enforce
+    allowed_domains:
+      - wttr.in
+    mitm_domains:
+      - domain: wttr.in
+        add_headers:
+          Accept-Language: el
+        detailed_http_logs: true
+        detailed_http_logs_full_headers:
+          - User-Agent
+default:
+  name: default
+  project: security
+  action: enforce
+  allowed_domains: []
+```
+
+#### Run
+
+```bash
+# Run smokescreen (in a different shell)
+go run . --config-file config.yaml --egress-acl-file acl.yaml
+
+# Curl (weather should be in Greek since we set the Accept-Language header)
+curl --proxytunnel -x https://localhost:4750 --cacert vendor/github.com/stripe/goproxy/ca.pem --proxy-cacert mtls_setup/server-ca.crt --proxy-cert mtls_setup/client.crt --proxy-key mtls_setup/client.key https://wttr.in
+# Curl with HTTPS_PROXY
+HTTPS_PROXY=https://localhost:4750 curl --cacert vendor/github.com/stripe/goproxy/ca.pem --proxy-cacert mtls_setup/server-ca.crt --proxy-cert mtls_setup/client.crt --proxy-key mtls_setup/client.key https://wttr.in
+```

README.md

@@ -171,41 +171,7 @@ If a domain matches both the `global_allow_list` and the `global_deny_list`, the
 
 # Development and Testing
 
-## Running locally
-
-To run Smokescreen locally, you can provide a minimal configuration file and use `curl` as a client. For example:
-
-```yaml
-# config.yaml
----
-allow_missing_role: true  # skip mTLS client validation
-statsd_address: 127.0.0.1:8200
-```
-
-If you want to see metrics Smokescreen emits, listen on a local port:
-
-```shellsession
-$ nc -uklv 127.0.0.1 8200
-```
-
-Build and run Smokescreen:
-
-```shellsession
-$ go run . --config-file config.yaml
-{"level":"info","msg":"starting","time":"2022-11-30T15:19:08-08:00"}
-```
-
-Make a request using `curl`:
-
-```shellsession
-$ curl --proxytunnel -x localhost:4750 https://stripe.com/
-```
-
-## Testing
-
-```shellsession
-$ go test ./...
-```
+See [Development.md](Development.md)
 
 # Contributors
 
@@ -220,3 +186,4 @@ $ go test ./...
 - Evan Broder
 - Marc-André Tremblay
 - Ryan Koppenhaver
+- Harold Simpson

go.mod

@@ -12,7 +12,7 @@ require (
 	github.com/rs/xid v1.2.1
 	github.com/sirupsen/logrus v1.9.0
 	github.com/stretchr/testify v1.8.0
-	github.com/stripe/goproxy v0.0.0-20240711170433-75b93c00dfb0
+	github.com/stripe/goproxy v0.0.0-20240909161309-906e92b723dd
 	golang.org/x/net v0.17.0
 	gopkg.in/urfave/cli.v1 v1.20.0
 	gopkg.in/yaml.v2 v2.4.0

go.sum

@@ -224,6 +224,8 @@ github.com/stripe/goproxy v0.0.0-20240702232545-72d7dbc6d4fe h1:RzjmXDVOjWq9sPRc
 github.com/stripe/goproxy v0.0.0-20240702232545-72d7dbc6d4fe/go.mod h1:hF2CVgH4++5ijZiy9grGVP8Fsi4u+SMOtbnIKYbMUjY=
 github.com/stripe/goproxy v0.0.0-20240711170433-75b93c00dfb0 h1:1xKgjoLWAosf0yoKocNN4mel1++AKqJw9axpJyz1JRw=
 github.com/stripe/goproxy v0.0.0-20240711170433-75b93c00dfb0/go.mod h1:hF2CVgH4++5ijZiy9grGVP8Fsi4u+SMOtbnIKYbMUjY=
+github.com/stripe/goproxy v0.0.0-20240909161309-906e92b723dd h1:+GFSUmU0/RS984Zp8KvYxjU95DxVIGJflsOfITHkF68=
+github.com/stripe/goproxy v0.0.0-20240909161309-906e92b723dd/go.mod h1:hF2CVgH4++5ijZiy9grGVP8Fsi4u+SMOtbnIKYbMUjY=
 github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=