Skip to content

Commit

Permalink
Rewrite project readme
Browse files Browse the repository at this point in the history
  • Loading branch information
@ErikPelli @elazarl
ErikPelli authored and elazarl committed Dec 19, 2024
1 parent 3a52544 commit 42e77ad
Showing 1 changed file with 134 additions and 103 deletions.
237 changes: 134 additions & 103 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,57 +1,70 @@
# Introduction
# GoProxy

[![GoDoc](https://godoc.org/github.com/elazarl/goproxy?status.svg)](https://godoc.org/github.com/elazarl/goproxy)
[![Join the chat at https://gitter.im/elazarl/goproxy](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/elazarl/goproxy?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![GoDoc](https://pkg.go.dev/badge/github.com/elazarl/goproxy)](https://pkg.go.dev/github.com/elazarl/goproxy)
![Status](https://github.com/elazarl/goproxy/workflows/Go/badge.svg)

Package goproxy provides a customizable HTTP proxy library for Go (golang),

It supports regular HTTP proxy, HTTPS through CONNECT, and "hijacking" HTTPS
connection using "Man in the Middle" style attack.

The intent of the proxy is to be usable with reasonable amount of traffic,
yet customizable and programmable.

The proxy itself is simply a `net/http` handler.

In order to use goproxy, one should set their browser to use goproxy as an HTTP
proxy. Here is how you do that [in Chrome](https://support.google.com/chrome/answer/96815?hl=en)
and [in Firefox](http://www.wikihow.com/Enter-Proxy-Settings-in-Firefox).

For example, the URL you should use as proxy when running `./bin/basic` is
`localhost:8080`, as this is the default binding for the basic proxy.

## Mailing List

New features will be discussed on the [mailing list](https://groups.google.com/forum/#!forum/goproxy-dev)
before their development.

## Latest Stable Release

Get the latest goproxy from `gopkg.in/elazarl/goproxy.v1`.

# Why not Fiddler2?

Fiddler is an excellent software with similar intent. However, Fiddler is not
as customizable as goproxy intends to be. The main difference is, Fiddler is not
intended to be used as a real proxy.

A possible use case that suits goproxy but
not Fiddler, is gathering statistics on page load times for a certain website over a week.
With goproxy you could ask all your users to set their proxy to a dedicated machine running a
goproxy server. Fiddler is a GUI app not designed to be run like a server for multiple users.

# A taste of goproxy

To get a taste of `goproxy`, a basic HTTP/HTTPS transparent proxy
GoProxy is a library to create a `customized` HTTP/HTTPS `proxy server` using
Go (aka Golang), with several configurable settings available.
The target of this project is to offer an `optimized` proxy server, usable with
reasonable amount of traffic, yet `customizable` and `programmable`.

The proxy itself is simply a `net/http` handler, so you can add multiple
middlewares (panic recover, logging, compression, etc.) over it. It can be
easily integrated with any other HTTP network library.

In order to use goproxy, one should set their browser (or any other client)
to use goproxy as an HTTP proxy.
Here is how you do that in [Chrome](https://www.wikihow.com/Connect-to-a-Proxy-Server)
and in [Firefox](http://www.wikihow.com/Enter-Proxy-Settings-in-Firefox).
If you decide to start with the `base` example, the URL you should use as
proxy is `localhost:8080`, which is the default one in our example.

## Features
- Perform certain actions only on `specific hosts`, with a single equality comparison or with regex evaluation
- Manipulate `requests` and `responses` before sending them to the browser
- Use a `custom http.Transport` to perform requests to the target server
- You can specify a `MITM certificates cache`, to reuse them later for other requests to the same host, thus saving CPU. Not enabled by default, but you should use it in production!
- Redirect normal HTTP traffic to a `custom handler`, when the target is a `relative path` (e.g. `/ping`)

## Proxy modes
1. Regular HTTP proxy
2. HTTPS through CONNECT
3. HTTPS MITM ("Man in the Middle") proxy server, in which the server generate TLS certificates to parse request/response data and perform actions on them
4. "Hijacked" proxy connection, where the configured handler can access the raw net.Conn data

## Maintainers
- [Elazar Leibovich](https://github.com/elazarl): Creator of the project, Software Engineer
- [Erik Pellizzon](https://github.com/ErikPelli): Maintainer, Freelancer (open to collaborations!)

## Contributions
If you have any trouble, suggestion, or if you find a bug, feel free to reach
out by opening a GitHub `issue`.
This is an `open source` project managed by volunteers, and we're happy
to discuss anything that can improve it.

Make sure to explain everything, including the reason behind the issue
and what you want to change, to make the problem easier to understand.
You can also directly open a `Pull Request`, if it's a small code change, but
you need to explain in the description everything.
If you open a pull request named `refactoring` with `5,000` lines changed,
we won't merge it... `:D`

The code for this project is released under the `BSD 3-Clause` license,
making it useful for `commercial` uses as well.

## A taste of GoProxy

To get a taste of `goproxy`, here you are a basic HTTP/HTTPS proxy
that just forward data to the destination:

```go
package main

import (
"github.com/elazarl/goproxy"
"log"
"net/http"

"github.com/elazarl/goproxy"
)

func main() {
Expand All @@ -61,7 +74,9 @@ func main() {
}
```

This line will add `X-GoProxy: yxorPoG-X` header to all requests sent through the proxy
### Request handler
This line will add `X-GoProxy: yxorPoG-X` header to all requests sent through the proxy,
before sending them to the destination:

```go
proxy.OnRequest().DoFunc(
Expand All @@ -71,100 +86,116 @@ proxy.OnRequest().DoFunc(
})
```

`DoFunc` will process all incoming requests to the proxy. It will add a header to the request
and return it. The proxy will send the modified request.
When the `OnRequest()` input is empty, the function specified in `DoFunc`
will process all incoming requests to the proxy. In this case, it will add
a header to the request and return it to the caller.
The proxy will send the modified request to the destination.
You can also use `Do` instead of `DoFunc`, if you implement the specified
interface in your type.

Note that we returned nil value as the response. Had we returned a response, goproxy would
have discarded the request and sent the new response to the client.
> ⚠️ Note we returned a nil value as the response.
> If the returned response is not nil, goproxy will discard the request
> and send the specified response to the client.
In order to refuse connections to reddit at work time
### Conditional Request handler
Refuse connections to www.reddit.com between 8 and 17 in the server
local timezone:

```go
proxy.OnRequest(goproxy.DstHostIs("www.reddit.com")).DoFunc(
func(r *http.Request,ctx *goproxy.ProxyCtx)(*http.Request,*http.Response) {
func(req *http.Request,ctx *goproxy.ProxyCtx)(*http.Request,*http.Response) {
if h,_,_ := time.Now().Clock(); h >= 8 && h <= 17 {
return r,goproxy.NewResponse(r,
goproxy.ContentTypeText,http.StatusForbidden,
"Don't waste your time!")
resp := goproxy.NewResponse(r, goproxy.ContentTypeText, http.StatusForbidden, "Don't waste your time!")
return req, resp
}
return r,nil
return req,nil
})
```

`DstHostIs` returns a `ReqCondition`, that is a function receiving a `Request` and returning a boolean.
We will only process requests that match the condition. `DstHostIs("www.reddit.com")` will return
a `ReqCondition` accepting only requests directed to "www.reddit.com".

`DoFunc` will receive a function that will preprocess the request. We can change the request, or
return a response. If the time is between 8:00am and 17:00pm, we will reject the request, and
return a pre-canned text response saying "do not waste your time".

See additional examples in the examples directory.
`DstHostIs` returns a `ReqCondition`, which is a function receiving a `*http.Request`
and returning a boolean that checks if the request satisfies the condition (and that will be processed).
`DstHostIs("www.reddit.com")` will return a `ReqCondition` that returns true
when the request is directed to "www.reddit.com".
The host equality check is `case-insensitive`, to reflect the behaviour of DNS
resolvers, so even if the user types "www.rEdDit.com", the comparison will
satisfy the condition.
When the hour is between 8:00am and 5:59pm, we directly return
a response in `DoFunc()`, so the remote destination will not receive the
request and the client will receive the `"Don't waste your time!"` response.

### Let's start
```go
import "github.com/elazarl/goproxy"
```

There are some proxy usage examples in the `examples` folder, which
cover the most common cases. Take a look at them and good luck!

# Type of handlers for manipulating connect/req/resp behavior
## Request & Response manipulation

There are 3 kinds of useful handlers to manipulate the behavior, as follows:
There are 3 different types of handlers to manipulate the behavior of the proxy, as follows:

```go
// handler called after receiving HTTP CONNECT from the client, and before proxy establish connection
// with destination host
// handler called after receiving HTTP CONNECT from the client, and
// before proxy establishes connection with the destination host
httpsHandlers []HttpsHandler
// handler called before proxy send HTTP request to destination host

// handler called before proxy sends HTTP request to destination host
reqHandlers []ReqHandler
// handler called after proxy receives HTTP Response from destination host, and before proxy forward
// the Response to the client.

// handler called after proxy receives HTTP Response from destination host,
// and before proxy forwards the Response to the client
respHandlers []RespHandler
```

Depending on what you want to manipulate, the ways to add handlers to each handler list are:
Depending on what you want to manipulate, the ways to add handlers to each of the previous lists are:

```go
// Add handlers to httpsHandlers
proxy.OnRequest(Some ReqConditions).HandleConnect(YourHandlerFunc())
proxy.OnRequest(some ReqConditions).HandleConnect(YourHandlerFunc())

// Add handlers to reqHandlers
proxy.OnRequest(Some ReqConditions).Do(YourReqHandlerFunc())
proxy.OnRequest(some ReqConditions).Do(YourReqHandlerFunc())

// Add handlers to respHandlers
proxy.OnResponse(Some RespConditions).Do(YourRespHandlerFunc())
proxy.OnResponse(some RespConditions).Do(YourRespHandlerFunc())
```

For example:
Example:

```go
// This rejects the HTTPS request to *.reddit.com during HTTP CONNECT phase.
// Reddit URL check is case-insensitive, so the block will work also if the user types something like rEdDit.com.
// Reddit URL check is case-insensitive because of (?i), so the block will work also if the user types something like rEdDit.com.
proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("(?i)reddit.*:443$"))).HandleConnect(goproxy.AlwaysReject)

// This will NOT reject the HTTPS request with URL ending with gif, due to the fact that proxy
// only got the URL.Hostname and URL.Port during the HTTP CONNECT phase if the scheme is HTTPS, which is
// quiet common these days.
// Be careful about this example! It shows you a common error that you
// need to avoid.
// This will NOT reject the HTTPS request with URL ending with .gif because,
// if the scheme is HTTPS, the proxy will receive only URL.Hostname
// and URL.Port during the HTTP CONNECT phase.
proxy.OnRequest(goproxy.UrlMatches(regexp.MustCompile(`.*gif$`))).HandleConnect(goproxy.AlwaysReject)

// The correct way to manipulate the HTTP request using URL.Path as condition is:
// To fix the previous example, here there is the correct way to manipulate
// an HTTP request using URL.Path (target path) as a condition.
proxy.OnRequest(goproxy.UrlMatches(regexp.MustCompile(`.*gif$`))).Do(YourReqHandlerFunc())
```

# What's New

1. Ability to `Hijack` CONNECT requests. See
[the eavesdropper example](https://github.com/elazarl/goproxy/blob/master/examples/goproxy-eavesdropper/main.go#L27)
2. Transparent proxy support for http/https including MITM certificate generation for TLS. See the [transparent example.](https://github.com/elazarl/goproxy/tree/master/examples/goproxy-transparent)

# License

I put the software temporarily under the Go-compatible BSD license.
If this prevents someone from using the software, do let me know and I'll consider changing it.

At any rate, user feedback is very important for me, so I'll be delighted to know if you're using this package.

# Beta Software

I've received positive feedback from a few people who use goproxy in production settings.
I believe it is good enough for usage.

I'll try to keep reasonable backwards compatibility. In case of a major API change,
I'll change the import path.
## Project Status
This project has been created `10 years` ago, and has reached a stage of
`maturity`. It can be safely used in `production`, and many projects
already do that.

If there will be any `breaking change` in the future, a `new version` of the
Go module will be released (e.g. v2).

## Trusted, as a direct dependency, by:
<p align="left">
<a href="https://github.com/stripe/goproxy" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/856813?s=50" alt="Stripe" title="Stripe" /> </a>
<a href="https://github.com/dependabot/goproxy" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/27347476?s=50" alt="Dependabot" title="Dependabot" /> </a>
<a href="https://github.com/go-git/go-git" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/57653224?s=50" alt="Go Git" title="Go Git" /> </a>
<a href="https://github.com/google/oss-rebuild" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/1342004?s=50" alt="Google" title="Google" /> </a>
<a href="https://github.com/grafana/grafana-plugin-sdk-go" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/7195757?s=50" alt="Grafana" title="Grafana" /> </a>
<a href="https://github.com/superfly/tokenizer" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/22525303?s=50" alt="Fly.io" title="Fly.io" /> </a>
<a href="https://github.com/kubernetes/minikube" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/13629408?s=50" alt="Kubernetes / Minikube" title="Kubernetes / Minikube" /> </a>
<a href="https://github.com/newrelic/newrelic-client-go" target="_blank" rel="noreferrer"> <img src="https://avatars.githubusercontent.com/u/31739?s=50" alt="New Relic" title="New Relic" /> </a>
</p>

0 comments on commit 42e77ad

Please sign in to comment.