Noise v1.1.3: Performance improvements and dependency updates.

ecdh, readme, mod: remove dependency on agl/ed25519

Noise v1.1.2: Fearless, decentralized p2p networking.

The latest release, Noise v1.1.2, finally arrives 9 months after the initial release of Noise.

The release involves a complete refactor that has allowed for Noise to adopt significant performance, security, privacy, and developer ergonomic improvements.

Noise is connectionless.

A bounded connection pool was introduced to alleviate developers from having to arduously maintain the lifecycle of connections and resources associated with a single peer.

The connection pool by default is bounded such that a single node may only have, at most, a specified number of inbound and outbound connections open at any time.

Should the pool be full and a new connection needs to be established, the pool will gracefully disconnect the last-inserted connection (block the current goroutine until the last-inserted connections entire read/write buffer is emptied) to make room for the new connection.

This change brings together a much simpler user experience for developers using Noise.

To send/receive messages, and send requests/responses across peers requires no management over connections whatsoever from a developer.

Network protocols implemented on top of Noise therefore no longer need to worry about assumptions behind the state or lifecycle of peer connections.

Noise builds networks.

With the introduction of connection pooling comes a decoupling of peer identities from long-lived connections to external nodes.

Peer identities are now optionally persisted away from the instance of connection in a decouplable Kademlia routing table that may be persisted on-disk/in-memory.

Additionally, the Kademlia API for Noise now supports sending/receiving messages to/from peers, or connecting to peers by their Ed25519 public key.

The peer discovery module has been generalized to support searching for peer IDs (comprised of their public IP/port and Ed25519 public key) should you have their Ed25519 public key.

Noise is quiescent.

To improve the anonymity, privacy, and security of nodes, Noise now strictly assumes all connections are to be short-lived.

Connections by default timeout after a specified time duration should no read/write activity occur.

Having an idle timeout allows for nodes to gracefully close unused connections and release extraneous resources.

Frequently recycling connections also allow for sensitive cryptographic primitives associated with encrypting/decrypting messages over-the-wire across peers to have their keys be frequently recycled.
Noise uses the best of best practices.

Peer connections by default are now encrypted using AES 256-bit Galois Counter Mode (GCM) with a Curve25519 shared key established by an Elliptic-Curve Diffie-Hellman Handshake.

In detail, the handshake protocol executed upon establishing a connection with a peer goes as follows:

1. peers send each other their ephemeral Ed25519 public keys,
2. peers convert the public keys they received into Curve25519 public keys,
3. peers convert their ephemeral Ed25519 private keys into Curve25519 private keys,
4. peers establish a shared secret by performing ECDH with their private Curve25519 private key and their peers Curve25519 public key,
5. peers use the shared secret as a symmetric key and communicate from then-on with messages encrypted/decrypted via. AES 256-bit GCM with a randomly-generated 12-byte nonce.

The process is akin to how handshakes are performed over TLS 1.2 without the use of key derivation function (KDF), but without requiring importing and maintaining the whole TLS suite into your project.

Peer IDs are represented by an IPv4/IPv6 address and an unsigned 16-bit integer port, and an Ed25519 public key. IDs are exchanged after an encrypted session is established over a connection across peers.

In a later release, options will become available to disable/customize the handshake protocol used for nodes.

The idea for customizing the handshake protocol comes from the following Github issue submitted by a community member.

Noise is extensible.

New methods were introduced for creating plugins that may hook onto events emitted throughout a node's lifecycle.

Kademlia is now fully-decoupled and is now an optional plugin that may be implemented into your application.

It may be integrated into your application simply by calling the Bind method to your node before you have your node start listening for new peers.

A plethora of functional option APIs is also provided to customize timeouts, settings, and logging as well.

Noise provides sane defaults.

All timeouts, connection pool settings, and logging are provided sane defaults so that you can use Noise from the get-go. More explicitly:

1. No logs are printed by default.
2. A random Ed25519 key pair is generated for a new node.
3. Peers attempt to be dialed at most three times.
4. A total of 128 outbound connections are allowed at any time.
5. A total of 128 inbound connections are allowed at any time.
6. Connections timeout after 10 seconds if no reads/writes occur.

Noise is minimal.

A lot of dependencies have been cut out to their bare minimum. In total, Noise v1.1.2 only comprises of approximately 4000 lines of code.

The biggest change is a shift in logging libraries from rs/zerolog to uber-go/zap after performance benchmarks in applications we have made internally with Noise.

A lot of our applications are performance-critical and need to trace large amounts of logs. The built-in buffering of logs with uber-go/zap has been extremely helpful.

More explicitly:

1. Logging is handled by uber-go/zap.
2. Byte buffer pooling is handled by valyala/bytebufferpool.
3. Unit tests are handled by stretchr/testify.
4. Ed25519 signatures are handled by oasislabs/ed25519.
5. Elliptic-curve Diffie Hellman Key Exchange (ECDH) over Curve25519 is handled by agl/ed25519.

Noise simplifies formatting messages.

A serialization and deserialization function may optionally be specified for Go types that are to be sent to/from peers.

This allows you to specify the wire format of messages sent between your peers to be in a wide variety of formats such as protobuf, msgpack, json, or even raw binary (click here for an example!).

All methods that end with a suffix of *Message in the API for Node, such as RegisterMessage for example automatically handle on-the-wire serialization/deserialization.

Refer to the examples and documentation here to learn more.

Noise embraces concurrency.

Several methods (wait until handshake finished, wait until connection closed) were introduced to maintain and track the lifecycle of goroutines, node instances, peer instances, and even connection resources.

This has helped us a lot in creating unit tests and integration tests that check networking edge cases for highly-concurrent applications we have implemented in-house.

In total, a single connection spawns 4 goroutines:

1. one for handling protocol logic,
2. one for recycling a connection should it timeout,
3. one for reading and buffering raw bytes from a connection, and
4. one for sending and buffering raw bytes to a connection.

A single node spawns only 1 goroutine for listening for new connections.

Noise is fast.

Benchmarks measure CPU time and allocations of a single node sending messages, requests, and responses to/from itself over 8 logical cores on a loopback adapter.

Take these benchmark numbers with a grain of salt.

% cat /proc/cpuinfo | grep 'model name' | uniq
model name : Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz

% go test -bench=. -benchtime=30s -benchmem
goos: linux
goarch: amd64
pkg: github.com/perlin-network/noise
BenchmarkRPC-8           2978550             14136 ns/op            1129 B/op         27 allocs/op
BenchmarkSend-8          9239581              4546 ns/op             503 B/op         12 allocs/op
PASS
ok      github.com/perlin-network/noise 101.966s

You can also run your own separate benchmarks locally on a single core by running either one of these commands:

$ go run github.com/perlin-network/noise/cmd/benchmark_send
$ go run github.com/perlin-network/noise/cmd/benchmark_rpc

Noise finally adopts semantic versioning.

Releases from now on are marked with a version number formatted as MAJOR.MINOR.PA...