chore(docs): new docs format

.github/workflows/ci.yml

@@ -77,15 +77,24 @@ jobs:
       - name: buf lint
         uses: bufbuild/buf-lint-action@v1
 
-  spellcheck:
+  doc-lint:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout sources
         uses: actions/checkout@v3
+
       - name: Install codespell
         run: sudo pip3 install codespell tomli
+
       - name: Check spelling
         run: codespell --toml .github/workflows/codespell.toml *.md docs/*.md 
+
+      - name: markdown link check
+        uses: gaurav-nelson/github-action-markdown-link-check@v1
+        with:
+          folder-path: 'docs'
+          file-path: 'README.md,CONTRIBUTING.md,SECURITY.md'
+          config-file: '.github/workflows/mlc_config.json'
 
   cog_check_job:
     runs-on: ubuntu-latest

.github/workflows/mlc_config.json

@@ -0,0 +1,7 @@
+{
+    "ignorePatterns": [
+      {
+        "pattern": "^https://github.com/OMGWINNING/rundler"
+      }
+    ]
+}

docs/CONTRIBUTING.md → CONTRIBUTING.md

@@ -84,10 +84,10 @@ Tests that use forking must contain "fork" in their name.
 It is a recommended best practice to keep your changes as logically grouped as possible within individual commits. There is no limit to the number of commits any single pull request may have, and many contributors find it easier to review changes that are split across multiple commits.
 That said, if you have a number of commits that are "checkpoints" and don't represent a single logical change, please squash those together.
 
-We follow the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) specification to make sure that all of our commits are formatted well and are in proporly tailored to the change that the commit is making.
+We follow the [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) specification to make sure that all of our commits are formatted well and are properly tailored to the change that the commit is making.
 
 #### Opening the pull request
 
 From within GitHub, opening a new pull request will present you with a template that should be filled out. Please try your best at filling out the details, but feel free to skip parts if you're not sure what to put.
 
-[dev-tg]: https://t.me/+F_xS9IVOdJZmZjQx
+[dev-tg]: https://t.me/rundler

Cargo.toml

@@ -1,16 +1,7 @@
 [workspace]
 members = [
-    "bin/rundler",
-    "bin/tools",
-    "crates/builder",
-    "crates/dev",
-    "crates/pool",
-    "crates/provider",
-    "crates/rpc",
-    "crates/sim",
-    "crates/task",
-    "crates/types",
-    "crates/utils"
+    "bin/*",
+    "crates/*",
 ]
 default-members = ["bin/rundler"]
 

README.md

@@ -1,82 +1,118 @@
 # Rundler
 
-## Development
+[![gh_ci_badge]][gh_ci_link]
+[![tg_badge]][tg_link]
 
-### Build & Test
+[gh_ci_badge]: https://github.com/OMGWINNING/rundler/workflows/ci/badge.svg
+[gh_ci_link]: https://github.com/OMGWINNING/rundler/actions/workflows/ci.yml
+[tg_badge]: https://img.shields.io/endpoint?color=neon&logo=telegram&label=chat&url=https://mogyo.ro/quart-apis/tgmembercount?chat_id=rundler
+[tg_link]: https://t.me/rundler
 
-Prerequisites:
+**High-performance, modular implementation of an ERC-4337 bundler**
 
-* [Rust & Cargo](https://rustup.rs/)
-    * Nightly rust required for `rustfmt`:
-```
-rustup toolchain add nightly
-```
-* [Cocogitto](https://github.com/cocogitto/cocogitto)
-    * `cargo install --locked cocogitto`
-* [Docker](https://docs.docker.com/engine/install/)
+[Run](#running) | [Developer Docs](./docs)
 
-Run tests:
-```
-git clone https://github.com/OMGWINNING/rundler
-cd rundler
-cargo test --all
-```
+🚧 *Under active development, see [status](#status) below.* 🚧
+
+## What's a Rundler?
+
+**Rundler** (**R**ust B**undler**) is an [ERC-4337][eip_link] bundler implementation written in Rust. Rundler is designed to achieve high-performance and high-reliability in cloud deployments via a modular architecture. 
+
+Built, maintained, and used by [Alchemy](https://www.alchemy.com/) to push the limits of user experience on EVM chains via [Account Abstraction](https://www.alchemy.com/blog/account-abstraction).
+
+## Goals
+
+Rundler is meant to power the ERC-4337 ecosystem with an implementation that users can rely on to scale to their needs while maintaining high reliability. As ERC-4337 is a nascent technology, Rundler strives to be on the bleeding edge of feature support and the team will be contributing to push the protocol forward.
+
+Our goals with Rundler:
+
+1. **ERC-4337 Specification Compliance**: Rundler strives to implement the full ERC-4337 specification and to maintain support as the specification changes and new onchain components are released. This includes new Entry Point contract support, support for the upcoming P2P mempool specification, support for alternative mempools, and more.
+2. **High Performance and Reliability**:  Rundler strives to power the most demanding workloads in cloud environments. Rust was chosen for its high performance and memory safety. Rundler's modular architecture lets providers choose to run the stateless components (RPC, builder) in a fully horizontally scalable manner connecting to the stateful components (mempool, p2p, event cache) via the network. Rundler's bundle builder is designed to be able to support the full gas throughput of the network its building for.
+3. **Extendability/Chain Support**: ERC-4337 is designed to support any EVM chain. However, different EVM supporting networks have different rules around how they support things like gas usage, gas fees, precompiles, etc. Rundler is designed to be extendable and easily adapted to support any EVM chain.
+4. **Modularity**: Rundler is written in a modular manner, allowing its components to be run as a single integrated binary, or as a distributed system. Rundler also strives for its individual crates to be used to support future ERC-4337 tooling.
+
+## Status
+
+Rundler is under active development. It is used in Alchemy's cloud to power its account abstraction APIs. However, it is rapidly being upgraded, features are being added, interfaces will have breaking changes, and the ERC-4337 spec is evolving from onchain learnings.
+
+The documentation is work in progress, and we are working to improve it. Please [reach out](#help) with any questions.
+
+**Use in production at your own risk.**
 
-### Run locally:
+### ERC-4337 Support
 
-Running locally requires access to an Ethereum RPC node on the network the Rundler is running on. For testing it is recommended to run a local development node.
+Rundler currently supports the [v0.6 release](https://github.com/eth-infinitism/account-abstraction/tree/v0.6.0) of Entry Point contract.
 
-Steps to run `Geth` in dev mode locally:
+### Chain Support
 
-1. `cd test`
-2. `docker compose up` (use `-d`) to run in background.
+Rundler currently supports and has been tested on the following networks and their testnets:
 
-Steps to run `Rundler` locally:
+* Ethereum
+* Optimism
+* Base
+* Arbitrum One
+* Polygon POS
 
-1. Copy the default .env file. By default, this will be pointing at the local `Geth` node.
+## Developers
+
+### Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md).
+
+### Running
+
+The easiest way to run Rundler is via a docker container. There is currently no pre-built image. See our [docker documentation](docs/docker.md) for further instructions.
+
+### Developing
+
+For detailed instructions see [developing](docs/developing.md).
+
+Install [prerequisites](docs/developing.md#setup).
+
+Clone the repository and checkout submodules:
 ```
-cp test/.env.default .env
+git clone https://github.com/OMGWINNING/rundler
+cd rundler
+git submodule update --init --recursive
 ```
 
-2. (Optional) Fund and deploy contracts. Contracts are deployed from the [account-abstraction](https://github.com/eth-infinitism/account-abstraction) repo. This does the following: 
-    * Creates an account for the bundler to use.
-    * Deploys the `EntryPoint`.
-    * Deploys a `SimpleAccountFactory`.
-    * Deploys a `VerifyingPaymaster`.
-    * Deploys a `SimpleAccount`.
-    * Funds each of the accounts.
+Run unit tests:
 ```
-cargo run --bin deploy_dev_contracts
+make test-unit
 ```
 
-3. (Optional) Modify the `BUNDLER_PRIVATE_KEY` field if using a different account than was funded in #2.
-
-4. Start the Rundler
+Run ERC-4337 spec tests:
 ```
-cargo run node
+cd test/spec-tests/bundler-spec-tests && pdm install && pdm run update-deps
+make test-spec-integrated
 ```
 
-### Documentation
+### Help
+
+If you have questions regarding the Rundler codebase, please first look through our [documentation](./docs/).
+
+With further questions:
+
+* Github discussions
+* [Telegram][tg_link]
+* [Github issues](https://github.com/OMGWINNING/rundler/issues/new/choose)
+
+If you have questions regarding Alchemy's APIs or service, kindly refrain from discussion here. Please join the [Discord](https://discord.com/invite/alchemyplatform) or reach out to [email protected].
+
+### Security
 
-- [configuration](docs/config.md)
-- [transactions](docs/transactions.md)
-- [contributing](docs/CONTRIBUTING.md)
-- [architecture](docs/architecture.md)
+For security concerns do not file a public ticket, please reach out to [email protected].
 
-### Goals
+See [SECURITY.md](SECURITY.md) for details.
 
-- Implement P2P mempool
-- Implement persistent mempool
-- Keep our spec test fork in like with origin 
-- Deploy public docker image
-- Cross platform compilation
+## Acknowledgements
 
-### Resources
+We stand on the shoulders of the following teams:
 
-- [EIP-4337](https://eips.ethereum.org/EIPS/eip-4337)
+- [ERC-4337 team](https://github.com/eth-infinitism/account-abstraction): The ERC-4337 team has pioneered the standard, and has answered countless questions during our development. They developed the [bundler reference implementation](https://github.com/eth-infinitism/bundler) and [spec tests](https://github.com/eth-infinitism/bundler-spec-tests) that were invaluable during our development process. We are excited to continue to work with this team to push ERC-4337 ahead.
+- [Reth](https://github.com/paradigmxyz/reth): Big thanks to the Reth team, from which we've taken inspiration for many of our practices in this repo (including this README). They are pushing the Ethereum Rust ecosystem forward in an open way. We thank the Reth team for their continued contributions.
 
-### Communication:
-- [Telegram](https://t.me/+F_xS9IVOdJZmZjQx)
+[eip_link]: https://eips.ethereum.org/EIPS/eip-4337
 
 ## License
 

docs/SECURITY.md → SECURITY.md

@@ -8,4 +8,4 @@
 
 ## Reporting a Vulnerability
 
-If you would like to report a vulnerability, please open a github issue and tag it with the `security` label.
+If you would like to report a vulnerability, please contact security@alchemy.com

crates/builder/proto/README.md

@@ -1,18 +1,3 @@
-# TODO
+Protocol buffer definitions for the builder module.
 
-# Protocol Buffer Definitions
-
-This directory stores the schema definitions used by Rundler and the supporting gRPC services of it's internal components.
-
-## Building
-
-Rundler builds on [Tonic](https://github.com/hyperium/tonic) to power lightning quick gRPC interfaces between internal components. To re-compile changes to the protocol buffers:
-
-1. Make sure `protoc` is installed locally. On Macs one can run `brew install protobuf`. Otherwise see the official [protobuf docs](https://grpc.io/docs/protoc-installation/) for installation instructions.
-2. Run `cargo build` to recompile the protocol buffers as part of the overall binary build. This is configured in `build.rs` in the `generate_protos` function. Adding schemas in new files will require a change to `build.rs`.
-
-## Protocol Buffer to Memory Representation Translation
-
-## Style Guide
-
-Rundler largely relies on the canonical (protobuf style guide)[https://protobuf.dev/programming-guides/style/]. There are a few minor differences captured in `buf.yaml` to support Tonic best practices. Rundler use (buf)[https://buf.build/] to lint the `.proto` files. To use `buf` run `buf lint protos/.`. To install `buf` on Macs, run `brew install bufbuild/buf/buf`.
+Refer to [proto.md](../../../docs/proto.md) for best practices.

crates/pool/proto/README.md

@@ -1 +1,3 @@
-# TODO
+Protocol buffer definitions for the pool module.
+
+Refer to [proto.md](../../../docs/proto.md) for best practices.

docs/architecture/architecture.md

@@ -0,0 +1,25 @@
+# Rundler Architecture
+
+Rundler consists of 3 main "tasks" that work in tandem to implement the ERC-4337 bundler specification.
+
+- [**RPC**](./rpc.md): The `RPC` task implements the defined [ERC-4337 RPC methods](https://eips.ethereum.org/EIPS/eip-4337#rpc-methods-eth-namespace) under both the `eth_` and `debug_` namespaces. It also implements Rundler specific methods under the `rundler_` namespace and a health check endpoint.
+
+- [**Pool**](./pool.md): The `Pool` task implements a User Operation (UO) mempool. The mempool validates and simulates User Operations per the rules in the spec. It maintains the UOs in memory until they are mined onchain. The pool handles chain reorgs up to a defined depth using a cache of mined UOs.
+
+- [**Builder**](./builder.md): The `Builder` task is responsible for constructing bundles of User Operations, submitting them as transactions, and monitoring the status of those transactions.
+
+## Task Communication
+
+The tasks communicate with each other via message passing mechanisms. The `Pool` and `Builder` each run a "server" component responsible for receiving, acting upon, and responding to messages.
+
+**RPC -> Pool**: The `RPC` module submits UOs to the `Pool` during the `eth_sendUserOperation` RPC method. Future RPC method implementations may use this communication path to retrieve UO status.
+
+**RPC -> Builder**: This communication path is only used when the `debug_` namespace is enabled. It is used to set the builder's bundling mode (to `manual` in tests) and trigger bundle submission (in `manual` mode).
+
+**Builder <-> Pool**: The `Builder` and the `Pool` coordinate to create bundles that contain valid, unmined, and profitable user operations. The `Builder` subscribes to a stream up of updates from the `Pool` and is notified after each new block is processed. The builder then queries the `Pool` for its most valuable UOs, re-simulates them and builds bundles. Any operations that fail 2nd simulation are communicated back to the `Pool` for removal.
+
+## Communication Modes
+
+Both the `Builder` and the `Pool` tasks can be configured to run a gRPC server capable of receiving and responding to messages from the network. Thus, Rundler can be configured to run in a distributed mode where its tasks run in separate processes.
+
+The `Builder` and `RPC` modules can be configured to communicate to other tasks via in-memory message passing (if running in the same process) or via gRPC (if running in separate processes).