This tool uses Ethereum's peer-discovery protocol to measure the size of the Ethereum network (testnets included)
The light-crawler creates an Ethereum node with a new generated ecdsa Private Key to launch a Discovery5 service.
Once the node and the service are running, the light-crawler will request random ENRs from the network, indexing in an SQL database all the different atributes from the discovered Nodes (we index all the available ENRs in the network, or in other words, all the Ethereum networks).
- (recomended) Go1.17+
- (must) make
- (must) local or remote postgreSQL database
Once the repo is cloned, you can compile it and generate a binary running:
$ make build
Once the binary was generated, you can execute it by running:
USAGE:
./build/eth-light-crawler [subcommands] [arguments]
COMMANDS:
discv5 crawl Ethereum's public DHT thought the Discovery 5.1 protocol
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help (default: false)
The current crawler only support the discovery5.1 protocol, so to crawl it your can run the following command:
USAGE:
./build/eth-light-crawler discv5 [arguments...]
OPTIONS:
--log-level value verbosity of the logs that will be displayed [debug,warn,info,error] (default: "info") [$IPFS_CID_HOARDER_LOGLEVEL]
--db-endpoint value login endpoint to the database (default: "postgres://test:password@localhost:5432/eth_light_crawler") [$IPFS_CID_HOARDER_DB_ENDPOINT]
--port value port number that we want to use/advertise in the Ethereum network (default: 9001)
--reset-db reset the content of the db tables (default: false)
--help, -h show help (default: false)
NOTE: the light-crawler will require to have a postgreSQL database created before running it, it will only create the required tables to run.
Miga Labs / @cortze
This tool is open for any kind of feedback and contribution, so please feel free to approach us!