diff --git a/LIGHT_TESTING.md b/LIGHT_TESTING.md new file mode 100644 index 0000000..12c928c --- /dev/null +++ b/LIGHT_TESTING.md @@ -0,0 +1,188 @@ +# Light-client Testing guide + +Melodot Light-client includes both unit tests and e2e tests, which can be executed locally or within a Docker container. + +## ****Local Testing**** + +Ensure you have the necessary environment set up for Rust. + +### ****Unit Tests**** + +Run all unit tests using the following command: + +```bash +make test +``` + +### Building + +We need to compile two projects: the melodot node and the light-client. + +Known issue: Unable to compile under Mac environment, we will fix this issue later. + +1. Build the melodot node using the following command, this may take some time: + +```bash +make build-default +``` + +2. Compile the light-client using the following command: + +```bash +make build-light +``` + +### ****Launching the Development Network**** + +To initiate the development network, use the command below: + +```bash +make run-dev +``` + +Once you observe the following output, it indicates that the development network is up and running, and blocks are being produced normally: + +```bash +2023-11-09 21:36:31 Melodot Node +2023-11-09 21:36:31 โœŒ๏ธ version 0.0.1-1df3a1f033a +2023-11-09 21:36:31 โค๏ธ by DKLee , 2017-2023 +2023-11-09 21:36:31 ๐Ÿ“‹ Chain specification: Development +2023-11-09 21:36:31 ๐Ÿท Node name: accessible-cobweb-6597 +2023-11-09 21:36:31 ๐Ÿ‘ค Role: AUTHORITY +2023-11-09 21:36:31 ๐Ÿ’พ Database: RocksDb at /tmp/substrateJ6MN0y/chains/dev/db/full +2023-11-09 21:36:31 โ›“ Native runtime: melodot-1 (melodot-1.tx1.au1) +2023-11-09 21:36:32 [0] ๐Ÿ’ธ generated 1 npos voters, 1 from validators and 0 nominators +2023-11-09 21:36:32 [0] ๐Ÿ’ธ generated 1 npos targets +2023-11-09 21:36:33 ๐Ÿ”จ Initializing Genesis block/state (state: 0x2538โ€ฆ5e46, header-hash: 0xac37โ€ฆe2d5) +2023-11-09 21:36:33 ๐Ÿ‘ด Loading GRANDPA authority set from genesis on what appears to be first startup. +2023-11-09 21:36:34 ๐Ÿ‘ถ Creating empty BABE epoch changes on what appears to be first startup. +2023-11-09 21:36:34 Failed to register metrics: Duplicate metrics collector registration attempted +2023-11-09 21:36:34 Using default protocol ID "sup" because none is configured in the chain specs +2023-11-09 21:36:34 ๐Ÿท Local node identity is: 12D3KooWF8kNQjNivebHiCnTvkACt2SrNW6uEkJbyxWqzu1PAbVg +2023-11-09 21:36:34 ๐Ÿš€ Starting transaction pool listener. +2023-11-09 21:36:34 ๐Ÿ’ป Operating system: linux +2023-11-09 21:36:34 ๐Ÿ’ป CPU architecture: x86_64 +2023-11-09 21:36:34 ๐Ÿ’ป Target environment: gnu +2023-11-09 21:36:34 ๐Ÿ’ป CPU: Intel(R) Xeon(R) Platinum +2023-11-09 21:36:34 ๐Ÿ’ป CPU cores: 4 +2023-11-09 21:36:34 ๐Ÿ’ป Memory: 7283MB +2023-11-09 21:36:34 ๐Ÿ’ป Kernel: 5.15.0-79-generic +2023-11-09 21:36:34 ๐Ÿ’ป Linux distribution: Ubuntu 22.04.3 LTS +2023-11-09 21:36:34 ๐Ÿ’ป Virtual machine: yes +2023-11-09 21:36:34 ๐Ÿ“ฆ Highest known block at #0 +2023-11-09 21:36:34 ใ€ฝ๏ธ Prometheus exporter started at 127.0.0.1:9615 +2023-11-09 21:36:34 Running JSON-RPC HTTP server: addr=127.0.0.1:9933, allowed origins=["*"] +2023-11-09 21:36:34 Running JSON-RPC WS server: addr=0.0.0.0:9944, allowed origins=["*"] +2023-11-09 21:36:34 ๐Ÿ‘ถ Starting BABE Authorship worker +2023-11-09 21:36:36 ๐Ÿ™Œ Starting consensus session on top of parent 0xac37c22f067cbea3a82a9952ed61a40a0a32eabb4a46fa96ebc230e63855e2d5 +2023-11-09 21:36:36 ๐ŸŽ Prepared block for proposing at 1 (0 ms) [hash: 0x94cbc9ee49438476b9291ed9ea4dd722201bff89aaa61c803a6a484e218e3c82; parent_hash: 0xac37โ€ฆe2d5; extrinsics (1): [0x6d27โ€ฆd03d]] +2023-11-09 21:36:36 ๐Ÿ”– Pre-sealed block for proposal at 1. Hash now 0x0b832715fa87a6e813606832ab364150830465fae6fd43f9b740763ba0eba75a, previously 0x94cbc9ee49438476b9291ed9ea4dd722201bff89aaa61c803a6a484e218e3c82. +2023-11-09 21:36:36 ๐Ÿ‘ถ New epoch 0 launching at block 0x0b83โ€ฆa75a (block slot 283256166 >= start slot 283256166). +2023-11-09 21:36:36 ๐Ÿ‘ถ Next epoch starts at slot 283256766 +2023-11-09 21:36:36 ๐Ÿ˜ด Block 1 has no blob +2023-11-09 21:36:36 โœจ Imported #1 (0x0b83โ€ฆa75a) +2023-11-09 21:36:38 Accepting new connection 1/100 +2023-11-09 21:36:39 discovered: 12D3KooW9wv5DVBvtUv9fy46PCkEDo9K1jyXzPS3SKiBbhW4rfty /ip4/172.19.0.1/tcp/4418 +2023-11-09 21:36:39 discovered: 12D3KooW9wv5DVBvtUv9fy46PCkEDo9K1jyXzPS3SKiBbhW4rfty /ip4/172.16.7.77/tcp/4418 +2023-11-09 21:36:39 discovered: 12D3KooW9wv5DVBvtUv9fy46PCkEDo9K1jyXzPS3SKiBbhW4rfty /ip4/172.17.0.1/tcp/4418 +2023-11-09 21:36:39 ๐Ÿ’ค Idle (0 peers), best: #1 (0x0b83โ€ฆa75a), finalized #0 (0xac37โ€ฆe2d5), โฌ‡ 0 โฌ† 0 +2023-11-09 21:36:42 ๐Ÿ™Œ Starting consensus session on top of parent 0x0b832715fa87a6e813606832ab364150830465fae6fd43f9b740763ba0eba75a +2023-11-09 21:36:42 ๐ŸŽ Prepared block for proposing at 2 (0 ms) [hash: 0x28b41376f2b51efd8def9083bffc3e5c5f98f15d266dfb1986172de9e09e26fc; parent_hash: 0x0b83โ€ฆa75a; extrinsics (1): [0x57d3โ€ฆa11a]] +2023-11-09 21:36:42 ๐Ÿ”– Pre-sealed block for proposal at 2. Hash now 0xb0134001cfed9449650f3c8c6af26230dd2d6ac682b06391b5ec4187c4e365ff, previously 0x28b41376f2b51efd8def9083bffc3e5c5f98f15d266dfb1986172de9e09e26fc. +2023-11-09 21:36:42 ๐Ÿ˜ด Block 2 has no blob +2023-11-09 21:36:42 โœจ Imported #2 (0xb013โ€ฆ65ff) +2023-11-09 21:36:44 ๐Ÿ’ค Idle (0 peers), best: #2 (0xb013โ€ฆ65ff), finalized #0 (0xac37โ€ฆe2d5), โฌ‡ 0 โฌ† 0 +2023-11-09 21:36:48 ๐Ÿ™Œ Starting consensus session on top of parent 0xb0134001cfed9449650f3c8c6af26230dd2d6ac682b06391b5ec4187c4e365ff +2023-11-09 21:36:48 ๐ŸŽ Prepared block for proposing at 3 (0 ms) [hash: 0x73880cfd4f67132321ac78829d30389869c233e8ba528c6b19627ef4b7db8c48; parent_hash: 0xb013โ€ฆ65ff; extrinsics (1): [0xb35bโ€ฆ16f5]] +``` + +### ****Running Light-clent**** + +Open a new terminal and execute the following command to run the light-client: + +```bash +make run-light-dev +``` + +Once you observe the following output, it indicates that the light-client is up and running: + +```bash + INFO ๐Ÿš€ Melodot Light Client starting up + INFO ๐Ÿ‘‚ RPC server started at: 127.0.0.1:4177 + INFO creating instance on iface 172.16.7.77 + INFO creating instance on iface 172.19.0.1 + INFO creating instance on iface 172.17.0.1 + INFO ๐ŸŒ Subscribed to finalized block headers + INFO โœ… Received finalized block header #0 + INFO discovered: 12D3KooWKdiBnPzEWuXEk6nwJvmxXt7QrkC71eCMBEXP1jBQiYgf /ip4/172.17.0.1/tcp/4417 + INFO discovered: 12D3KooWKdiBnPzEWuXEk6nwJvmxXt7QrkC71eCMBEXP1jBQiYgf /ip4/172.16.7.77/tcp/4417 + INFO discovered: 12D3KooWKdiBnPzEWuXEk6nwJvmxXt7QrkC71eCMBEXP1jBQiYgf /ip4/172.19.0.1/tcp/4417 + INFO connection{remote_addr=127.0.0.1:38114 conn_id=0}: Accepting new connection 1/100 + INFO โœ… Received finalized block header #1 + INFO โœ… Received finalized block header #2 + INFO โœ… Received finalized block header #3 +``` + +### ****Running e2e Tests**** + +Ensure the test network and light-client are running. Open a new terminal and execute the e2e tests using the following command: + +```bash +make run-light-e2e +``` + +This will start the e2e tests, submitting data transactions to the node, and obtaining light-client sample data when the block is finalized to verify if the data is indeed available. Afterward, it will submit another data transaction and delete most of the data from the network after it is successfully included in the block, to verify if the light-client can correctly identify the unavailability of data through sampling. If you see the following output, it indicates all tests have been successfully completed: + +```bash + INFO ๐Ÿš€ Melodot Light Client e2e starting up + INFO ๐ŸŒŸ Start: Running data availability + INFO โœ… Success Application created, block hash: 0xb3c3โ€ฆd118 + INFO โœ… Success: Data submitted, tx_hash: 0x986f0fea84a91a7b5eb78228df50870580b01895979dd1acb64a4808928ddeab + INFO โณ Data not verified yet, current block number: 6 + INFO โณ Data not verified yet, current block number: 7 + INFO โœ… Success Data should have been verified by the validators at: 8 + INFO โณ Data not finalized yet, current block number: 5 + INFO โณ Data not finalized yet, current block number: 6 + INFO โณ Data not finalized yet, current block number: 7 + INFO โœ… Success Data finalized at block: 8 + INFO โณ Wait for the sampling to complete. + INFO โœ… Success: Block confidence is above 99.99%: 999985 + INFO ๐Ÿ’ฏ All success : Module data_availability + INFO ๐ŸŒŸ Start: Running data_unavailable + INFO โœ… Success: Data submitted, tx_hash: 0x9d1818b12bfcdfa34ae9943903f7bb2044ab05e8bdb062a95a303811b26eb0b5 + INFO โณ Data not verified yet, current block number: 10 + INFO โณ Data not verified yet, current block number: 11 + INFO โœ… Success Data should have been verified by the validators at: 12 + INFO โณ: Waiting for data to be propagated across the network. + INFO โœ… Success: 75% of data has been deleted + INFO โณ Data not finalized yet, current block number: 10 + INFO โณ Data not finalized yet, current block number: 11 + INFO โœ… Success Data finalized at block: 12 + INFO โณ: Wait for the sampling to complete. + INFO โœ… Success: Block confidence is less than 99.99%: 750000 + INFO ๐Ÿ’ฏ All success: Module data_unavailable +``` + +## **Using Docker** + +First, install Docker and Docker Compose. + +You need to run the following commands in the root directory of **`melodot`**: + +```bash +./scripts/docker_run.sh +``` + +This command will build a Docker image and start a Docker container. Within the container, you can carry out the same steps as in the previous section for unit testing, building, and running the development network. + +You can then open a new Docker terminal with the following command, running the light-client. + +```bash +./scripts/docker_run.sh new +``` + +Finally, open another new Docker terminal to run the e2e tests: + +```bash +make run-light-e2e +``` diff --git a/README.md b/README.md index fb4ea4f..c848540 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,8 @@ mac brew install cmake pkg-config openssl git llvm ``` +ๅทฒ็Ÿฅ้—ฎ้ข˜๏ผš ็›ฎๅ‰ๆ— ๆณ•ๅœจ mac ไธŠ็ผ–่ฏ‘๏ผŒๆˆ‘ไปฌ็จๅŽ่งฃๅ†ณ่ฟ™ไธช้—ฎ้ข˜ใ€‚ + Linux ```bash @@ -48,8 +50,16 @@ More๏ผšMelodot is based on Substrate, for more information please go toย [Substr ### Build +1. ็ผ–่ฏ‘ melodot-node + +```bash +make build-default +``` + +2. ็ผ–่ฏ‘่ฝป่Š‚็‚น๏ผŒ่ฟ™ๅฏ่ƒฝไผš่‡ชๅŠจๅฎ‰่ฃ… sqlite ๆ•ฐๆฎๅบ“ + ```bash -make build-release +make build-light ``` ## 3. Run @@ -60,6 +70,12 @@ You can start a development chain with: make run-dev ``` +ๅฏๅŠจ่ฝป่Š‚็‚น + +```bash +make run-light-dev +``` + ## 4. Development ### Test All @@ -70,7 +86,7 @@ Use the following command to run all tests: make test ``` -You can learn more detailed testing methods from the [testing guide](./TESTING.md). +You can learn more detailed testing methods from the [testing guide](./TESTING.md) and [light client testing guide](./LIGHT_TESTING.md). ## 5. Docker @@ -80,7 +96,7 @@ Start a Docker container: ./scripts/docker_run.sh ``` -You can learn more about Docker examples from the [testing guide](./TESTING.md). +You can learn more about Docker examples from the [testing guide](./TESTING.md) and [light client testing guide](./LIGHT_TESTING.md). ## Reference diff --git a/crates/daser/README.MD b/crates/daser/README.MD new file mode 100644 index 0000000..eaaa491 --- /dev/null +++ b/crates/daser/README.MD @@ -0,0 +1,3 @@ +# Melodot Daser + +An abstract data availability sampler that can be used in light clients, full nodes, and farmer clients. It wraps the `DasNetwork` used for DAS and provides commonly used sampling operations. Additionally, it offers a transaction pool listener and implements sampling for both application data and finalized block data. \ No newline at end of file diff --git a/light/README.MD b/light/README.MD new file mode 100644 index 0000000..a609780 --- /dev/null +++ b/light/README.MD @@ -0,0 +1,67 @@ +# Melodot Light Client + +## 1. Introduction + +The Melodot network's light client is currently operational in the development network. Light nodes can access the finalized block headers from the Melodot network through RPC and validate the data's authenticity by sampling. + +## 2. Operating Mechanism + +Melodot light nodes employ sampling to assess data availability. They do not need to trust any nodes or download the entire dataset. + +Light nodes initially acquire the latest finalized block header through a full node. They then construct sample IDs using the metadata in the block header. Subsequently, these random samples are sourced from a separate data network and validated against the KZG commitments in the block header. + +## 3. Building + +Building from source + +```bash +git clone git@github.com:ZeroDAO/melodot.git +cd melodot +make build-light +``` + +## 4. Running + +First, ensure that the Melodot development network is running, then execute the light node to join the development network + +```bash +make run-light-dev +``` + +## 5. RPC Interface + +### POST /das_blockConfidence + +Retrieve the confidence level for a block hash, returning a confidence level measured in parts per million. + +```bash +curl --data '{"method":"das_blockConfidence","params":["0xc964c3636fdf33bcc4ccc7ad854b32862e02ec50f02a00ba43f0b02c4fbb67e3"],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST localhost:4177 +``` + +***Responses*** + +```json +{"jsonrpc":"2.0","result":937500,"id":1} +``` + +### POST /das_isAvailable + +Check if the data for a block hash is available. A return of null indicates no data for that block. + +```bash +curl --data '{"method":"das_isAvailable","params":["0xc964c3636fdf33bcc4ccc7ad854b32862e02ec50f02a00ba43f0b02c4fbb67e3"],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST localhost:4177 +``` + +***Responses*** + +```json +{"jsonrpc":"2.0","result":false,"id":1} +``` + +## 5. Planned Features + +- [ ] Validate finalized block headers +- [ ] Send transactions and push data to the network +- [ ] Retrieve application data using nonce and app_id +- [ ] Access data by rows and columns +- [ ] Support data reconstruction \ No newline at end of file