Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

E2E flaky: test_prover_transaction_chaining #1396

Open
kpp opened this issue Oct 30, 2024 · 1 comment
Open

E2E flaky: test_prover_transaction_chaining #1396

kpp opened this issue Oct 30, 2024 · 1 comment
Labels
T - bug Something isn't working

Comments

@kpp
Copy link
Contributor

kpp commented Oct 30, 2024

Summary

 thread 'bitcoin_e2e::tx_chain::test_prover_transaction_chaining' panicked at bin/citrea/tests/bitcoin_e2e/tx_chain.rs:349:9:
assertion `left == right` failed: Block should contain exactly 3 transactions

https://github.com/chainwayxyz/citrea/actions/runs/11594946605/job/32282402296?pr=1394#step:10:363
@kpp kpp added the T - bug Something isn't working label Oct 30, 2024
@jfldde
Copy link
Contributor

jfldde commented Oct 31, 2024

In case of failure, sequencer is sending commitments twice.

Succesful logs :

2024-10-31T21:59:06.348145Z  INFO sov_schema_db: Opened RocksDB. rocksdb_name="ledger-db"
2024-10-31T21:59:06.362428Z  INFO sov_schema_db: Opened RocksDB. rocksdb_name="state-db"
2024-10-31T21:59:06.367017Z  INFO sov_schema_db: Opened RocksDB. rocksdb_name="native"
2024-10-31T21:59:06.369448Z  INFO citrea::rollup: Initialize sequencer at genesis.
2024-10-31T21:59:06.369535Z  INFO citrea_sequencer::runner: No history detected. Initializing chain...
2024-10-31T21:59:06.381429Z  INFO citrea_sequencer::runner: Chain initialization is done. Genesis root: 0xdacb59b0ff5d16985a8418235133eee37758a3ac1b76ab6d1f87c6df20e4d4da
2024-10-31T21:59:06.418328Z  INFO citrea_sequencer::runner: Starting RPC server at 127.0.0.1:65172 
2024-10-31T21:59:08.329895Z  INFO Sequencer: citrea_sequencer::runner: New block #1, DA #168, Tx count: #0
2024-10-31T21:59:08.416463Z  INFO Sequencer: citrea_sequencer::runner: New block #2, DA #168, Tx count: #0
2024-10-31T21:59:08.519171Z  INFO Sequencer: citrea_sequencer::runner: New block #3, DA #168, Tx count: #0
2024-10-31T21:59:08.623090Z  INFO Sequencer: citrea_sequencer::runner: New block #4, DA #168, Tx count: #0
2024-10-31T21:59:08.662692Z  INFO citrea_sequencer::commitment: Sent commitment to DA queue. L2 range: #1-4
reveal_script_builder: OP_PUSHBYTES_32 f1f5e2677b274eab88f10b4b3279e1dd4ea9643d679997d8a4a9b6b85104b576 OP_CHECKSIGVERIFY OP_PUSHBYTES_2 0000 OP_0 OP_IF OP_PUSHBYTES_64 0b906846607a24d5dcf0bfb5165c90df53d53d716e1eeef3a871e5a17ead603023548090c326b29c94dbf332698b98defdfc54427e11d2e76cf64b70494d476c OP_PUSHBYTES_33 0388c3120bbb1925266db8edf7d083dba24623aeea2feff73c33aef2f770bade1e OP_PUSHBYTES_49 00d9507cfc6d76aa08b388c41ada20c07fbe8cb6d6fc4adcebda62a344f5d28e6c01000000000000000400000000000000 OP_ENDIF
2024-10-31T21:59:10.633898Z  INFO bitcoin_da::service: Blob inscribe tx sent. Hash: 490b09ef82ec633bca34e05899ccd021f2fd2c0c34de87a94ba5c23c1aba4b03
2024-10-31T21:59:10.633931Z  INFO bitcoin_da::service: Sent tx to BitcoinDA tx.id=490b09ef82ec633bca34e05899ccd021f2fd2c0c34de87a94ba5c23c1aba4b03
2024-10-31T21:59:10.634174Z  INFO citrea_sequencer::commitment: New commitment. L2 range: #1-4
2024-10-31T21:59:10.784102Z  WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 7 blocks
2024-10-31T21:59:43.879075Z  INFO Sequencer: citrea_sequencer::runner: New block #5, DA #169, Tx count: #0
2024-10-31T21:59:43.890182Z  INFO Sequencer: citrea_sequencer::runner: New block #6, DA #170, Tx count: #0
2024-10-31T21:59:43.900832Z  INFO Sequencer: citrea_sequencer::runner: New block #7, DA #171, Tx count: #0
2024-10-31T21:59:43.910849Z  INFO Sequencer: citrea_sequencer::runner: New block #8, DA #172, Tx count: #0
2024-10-31T21:59:43.920423Z  INFO Sequencer: citrea_sequencer::runner: New block #9, DA #173, Tx count: #0
2024-10-31T21:59:43.929874Z  INFO Sequencer: citrea_sequencer::runner: New block #10, DA #174, Tx count: #0
2024-10-31T21:59:43.938828Z  INFO Sequencer: citrea_sequencer::runner: New block #11, DA #175, Tx count: #0
2024-10-31T21:59:43.947174Z  INFO Sequencer: citrea_sequencer::runner: New block #12, DA #176, Tx count: #0
2024-10-31T21:59:43.978396Z  INFO Sequencer: citrea_sequencer::runner: New block #13, DA #177, Tx count: #0
2024-10-31T21:59:43.991372Z  INFO citrea_sequencer::commitment: Sent commitment to DA queue. L2 range: #5-12
reveal_script_builder: OP_PUSHBYTES_32 51711c392ca2df61bd323202a1e78695d0e42b54727a44ef057696ea5c206822 OP_CHECKSIGVERIFY OP_PUSHBYTES_2 0000 OP_0 OP_IF OP_PUSHBYTES_64 d81562b28f408bd2d382a757b27ffaa036af1b701087a1d0956bb07da2801c685bef370d13fd649684f2cc86fd68d616418d4a8adbd20cb92961b2e66b176e72 OP_PUSHBYTES_33 0388c3120bbb1925266db8edf7d083dba24623aeea2feff73c33aef2f770bade1e OP_PUSHBYTES_49 0004061db629412dc0e76d2365f9b0981b669ba2856b2d95f334768d0a8eaea8c505000000000000000c00000000000000 OP_ENDIF
2024-10-31T21:59:44.075173Z  INFO Sequencer: citrea_sequencer::runner: New block #14, DA #177, Tx count: #0
2024-10-31T21:59:44.178082Z  INFO Sequencer: citrea_sequencer::runner: New block #15, DA #177, Tx count: #0
2024-10-31T21:59:51.271626Z  INFO bitcoin_da::service: Blob inscribe tx sent. Hash: 498149ce52f217f612741c19c810e6e6df152217edc05f091e60bf48bee39a99
2024-10-31T21:59:51.271877Z  INFO bitcoin_da::service: Sent tx to BitcoinDA tx.id=498149ce52f217f612741c19c810e6e6df152217edc05f091e60bf48bee39a99
2024-10-31T21:59:51.272112Z  INFO citrea_sequencer::commitment: New commitment. L2 range: #5-12
2024-10-31T21:59:51.408002Z  WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 7 blocks

Failing test logs :

2024-10-31T21:31:58.792409Z  INFO sov_schema_db: Opened RocksDB. rocksdb_name="ledger-db"
2024-10-31T21:31:58.806229Z  INFO sov_schema_db: Opened RocksDB. rocksdb_name="state-db"
2024-10-31T21:31:58.810614Z  INFO sov_schema_db: Opened RocksDB. rocksdb_name="native"
2024-10-31T21:31:58.812905Z  INFO citrea::rollup: Initialize sequencer at genesis.
2024-10-31T21:31:58.812969Z  INFO citrea_sequencer::runner: No history detected. Initializing chain...
2024-10-31T21:31:58.824588Z  INFO citrea_sequencer::runner: Chain initialization is done. Genesis root: 0xdacb59b0ff5d16985a8418235133eee37758a3ac1b76ab6d1f87c6df20e4d4da
2024-10-31T21:31:58.859329Z  INFO citrea_sequencer::runner: Starting RPC server at 127.0.0.1:64835 
2024-10-31T21:32:00.763803Z  INFO Sequencer: citrea_sequencer::runner: New block #1, DA #168, Tx count: #0
2024-10-31T21:32:00.843714Z  INFO Sequencer: citrea_sequencer::runner: New block #2, DA #168, Tx count: #0
2024-10-31T21:32:00.947128Z  INFO Sequencer: citrea_sequencer::runner: New block #3, DA #168, Tx count: #0
2024-10-31T21:32:01.049713Z  INFO Sequencer: citrea_sequencer::runner: New block #4, DA #168, Tx count: #0
2024-10-31T21:32:01.093352Z  INFO citrea_sequencer::commitment: Sent commitment to DA queue. L2 range: #1-4
reveal_script_builder: OP_PUSHBYTES_32 15334935829f2c0c665d2bfd215bf0b663ec61b0b05032f0cbbcf0d09b9bb488 OP_CHECKSIGVERIFY OP_PUSHBYTES_2 0000 OP_0 OP_IF OP_PUSHBYTES_64 f69d6092bc14afd3fdd3e8b13aa66c3eb198b94df653f74bf1b31a8077014f093c3d2ae763f3920a8c7a70cf22cff8fd5673ba04669ae539ea0cdb76286314af OP_PUSHBYTES_33 0388c3120bbb1925266db8edf7d083dba24623aeea2feff73c33aef2f770bade1e OP_PUSHBYTES_49 000de8f06294171362e02ccf7408cf10a9b0614cb4fd6fcc80a9d629c19b1b577c01000000000000000400000000000000 OP_ENDIF
2024-10-31T21:32:05.540495Z  INFO bitcoin_da::service: Blob inscribe tx sent. Hash: 92461a1d5d4fceb64e72dad7c9f845f36116ee8e1fba351286a979041c7a4422
2024-10-31T21:32:05.540535Z  INFO bitcoin_da::service: Sent tx to BitcoinDA tx.id=92461a1d5d4fceb64e72dad7c9f845f36116ee8e1fba351286a979041c7a4422
2024-10-31T21:32:05.540840Z  INFO citrea_sequencer::commitment: New commitment. L2 range: #1-4
2024-10-31T21:32:05.705872Z  WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 4 blocks
2024-10-31T21:32:26.805183Z  INFO Sequencer: citrea_sequencer::runner: New block #5, DA #169, Tx count: #0
2024-10-31T21:32:26.820144Z  INFO Sequencer: citrea_sequencer::runner: New block #6, DA #170, Tx count: #0
2024-10-31T21:32:26.831604Z  INFO Sequencer: citrea_sequencer::runner: New block #7, DA #171, Tx count: #0
2024-10-31T21:32:26.842299Z  INFO Sequencer: citrea_sequencer::runner: New block #8, DA #172, Tx count: #0
2024-10-31T21:32:26.851563Z  INFO Sequencer: citrea_sequencer::runner: New block #9, DA #173, Tx count: #0
2024-10-31T21:32:26.851877Z  WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 3 blocks
2024-10-31T21:32:26.893892Z  INFO Sequencer: citrea_sequencer::runner: New block #10, DA #174, Tx count: #0
2024-10-31T21:32:26.897544Z  INFO citrea_sequencer::commitment: Sent commitment to DA queue. L2 range: #5-9
reveal_script_builder: OP_PUSHBYTES_32 79250755d56141467cc9b9ceae0f952ca684108bbae5019c6abd20e7c02932b0 OP_CHECKSIGVERIFY OP_PUSHBYTES_2 0000 OP_0 OP_IF OP_PUSHBYTES_64 0808354fddade660f5d210e7fac45b7b2957c624e6fc1fb61acc3efb0698a9eb44cb48ad7ac5e4abe18cde0ece3272a759f9681ef385e06e3de2b1cd8d193dcf OP_PUSHBYTES_33 0388c3120bbb1925266db8edf7d083dba24623aeea2feff73c33aef2f770bade1e OP_PUSHBYTES_49 00b42c9dc1d87a58f82fce427db2c3946cef4e8f3394e8a886a57d7be5f8ec7fba05000000000000000900000000000000 OP_ENDIF
2024-10-31T21:32:26.902863Z  INFO Sequencer: citrea_sequencer::runner: New block #11, DA #175, Tx count: #0
2024-10-31T21:32:26.912130Z  INFO Sequencer: citrea_sequencer::runner: New block #12, DA #176, Tx count: #0
2024-10-31T21:32:26.920341Z  INFO Sequencer: citrea_sequencer::runner: New block #13, DA #177, Tx count: #0
2024-10-31T21:32:26.966313Z  INFO citrea_sequencer::commitment: Sent commitment to DA queue. L2 range: #10-13
2024-10-31T21:32:26.990312Z  INFO Sequencer: citrea_sequencer::runner: New block #14, DA #177, Tx count: #0
2024-10-31T21:32:27.093267Z  INFO Sequencer: citrea_sequencer::runner: New block #15, DA #177, Tx count: #0
2024-10-31T21:32:40.722023Z  INFO bitcoin_da::service: Blob inscribe tx sent. Hash: 06ad3b68845e4340ed89e2ba2b12475411e7623c9b752032263f1a5da3719a7a
2024-10-31T21:32:40.722051Z  INFO bitcoin_da::service: Sent tx to BitcoinDA tx.id=06ad3b68845e4340ed89e2ba2b12475411e7623c9b752032263f1a5da3719a7a
2024-10-31T21:32:40.722326Z  INFO citrea_sequencer::commitment: New commitment. L2 range: #5-9
reveal_script_builder: OP_PUSHBYTES_32 fe49b9b3607cfa417c773636a3343d8f9d0fdec2ead1c7adc4f2cd251a95c7d7 OP_CHECKSIGVERIFY OP_PUSHBYTES_2 0000 OP_0 OP_IF OP_PUSHBYTES_64 07cbfab25b2030e6cef1c463760ba090d1c41bef9bdb85e3fd0c3f8e13a5ad90506393a02f26b4ce2f3c832abcda5826cc462ba7bb8175a6828aa0e9c2ce11e2 OP_PUSHBYTES_33 0388c3120bbb1925266db8edf7d083dba24623aeea2feff73c33aef2f770bade1e OP_PUSHBYTES_49 0092e57b68c904f663937a434410265a4b834ada1d6f5d9677937f5d94238eee920a000000000000000d00000000000000 OP_ENDIF
2024-10-31T21:32:40.765718Z  WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 4 blocks
2024-10-31T21:33:04.232417Z  INFO bitcoin_da::service: Blob inscribe tx sent. Hash: 7be97dfe57315bb707b19f738fcf5357fd6fbc7d98dbb5303d478746835a2abe
2024-10-31T21:33:04.232752Z  INFO bitcoin_da::service: Sent tx to BitcoinDA tx.id=7be97dfe57315bb707b19f738fcf5357fd6fbc7d98dbb5303d478746835a2abe
2024-10-31T21:33:04.233142Z  INFO citrea_sequencer::commitment: New commitment. L2 range: #10-13

Main difference being :
WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 7 blocks
vs
2024-10-31T21:32:05.705872Z WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 4 blocks
2024-10-31T21:32:26.851877Z WARN Sequencer: citrea_sequencer::runner: Sequencer is falling behind on L1 blocks by 3 blocks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
T - bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kpp @jfldde