Second design for a proof network #197

markspanbroek · 2024-10-15T14:30:22Z

Improves on the previous design (#194) by:

combining the roles of validator and watcher (from our discussions at the offsite)
using a full consensus algorithm
selecting who goes on chain to mark a proof as missing
reducing computational overhead of signing each proof

bkomuves · 2024-10-15T16:53:09Z

design/proof-network.md

+There are a couple of advantages to this design in which we use a consensus
+protocol to sequence transactions.
+
+A storage provider only needs to send its proofs to a single validator, and


but any validator can decide to censor any provider (and especially if the same SPs play both roles, this can very easily happen), so sending to a single validator seems very dangerous to me

That is correct. I thought about that, but didn't write it down. When the validator doesn't give you an inclusion proof, you can simply switch to another validator.

I'll update the text to include this.

fredericosteixeira · 2024-10-22T17:27:04Z

design/proof-network.md

+The marketplace will then verify the correctness of *inputs* for the *slot id*
+and *period*, and checks that the combined signature is representative for > 2/3
+stake. If these conditions are met, it will then slash the storage provider
+collateral and reward the validator.


Is only the validator who sent the 1st MissingProof transaction rewarded?

Yes, that would be the simplest I think. The Mysticeti design suggests ordering the blocks from the validators based on a round-robin scheme, so every validator is equally capable of being the first to notice a proof as missing.

fredericosteixeira · 2024-10-22T17:27:19Z

design/proof-network.md

+
+### Faulty proofs ###
+
+The storage proofs that a storage provider submits can be faulty for a number of


Is it possible to assign different penalties to these faulty proofs?

i don't think that would make sense, a proof is either present and valid, or not. There are no "qualities" of missing/wrong proofs.

A more interesting case when storage providers also aggregate their proofs. Then it can happen that for example they submit a single proof for 100 slots, but only 99 is valid because they actually lost 1.

fredericosteixeira · 2024-10-22T17:28:06Z

design/proof-network.md

+[Mysticeti][6] algorithm seems to be particularly suited because it is highly
+performant.
+
+The consensus algorithm is only used to sequence transactions. Evaluation of the


Out of curiosity: why does sequencing transactions matter?

I guess, maybe to match the right/current period? If a transaction is added too late will it miss the period?

Sequencing is not strictly necessary for storage proofs, but for anything else it becomes necessary.

See my reply here: https://github.com/codex-storage/codex-research/pull/197/files#r1820368545

fredericosteixeira · 2024-10-22T17:28:48Z

design/proof-network.md

+The consensus algorithm is only used to sequence transactions. Evaluation of the
+transactions is done after sequencing. This means that storage proofs can be
+checked in parallel, which allows validators to scale up and support a large
+storage network that produces many proofs.


Do validators post onchain any result of these evaluations?

No, they don't.

fredericosteixeira · 2024-10-22T17:30:08Z

design/proof-network.md

+The marketplace smart contract only slashes a storage provider when there is a
+combined BLS signature that represents > 2/3 stake.
+
+It can be expensive to calculate the amount of stake associated with a combined


If the stake amount is constant, would be possible to confirm that >2/3 threshold by simply counting the number of signatures?

Yes, that is possible. However, you'd still need to figure out from how many signatures the combined signature was constructed, and that they are valid signatures from staked validators.

fredericosteixeira · 2024-10-22T17:34:59Z

design/proof-network.md

+introduce epochs in which some validators are chosen from a larger set of
+potential validators, but that comes at the expense of added complexity.
+
+The latency is larger than in a [previous iteration][7] of this design, because


If SPs are required to post 1 proof/day, validators would have enough time to reach consensus, wouldn't they?

It is 1 proof per dataset right? Not per SP.

If SPs are required to post 1 proof/day, validators would have enough time to reach consensus, wouldn't they?

Yes, latency is not really a problem for our use case.

It is 1 proof per dataset right? Not per SP.

Currently yes, but if we succeed in doing local proof aggregation, then it becomes one proof per storage provider.

leobago

Very well written @markspanbroek, very clear and easy to follow.

I see a lot of potential danger going through this route. It seems to me that we are introducing a whole new colossal attack surface by implementing a blockchain. I see very few advantages to this approach in comparison with storing on an L2.

leobago · 2024-10-23T20:12:55Z

design/proof-network.md

+
+A storage provider sends a `StorageProof` transaction to a validator to indicate
+to the network that it calculated a storage proof as required by the
+marketplace. This validator includes it in its next block.


The proof is sent to only one validator or the entire network of validators? This validator includes it in its next block? Or rather the validator in charge of producing the next block adds the transaction to it?

The proof is sent to only one validator. In the Mysticeti design, all validators produce a block each round, so the proof is included in the next block of the validator that you sent it to.

If it fails to provide you with proof that it included the transaction, then you're free to go to another validator.

How are proofs/transactions propagated across validators?

How are proofs/transactions propagated across validators?

As part of the consensus protocol blocks of transactions are exchanged between validators.

leobago · 2024-10-23T20:14:08Z

design/proof-network.md

+required for a slot in the current *period*, it gathers public *inputs* for the
+slot, including the random challenge and calculates a zero-knowledge storage
+*proof*. The provider then submits `StorageProof(slot id, period, inputs,
+proof)` to a single validator.


Why a single validator? What if this one is malicious?

Each validator produces a block every round in the Mysticeti design, and the proof only needs to end up in one of them. If the validator is malicious, it won't be able to give an inclusion proof for the transaction, and you can simply go to another validator to get the transaction included.

leobago · 2024-10-23T20:15:04Z

design/proof-network.md

+    storage provider  --------------------------------------->  validator
+
+
+The validator will include the transaction in the next block that it proposes to


How many validators are we talking about? How often do they produce blocks?

Not sure yet how many. In the Mysticeti paper they measured less than 5 validators to get sub-second latency numbers. But we don't really care about latency so much, so we can go higher than that. But I don't think it's practical to go for more than 100 validators, unless we go for a scheme where we pick a few validators from a larger pool every epoch.

leobago · 2024-10-23T20:19:22Z

design/proof-network.md

+
+The validator that sent the first `MissingProof` transaction for a missing proof
+can now request BLS signatures from the other validators to enact on-chain
+slashing of the storage provider for missing a proof. When the validator


Do we slash SP for every single proof missed? Even the very first one?

No, I do not want to change the rules around slashing with this. So I'd keep the way we slash in the Marketplace smart contract the same. Only the way proofs are as marked as missing changes, but the consequences do not.

leobago · 2024-10-23T20:28:12Z

design/proof-network.md

+[Mysticeti][6] algorithm seems to be particularly suited because it is highly
+performant.
+
+The consensus algorithm is only used to sequence transactions. Evaluation of the


I guess, maybe to match the right/current period? If a transaction is added too late will it miss the period?

leobago · 2024-10-23T20:32:01Z

design/proof-network.md

+-------
+
+The marketplace smart contract only slashes a storage provider when there is a
+combined BLS signature that represents > 2/3 stake.


So validators only make money when SPs fail proofs? Isn't this a motivation to attack SPs?

This is no different from our current implementation.

I also want to add fees for including storage proofs so that slashing isn't the only source of income for validators.

leobago · 2024-10-23T20:35:56Z

design/proof-network.md

+introduce epochs in which some validators are chosen from a larger set of
+potential validators, but that comes at the expense of added complexity.
+
+The latency is larger than in a [previous iteration][7] of this design, because


It is 1 proof per dataset right? Not per SP.

leobago · 2024-10-23T20:40:32Z

design/proof-network.md

+sequenced the transactions. That might be mitigated by using the fast path from
+the Mysticeti protocol for `StorageProof` transactions.
+
+[7]: https://github.com/codex-storage/codex-research/pull/194


I see many other drawbacks to this approach.

What motivates a node to become a validator? If everything goes well, they do a big part of the work for no reward.

How do we avoid SPs and validators colluding?

What motivates a node to become a validator? If everything goes well, they do a big part of the work for no reward.

I intend to add fees for including transactions.

How do we avoid SPs and validators colluding?

Through having multiple staked validators. The scheme works as long as >2/3 stake is not byzantine.

bkomuves · 2024-10-23T20:52:19Z

design/proof-network.md

@@ -0,0 +1,257 @@
+Storage proof network


I really would like a different name, as the network is most definitely not doing proofs. It's checking proofs.

This name is so misleading that already within the Codex team I've seen it to be referred as a "prover network", while it's the exact opposite of that.

I wouldn't mind renaming this, do you have a suggestion? Perhaps "proof checking network"?

I don't have a really good name in my mind. "Proof checking network" is certainly one possibility, "Verification network" or "Validator network" could be some more. From these 3 maybe I prefer the last one. But open to other suggestions

bkomuves · 2024-10-23T21:08:25Z

design/proof-network.md

+Notice that validators do not check the correctness of `StorageProof`
+transactions prior to including them in blocks. In this design sequencing and
+evaluation of transactions are separated. First, the validators reach consensus
+on a sequence of blocks of transactions. Then, each of the validators evaluates


What is the motivation for this design? (does it come from Mysticeti?)

It looks overly complicated on first sight:

we don't really need an ordering of events, proofs are independent from each other (ok maybe to select the first validator noticing a missing proof)

why try to include possible invalid proofs into a "block" when we can easily check if they are valid or not?

Also, what if the sequence ordering is such that a missing proof is first and the actual proof comes later?

What is the motivation for this design? (does it come from Mysticeti?)

It mimicks what SUI (and Mysticeti) do. It is part of the reason why they are able to scale to a very high number of transactions per second. It minimizes what is done while forming consensus, so that the rest of the work (evaluating transactions, updating state) can be done in parallel.

we don't really need an ordering of events, proofs are independent from each other (ok maybe to select the first validator noticing a missing proof)

That is correct, for proofs we can just about get away with it, as we argued in the first design.

However, for anything more consensus is necessary. If we don't want validators to race on-chain to mark a proof as missing, if we want to add payments for including a proof, if we want to punish validators for not doing their work properly, then we need consensus of some sort.

why try to include possible invalid proofs into a "block" when we can easily check if they are valid or not?

At some point, I'd like to introduce fees for including a transaction. This avoids the validators being spammed, and gives them an incentive to do their work. This is analogous to how you can submit an transaction that reverts to an Ethereum network, and even though it doesn't change the state, you still pay for the transaction fee.

Also, what if the sequence ordering is such that a missing proof is first and the actual proof comes later?

Then the proof was submitted too late. And we have consensus that this is so.

markspanbroek · 2024-10-29T09:31:27Z

I see a lot of potential danger going through this route. It seems to me that we are introducing a whole new colossal attack surface by implementing a blockchain. I see very few advantages to this approach in comparison with storing on an L2.

That is certainly a valid concern. But I don't think we can deliver on our promises without doing something in this direction. L2's are too expensive; the numbers don't work. L2's are also centralized; they do not provide protection against censorship.

Also, this design is not that different from the first design. Just because we're now calling things "transactions" and "blocks" doesn't mean that we introduce the complexity of a whole blockchain. We're just replacing the "light consensus" from the previous design with an actual consensus mechanism.

Second design for a proof network

c4078cd

markspanbroek marked this pull request as draft October 15, 2024 14:30

markspanbroek requested a review from fredericosteixeira October 15, 2024 14:30

bkomuves reviewed Oct 15, 2024

View reviewed changes

fredericosteixeira reviewed Oct 22, 2024

View reviewed changes

leobago reviewed Oct 23, 2024

View reviewed changes

bkomuves reviewed Oct 23, 2024

View reviewed changes


		### Faulty proofs ###

		The storage proofs that a storage provider submits can be faulty for a number of

		storage provider ---------------------------------------> validator


		The validator will include the transaction in the next block that it proposes to

Second design for a proof network #197

Are you sure you want to change the base?

Second design for a proof network #197

Conversation

markspanbroek commented Oct 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leobago left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markspanbroek Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markspanbroek Oct 29, 2024 • edited Loading

Choose a reason for hiding this comment

markspanbroek commented Oct 29, 2024

markspanbroek Oct 29, 2024 •

edited

Loading

markspanbroek Oct 29, 2024 •

edited

Loading