Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental Merkle Tree Implementation #72

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

stechu
Copy link
Contributor

@stechu stechu commented Aug 23, 2021

Description

This PR implements an incremental merkle tree data structure. This is an append only merkle tree that only maintains Log N internal nodes. An invariant this implementation tries to maintain is that all operations is Log N.

closes: #69


Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Targeted PR against correct branch (main)
  • Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
  • Wrote unit tests
  • Updated relevant documentation in the code
  • Added a relevant changelog entry to the Pending section in CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer

@Pratyush Pratyush requested a review from tsunrise August 24, 2021 01:16
Copy link
Member

@tsunrise tsunrise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall. Have some minor comments

Thanks for doing this! = )

Comment on lines +116 to +117
let hash_of_empty_node: P::InnerDigest = P::InnerDigest::default();
let hash_of_empty_leaf: P::LeafDigest = P::LeafDigest::default();
Copy link
Member

@tsunrise tsunrise Aug 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use the same hash function, do we want to have different values for empty inner digest and empty leaf digest, or it's fine in this context?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot think of any immediate harm here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it thanks! looks fine

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the intention is to be compatible with Zcash note commitment trees, then there's a sentinel value for empty leaves but there is no special case for empty internal nodes; they have the same value as would be expected from hashing their children. This doesn't require actually computing hashes for empty internal nodes, since the hash of a completely empty subtree only depends on its height (and can be computed in logarithmic time, then cached).

Copy link
Contributor Author

@stechu stechu Sep 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daira Thanks for your comments! Honestly I was just following the current arkwork's merkle tree implementation and didn't give too much thought. Now digging into it deeper, here is what I find:

For example, on JubJub, both P::InnerDigest::default() and P::LeafDigest::default() is {x:0, y: <one of 0's corresponding y on the curve> }. In the current implementation, the empty inner node a special value that is not equal to the "true" value where you actually compute from subtree. Like you said, it is easy to compute these empty inner nodes once and then use the cached value.

It is indeed not super "principled" but I am not too much concerned security wise due to the CRH (please let me know if i am wrong). If you think there is a concrete security issue here. We should fix both this PR and the arkworks merkle tree implementation.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is technically a small security issue in that if you use a sentinel value for internal nodes, you must argue that it was chosen such that it would be infeasible to find a preimage. (I.e. it is technically possible for there to be a back door where the sentinel is chosen to be the hash of some nonempty subtree.) But the main issue is just interoperability.

src/merkle_tree/mod.rs Show resolved Hide resolved
src/merkle_tree/incremental_merkle_tree.rs Outdated Show resolved Hide resolved
src/merkle_tree/tests/mod.rs Show resolved Hide resolved
@tsunrise
Copy link
Member

advertisement: You may also check if ark-sponge to see if that crate is also helpful for your application

Copy link
Member

@tsunrise tsunrise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM mod some nits

@stechu
Copy link
Contributor Author

stechu commented Sep 11, 2021

Since @daira's concern about the inner node empty value is not specific to this PR. Please let me know whether I should address her concern in this PR or not. @tsunrise @weikengchen @Pratyush

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incremental Merkle Tree
4 participants