Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement MPT part of generic extraction (Part 1) #385

Open
wants to merge 104 commits into
base: feat/tabular-queries
Choose a base branch
from

Conversation

silathdiir
Copy link
Contributor

@silathdiir silathdiir commented Oct 4, 2024

Related issue https://github.com/Lagrange-Labs/zkmr-tracking/issues/265

Design document MPT Extraction

Summary

  • Add the new ColumnGadget and MetadataGadget.
  • Update the leaf circuits of single and mapping variables, and add a new leaf circuit for mapping of mappings.
  • Update the integration test to make the values extraction work for the single and mapping variables (the rows digest check is failed, will update in a further PR).

@silathdiir silathdiir marked this pull request as draft October 4, 2024 14:31
mp2-v1/src/values_extraction/gadgets/column_gadget.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/gadgets/column_gadget.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/gadgets/column_gadget.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/gadgets/metadata_gadget.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/gadgets/column_gadget.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/leaves/mapping_var.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/mod.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/leaves/mapping_var.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/leaves/single_var.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/leaves/mapping_var.rs Outdated Show resolved Hide resolved
) -> KeccakMPTWires {
// location = keccak(inputs) + offset
let keccak_base = KeccakCircuit::<{ INPUT_PADDED_LEN }>::hash_to_bytes(b, &inputs);
let base = keccak_base.output.arr.pack(b, Endianness::Big);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed that hash_to_bytes decompose the output of Keccak in little-endian, while here you are packing it again in big-endian. Wouldn't be better to use hash_vector, which returns the output as u32 limbs which can be directly be employed to initialize a UInt256Target?

Copy link
Contributor Author

@silathdiir silathdiir Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the hash_vector, seems its result is little-endian packed u32s, but we need to initialize Uint256Target with big-endian u32s. So I keep the hash_to_bytes (then pack to big-endian) in the updating commit e5c21d1. Please correct me if I was wrong, thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really depends how EVM is interpreting the output of Keccak as a U256 to add offset. I think your current way of initializing the UInt256Target makes sense, but maybe a range-check on each byte of the hash_to_bytes output is needed, as hash_to_bytes should be safe to use without range-checking the output bytes only if those bytes are then fed as input to another Keccak, which is not the case here. Wdyt?

Copy link
Contributor Author

@silathdiir silathdiir Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I see, I added this byte range check in commit 3548cc5.

mp2-v1/src/values_extraction/api.rs Outdated Show resolved Hide resolved
mp2-v1/src/values_extraction/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@nicholas-mainardi nicholas-mainardi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite good for me, just a few simple changes left and should be good to go ;) Thanks for all the efforts

// Unwrap safe because input always fixed 32 bytes.
&InputData::Assigned(&Vector::from_vec(&inputs).unwrap()),
);
let location_bytes = inputs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to here most of the code looks like the same as assign_single. Isn't is possible to refactor it in a common method to avoid code duplication?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactor this code in commit aea77c6.

@@ -515,7 +619,7 @@ impl MappingSlot {
let inputs = mapping_key.into_iter().chain(padded_slot).collect_vec();
// Then compute the expected resulting hash for MPT key derivation.
let base = keccak256(&inputs).try_into().unwrap();
KeccakMPT::assign(pw, &wires.keccak_mpt, inputs, base, offset);
KeccakStructMPT::assign(pw, &wires.keccak_mpt, inputs, base, offset);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above: most of the code between assign_simple and assign_struct looks the same, wouldn't it be worthy to refactor the methods to avoid code duplication?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactor this code in commit aea77c6.

) -> KeccakMPTWires {
// location = keccak(inputs) + offset
let keccak_base = KeccakCircuit::<{ INPUT_PADDED_LEN }>::hash_to_bytes(b, &inputs);
let base = keccak_base.output.arr.pack(b, Endianness::Big);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really depends how EVM is interpreting the output of Keccak as a U256 to add offset. I think your current way of initializing the UInt256Target makes sense, but maybe a range-check on each byte of the hash_to_bytes output is needed, as hash_to_bytes should be safe to use without range-checking the output bytes only if those bytes are then fed as input to another Keccak, which is not the case here. Wdyt?

@silathdiir silathdiir changed the title feat: implement MPT part of generic extraction feat: implement MPT part of generic extraction (Part 1) Oct 23, 2024
Copy link
Contributor

@nicholas-mainardi nicholas-mainardi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Base automatically changed from feat/unproven-limit-offset-queries to feat/tabular-queries October 29, 2024 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants