Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore!: refactor spend struct #1930

Closed
wants to merge 5 commits into from
Closed

Conversation

maqi
Copy link
Member

@maqi maqi commented Jul 1, 2024

BREAKING CHANGE

related to the issue #1931

Description

Please provide a brief description of the changes made in this pull request. Highlight the purpose of the changes and the problem they address.

};
let mut descendants = BTreeMap::new();
for output in self.outputs.iter() {
// TODO: use proper OutputPurpose

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment

let mut descendants = BTreeMap::new();
for output in self.outputs.iter() {
// TODO: use proper OutputPurpose

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
impl OutputPurpose {
pub fn hash(&self) -> Hash {
match self {
Self::None => Hash::default(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hash::default doesn't make a lot of sense. Perhaps it's better to change the return type?

@maqi maqi force-pushed the refactor_spend_struct branch 2 times, most recently from 19a1a64 to 5a9d548 Compare July 5, 2024 13:21
dag.insert(spend.address(), spend.clone());
}
assert!(dag.record_faults(&genesis).is_ok());

// TODO: re-enable the assertion once figured out the root reason of the failure.

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note test

Suspicious comment
@maqi maqi force-pushed the refactor_spend_struct branch 13 times, most recently from 0eb0472 to 869a341 Compare July 8, 2024 09:13
@@ -17,6 +17,8 @@

const N_OUTPUTS: u64 = 100;

// TODO: re-enable if figured out whether it is still required and how to pass in correct keys

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note

Suspicious comment
@maqi maqi force-pushed the refactor_spend_struct branch 2 times, most recently from 9a04270 to 7d06255 Compare July 8, 2024 10:04
@maqi maqi force-pushed the refactor_spend_struct branch 3 times, most recently from 245b095 to cd4d8ee Compare July 8, 2024 11:18
@maqi maqi force-pushed the refactor_spend_struct branch from 99ded90 to 0b3ce7c Compare July 9, 2024 15:34
@maqi maqi force-pushed the refactor_spend_struct branch from 0b3ce7c to 7167cf6 Compare July 9, 2024 15:50
@@ -508,11 +507,17 @@
info!("TestWallet {id} verifying status of spend: {spend:?} : {status:?}");
match status {
SpendStatus::Utxo => {
available_cash_notes
// TODO: with the new spend struct requiring `middle payment`

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note test

Suspicious comment
@maqi maqi force-pushed the refactor_spend_struct branch 2 times, most recently from 2c2ecfd to f708838 Compare July 10, 2024 08:32
@maqi maqi force-pushed the refactor_spend_struct branch from f708838 to 33101f5 Compare July 10, 2024 08:44
}
SpendStatus::Spent => {
let addr = SpendAddress::from_unique_pubkey(spend);
let _spend = client.get_spend_from_network(addr).await?;
}
SpendStatus::Poisoned => {
// TODO:

Check notice

Code scanning / devskim

A "TODO" or similar was left in source code, possibly indicating incomplete functionality Note test

Suspicious comment
Comment on lines -390 to 389
if parent_tx == *sn_transfers::GENESIS_CASHNOTE_PARENT_TX
&& spends
.iter()
.all(|s| s.spend.unique_pubkey == *sn_transfers::GENESIS_SPEND_UNIQUE_KEY)
debug!("Depth {depth} - Got {spends_len} spends for parent: {addrs_to_verify:?}");
trace!("Spends for {addrs_to_verify:?} - {spends:?}");

// check if we reached the genesis spend
known_ancestors.extend(addrs_to_verify.clone());
if spends
.iter()
.all(|s| s.spend.unique_pubkey == *sn_transfers::GENESIS_SPEND_UNIQUE_KEY)
&& spends.len() == 1
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we use to check that the Genesis we have was the original Genesis meaning:

  • the unique pk is GENESIS
  • and the TX/descendants it was spent to is correct too

By removing the second check, we will miss out on rogue Genesises: Genesis not spent to the determined descendant!

Comment on lines 462 to 466
let actual_ancestor: Vec<_> = multiple_ancestors
.iter()
.filter(|(s, _)| s.spend.spent_tx.hash() == spend.spend.parent_tx.hash())
.filter(|(s, _)| s.address() == spend.address())
.map(|(s, _)| s.clone())
.collect();
Copy link
Member

@grumbach grumbach Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check should check that the s is indeed the parent of spend by checking that:

s.spend.descendant.contains(spend.address())

Comment on lines +747 to +755
let mut parents_collector = BTreeSet::new();
for ancestor in ancestor_spends {
let mut collector = BTreeSet::new();
let _ = collector.insert(ancestor);
let _ = parents_collector.insert(collector);
}

// verify the tx
if let Err(e) = spend
.spend
.parent_tx
.verify_against_inputs_spent(&ancestor_spends)
{
if let Err(e) = spend.verify_parent_spends(parents_collector.iter()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why BTreeSet<BTreeSet<_>>?

Comment on lines -129 to +132
return Err(NetworkError::NotEnoughPeers {
found: peers.len(),
required: CLOSE_GROUP_SIZE,
});
// return Err(NetworkError::NotEnoughPeers {
// found: peers.len(),
// required: CLOSE_GROUP_SIZE,
// });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commenting this is dangerous, we might forget it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can keep this error. The spend simulation is not mandatory for the CI to go through. Even if it fails, PR will get merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it not be mandatory? Or it's not yet stable?

Comment on lines -181 to -215
// check Txs and parent spends are valid
trace!("Validating parent spends");
for tx in parent_txs {
let tx_inputs_keys: Vec<_> = tx.inputs.iter().map(|i| i.unique_pubkey()).collect();

// get the missing inputs spends from the network
let mut tasks = JoinSet::new();
for input_key in tx_inputs_keys {
if parent_spends.iter().any(|s| s.unique_pubkey() == input_key) {
continue;
}
let self_clone = self.clone();
let addr = SpendAddress::from_unique_pubkey(input_key);
let _ = tasks.spawn(async move { self_clone.get_spend(addr).await });
}
while let Some(result) = tasks.join_next().await {
let signed_spend = result
.map_err(|e| NetworkError::FailedToGetSpend(format!("{e}")))?
.map_err(|e| NetworkError::InvalidTransfer(format!("{e}")))?;
let _ = parent_spends.insert(signed_spend.clone());
}

// verify the Tx against the inputs spends
let input_spends: BTreeSet<_> = parent_spends
.iter()
.filter(|s| s.spent_tx_hash() == tx.hash())
.cloned()
.collect();
tx.verify_against_inputs_spent(&input_spends).map_err(|e| {
NetworkError::InvalidTransfer(format!(
"Payment parent Tx {:?} invalid: {e}",
tx.hash()
))
})?;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we've removed all sort of transfer verification.
Letting people accepting transfers of money that isn't valid!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this passed CI checks, is there a test we can add for this so it's not missed in future?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should add a test for this!

Comment on lines +429 to +430
// With the new spend struct, the involved parent_spends of the cashnote are not polluted
// TODO: re-enable the following check block once confirmed the assumption retains.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this error on my PR too, so probably not linked to your work @maqi
@RolandSherwin do you know about this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was my mistake on my PR, so this assumption is probably wrong!

Comment on lines +43 to 45
pub fn token(&self) -> NanoTokens {
self.spend.amount()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would better be renamed to amount or value.

@@ -72,70 +60,14 @@ impl SignedSpend {
/// Verify a SignedSpend
///
/// Checks that
/// - the spend was indeed spent for the given Tx
/// - it was signed by the DerivedSecretKey that owns the CashNote for this Spend
/// - the signature is valid
/// - its value didn't change between the two transactions it is involved in (creation and spending)
Copy link
Member

@grumbach grumbach Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// - its value didn't change between the two transactions it is involved in (creation and spending)

No more tx checks here

Comment on lines +103 to 109
if let Some(parent) = p.first() {
if let Some(amount) = parent.spend.get_output_amount(unique_key) {
total_inputs += amount.as_nano();
} else {
return Err(TransferError::InvalidParentSpend(format!(
"Parent spend was spent in another transaction. Expected: {tx_our_cash_note_was_created_in:?} Got: {tx_parent_was_spent_in:?}"
"Parent spend {:?} doesn't contain self spend {unique_key:?} as one of its output", parent.unique_pubkey()
)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we're assuming that the first is the correct parent, BUT it might be the second and we error out before checking.

Comment on lines -205 to -210
// Here we check that the CashNote we're trying to spend was created in a valid tx
if let Err(e) = self
.spend
.parent_tx
.verify_against_inputs_spent(actual_parent_spends.iter())
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add back these checks:

  • we are an output of parent (same pk same value)
  • parent is an input of us (same pk same value)

Comment on lines 99 to 100
error!("While verifying parents of {unique_key}, found a parent double spend, but it contained more than one unique_pubkey. This is invalid. Erroring out.");
return Err(TransferError::InvalidParentSpend("Invalid parent double spend. More than one unique_pubkey in the parent double spend.".to_string()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My opinion is that if we're to reject double spent parents here, we might as well have an API that suggests this.
Instead of accepting a BTreeSet of BTreeSet of parent we should accept only a BTreeSet of all parents.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think?

Comment on lines 151 to +160
pub struct Spend {
/// UniquePubkey of input CashNote that this SignedSpend is proving to be spent.
pub unique_pubkey: UniquePubkey,
/// The transaction that the input CashNote is being spent in (where it is an input)
#[debug(skip)]
pub spent_tx: Transaction,
/// Reason why this CashNote was spent.
#[debug(skip)]
pub reason: SpendReason,
/// The amount of the input CashNote.
#[debug(skip)]
pub amount: NanoTokens,
/// The transaction that the input CashNote was created in (where it is an output)
#[debug(skip)]
pub parent_tx: Transaction,
/// Data to claim the Network Royalties (if any) from the Spend's descendants (outputs in spent_tx)
#[debug(skip)]
pub network_royalties: Vec<DerivationIndex>,
/// Inputs (parent spends) of this spend
pub ancestors: BTreeSet<UniquePubkey>,
/// Outputs of this spend
pub descendants: BTreeMap<UniquePubkey, (NanoTokens, OutputPurpose)>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels very good!

Comment on lines 138 to 141
spend.spend.unique_pubkey == *GENESIS_SPEND_UNIQUE_KEY
&& GENESIS_SPEND_UNIQUE_KEY.verify(&spend.derived_key_sig, bytes)
&& is_genesis_parent_tx(&spend.spend.parent_tx)
&& spend.spend.amount == NanoTokens::from(GENESIS_CASHNOTE_AMOUNT)
&& spend.spend.amount() == NanoTokens::from(GENESIS_CASHNOTE_AMOUNT)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check that the GENESIS.descendant and ancestor is correct too!

Comment on lines 499 to 509
let offline_transfer = OfflineTransfer::new(
available_cash_notes,
recipients,
self.address(),
spend_reason,
Some((
self.key.main_pubkey(),
derivation_index,
self.key.derive_key(&derivation_index),
)),
)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requiring the secret key self.key for the offline_transfer part of the code is a drawback of the middle spend.
We can leave it for now as this goes out of scope of this PR. But I think we should keep that in mind as it makes offline tx preparation impossible without the key!

Copy link
Member

@grumbach grumbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Impressive work @maqi!
Couple of security notes and comments.

Comment on lines 210 to -212
pub fn build(self) -> Result<Vec<(CashNote, NanoTokens)>> {
// Verify the tx, along with signed spends.
// Note that we do this just once for entire tx, not once per output CashNote.
self.spent_tx
.verify_against_inputs_spent(self.signed_spends.iter())?;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need some kind of verification here or else there is no difference between build and build_without_verifying.

@grumbach grumbach mentioned this pull request Jul 19, 2024
4 tasks
@grumbach
Copy link
Member

Changes + fixes included in #1989

@grumbach grumbach closed this Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants