add flag to bake non bootstrap validator stakes into genesis

[gregcusack]

Problem

There is currently no efficient way of to bake validator stakes into genesis without requiring the user to serialize and deserialize validator accounts into and out of a primordial-accounts-file file.

Summary of Changes

add flag to file for baking validator (non bootstrap) identity, vote, and stake accounts into genesis
add flag for baking validator (non bootstrap) stake and identity lamports into genesis.

validator-accounts-file format:

---
- balance_lamports: <balance-lamports0>
  stake_lamports: <stake-lamports0>
  identity_account: <identity-pubkey0>
  vote_account: <vote-pubkey0>
  stake_account: <stake-pubkey0>
- balance_lamports: <balance-lamports1>
  stake_lamports: <stake-lamports1>
  identity_account: <identity-pubkey1>
  vote_account: <vote-pubkey1>
  stake_account: <stake-pubkey1>
...
- balance_lamports: <balance-lamportsN>
  stake_lamports: <stake-lamportsN>
  identity_account: <identity-pubkeyN>
  vote_account: <vote-pubkeyN>
  stake_account: <stake-pubkeyN>

[CriesofCarrots]

I'm wondering if we want a more flexible interface here. I know you're reusing a lot of the --bootstrap-validator logic, and it is nice and clean. And I know it also matched the specific use-cases of /net and validator-lab. But the upshot is that this only enables two stake-levels of validators (bootstrap and internal).

What if internal_validator took something like this as input instead:

Arg::with_name("internal_validator")
    .number_of_values(5)
    .value_name("IDENTITY_PUBKEY VOTE_PUBKEY STAKE_PUBKEY IDENTITY_BALANCE_SOL STAKE_BALANCE_SOL")`

Oh, just saw the help text; it looks like you were originally headed that direction? https://github.com/anza-xyz/agave/pull/2704/files#diff-4ba9fb9faa2132f5a4b9dde7b311023011c423c34c21bebf6c8f6f638fae0bf3R321

[gregcusack]

I'm wondering if we want a more flexible interface here. I know you're reusing a lot of the --bootstrap-validator logic, and it is nice and clean. And I know it also matched the specific use-cases of /net and validator-lab. But the upshot is that this only enables two stake-levels of validators (bootstrap and internal).

What if internal_validator took something like this as input instead:

Arg::with_name("internal_validator")
    .number_of_values(5)
    .value_name("IDENTITY_PUBKEY VOTE_PUBKEY STAKE_PUBKEY IDENTITY_BALANCE_SOL STAKE_BALANCE_SOL")`

Oh, just saw the help text; it looks like you were originally headed that direction? https://github.com/anza-xyz/agave/pull/2704/files#diff-4ba9fb9faa2132f5a4b9dde7b311023011c423c34c21bebf6c8f6f638fae0bf3R321

omg i forgot to update the comment lol. and yesss I was. I was thinking it made it more confusing to have pubkeys and lamports in the same argument. But if we want this level of flexibility (sounds like we do), then it makes sense to add lamports to the end of that arg

[CriesofCarrots]

But if we want this level of flexibility (sounds like we do), then it makes sense to add lamports to the end of that arg

Alternately, we could read a yaml file that defines a list of validators with their keys and different funding levels. But I wasn't sure if this would be confusing alongside the primordial-accounts file, and/or harder to use than a long, multi-type CLI arg... What do you think?

[gregcusack]

But if we want this level of flexibility (sounds like we do), then it makes sense to add lamports to the end of that arg

Alternately, we could read a yaml file that defines a list of validators with their keys and different funding levels. But I wasn't sure if this would be confusing alongside the primordial-accounts file, and/or harder to use than a long, multi-type CLI arg... What do you think?

I do think that a yaml file is a little easier to manage. But, ya not sure it makes sense along side of primordial accounts file. I assume we do NOT want to change the functionality of the primordial-accounts file, right? Or would that be an option?

[CriesofCarrots]

I assume we do NOT want to change the functionality of the primordial-accounts file, right? Or would that be an option?

I'd say it's an option if we can do it in a backward compatible way; ie. still parse files written with the existing format. There are various ways I can think to do that. One way might be to replace Base64Account in the deserialization with an enum with Base64Account and some new validator-info type.

But I'm also not totally opposed to a 2nd yaml for validator defs. I think we could make it distinct enough and clear enough to work... This crate probably needs a readme to explain stuff like the primodial-accounts file syntax anyway.

[gregcusack]

I assume we do NOT want to change the functionality of the primordial-accounts file, right? Or would that be an option?

I'd say it's an option if we can do it in a backward compatible way; ie. still parse files written with the existing format. There are various ways I can think to do that. One way might be to replace Base64Account in the deserialization with an enum with Base64Account and some new validator-info type.

But I'm also not totally opposed to a 2nd yaml for validator defs. I think we could make it distinct enough and clear enough to work... This crate probably needs a readme to explain stuff like the primodial-accounts file syntax anyway.

i think the backwards compatibility makes the most sense since it's somewhat similar logic (Adding accounts into genesis). I'll take a stab at it.