Run Lodestar without persisting archive states at all or minimally

[philknows]

Problem description

For some testnet or dev networks, we do not need to archive any states and there should be a mode that will allow us to enable it. Instead of continually upgrading SSDs on these no-value networks or manually dumping the beacon data periodically, we should just enable this feature.

Solution description

Similar to sigp/lighthouse#3211 on Lighthouse, status-im/nimbus-eth2#4419 on Nimbus and Consensys/teku#6576 on Teku, we should discuss different strategies for how we persist archive states. For Holesky, this will be useful to prevent storing unnecessary information as the network will start at 2x mainnet size.

Additional context

No response

[wemeetagain]

For now this should be possible using --chain.archiveStateEpochFrequency Infinity or --chain.archiveStateEpochFrequency 9999999999. Not great UX but should get the job done for our Holesky nodes.

[philknows]

Will leave this issue open for a better UX solution in the future. But for now, the above flags will do the job! Thanks.

[philknows]

What does better UX look like for this feature? Should we implement something similar to other client teams or just add the notable comment by @wemeetagain to our docs?

[philknows]

This may change once we have state diffs and in V2, this may not be the default. Related to #7005