decoy selection: spend like coinbase-ish outputs

[jeffro256]

Implements solution recommended here (monero-project/research-lab#109) in an efficient manner.

To retrieve the ENTIRE post-RCT coinbase output distribution from a running daemon, execute: curl http://127.0.0.1:18081/json_rpc -d '{"jsonrpc":"2.0","id":"0","method":"get_output_distribution","params":{"amounts":[],"from_height":0,"to_height":0, "binary": false,"get_rct_coinbase":true}}' -H 'Content-Type: application/json'.

[jeffro256]

This PR is now ready to review. To test, create a bunch of transactions and make sure that each input ring contains only either coinbase enotes or non-coinbase enotes.

Here's a general overview of the design of the PR. To start off, the biggest assumption made for this PR is that for each RCT block (a block with major version >= 4), inside each block, all RCT coinbase enotes have their amount index equal to 0 for retrieval, and all RCT coinbase enote indexes are less than the non-coinbase enote indexes for a given block. This means that if a wallet knows the RCT coinbase offset distribution in addition to the total RCT offset distribution, the wallet can determine whether or not any global enote index is a RCT coinbase enote just from its index without any extra information.

On the daemon side, I added in a class called CoinbaseOutputDistributionCache, which caches the "RCT coinbase enote distribution" (how many RCT coinbase enotes there are per block). I also made a custom compression algorithm (in src/common/compress_uint) that is able to crush down this data very efficiently.

As for RPC, the get_output_distribution{.bin} endpoint response was modified to add an extra field called rct_coinbase_distribution. This field can be enabled by setting the get_rct_coinbase field in the request to true. If binary, and compress, and compress_one_span are also set to true, the coinbase_distribution field is compressed by using the aforementioned compression algorithm to get a lot of savings in bandwidth.

Finally, on the wallet side, the wallet does decoy selection by retrieving the total enote offsets rct_offsets and the coinbase enote offsets rct_coinbase_offsets. We can then calculate the non-coinbase enote offsets rct_non_coinbase_offsets by performing rct_offsets - rct_coinbase_offsets element-wise. Then, for each incoming transfers, we determine if the transfer is a coinbase enote or not, and point the gamma picker to rct_coinbase_offsets or rct_non_coinbase_offsets, respectively. Then gamma selection is done as normal, and we take the index returned by the gamma picker and readjust it back to a global enote index based on which distribution the gamma picker was using.

[plowsof]

Ive had a quick test of this. spent a coinbase output , and only coinbase outputs are used as decoys in this tx https://stagenet.xmrchain.net/tx/e075bf7e3ec463e05a9852cc5f7e6ac9e1b3298965533c04747a557b17abec57/1 , and the reverse for spending a non-coinbase output https://stagenet.xmrchain.net/tx/8f2f23db442e93e866adac7788645bb330e56a439ee10f5ec048c38d3222a00c/1

a sweep_all where one of the inputs was a non coinbase output https://stagenet.xmrchain.net/tx/851ef2ad3a13c0c125779cf107dc38155f56978f62b3ffa81b78230d1a69dbe9/1

[jeffro256]

I have rewrote the PR to use BlockchainLMDB acceleration in #8896. This speeds up fetching significantly. See this comment for technical description.

[jeffro256]

FCMP++ makes this obsolete