perf: optimize iteration on nested cache context

[yihuang]

Description

Closes: #10310
Solution:

• only call skipUntilExistsOrInvalid at construction and end of Next, cache the valid status
• use tidwall btree directly instead of memdb.

run benchmark: go test -run=^$ -bench=. -benchmem ./store/cachekv/benchmark_test.go -count=10.
benchstat compare result:

$ benchstat before.txt after.txt
name                   old time/op    new time/op    delta
DeepContextStack1-12     11.2µs ±12%     5.7µs ± 1%   -48.57%  (p=0.000 n=10+8)
DeepContextStack3-12     32.2µs ±16%     7.1µs ± 2%   -77.99%  (p=0.000 n=9+9)
DeepContextStack10-12     244ms ±17%       0ms ±10%   -99.99%  (p=0.000 n=10+10)
DeepContextStack13-12     16.5s ± 1%      0.0s ± 1%  -100.00%  (p=0.000 n=10+10)

name                   old alloc/op   new alloc/op   delta
DeepContextStack1-12     3.94kB ± 0%    3.03kB ± 0%   -23.12%  (p=0.000 n=10+10)
DeepContextStack3-12     6.33kB ± 0%    3.59kB ± 0%   -43.24%  (p=0.000 n=10+10)
DeepContextStack10-12    15.1kB ± 1%     5.6kB ± 0%   -63.27%  (p=0.000 n=10+10)
DeepContextStack13-12    20.8kB ± 0%     6.4kB ± 0%   -69.20%  (p=0.000 n=10+10)

name                   old allocs/op  new allocs/op  delta
DeepContextStack1-12       50.0 ± 0%      42.0 ± 0%   -16.00%  (p=0.000 n=10+10)
DeepContextStack3-12       74.0 ± 0%      50.0 ± 0%   -32.43%  (p=0.000 n=10+10)
DeepContextStack10-12       172 ± 2%        78 ± 0%   -54.62%  (p=0.000 n=10+10)
DeepContextStack13-12       272 ± 0%        90 ± 0%   -66.91%  (p=0.000 n=10+10)

---

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

• included the correct type prefix in the PR title
• added ! to the type prefix if API or client breaking change
• targeted the correct branch (see PR Targeting)
• provided a link to the relevant issue or specification
• followed the guidelines for building modules
• included the necessary unit and integration tests
• added a changelog entry to CHANGELOG.md
• included comments for documenting Go code
• updated the relevant documentation or specification
• reviewed "Files changed" and left comments if necessary
• confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

• confirmed the correct type prefix in the PR title
• confirmed ! in the type prefix if API or client breaking change
• confirmed all author checklist items have been addressed
• reviewed state machine logic
• reviewed API design and naming
• reviewed documentation is accurate
• reviewed tests and test coverage
• manually tested (if applicable)

[yihuang]

The test(02) failure seems like a non-deterministic one unrelated to this PR, I see it fail on some main branch commit as well: https://github.com/cosmos/cosmos-sdk/actions/runs/3467563489/jobs/5792555203

[yihuang]

After this optimization, the cpu profile points to the runtime stuff, I believe it's the go-routine used in memdb's iteration, why do we use a go-routine to iterate a in-memory data structure?
And the cache context don't need to be thread safe, so I'd suggest we use the btree directly here.

[yihuang]

Anther round of benchmark:

original

$ go test -cpuprofile profile.out -benchtime=10s -bench=. ./store/cachekv/benchmark_test.go -run ^$
goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkDeepContextStack1-12     	 1202187	     10004 ns/op
BenchmarkDeepContextStack3-12     	  412293	     29168 ns/op
BenchmarkDeepContextStack10-12    	      61	 184568144 ns/op
BenchmarkDeepContextStack13-12    	       1	11722949259 ns/op
PASS
ok  	command-line-arguments	59.099s

Cache valid status

$ go test -cpuprofile profile.out -benchtime=10s -bench=. ./store/cachekv/benchmark_test.go -run ^$
goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkDeepContextStack1-12     	 1275110	      9359 ns/op
BenchmarkDeepContextStack3-12     	  640540	     18609 ns/op
BenchmarkDeepContextStack10-12    	  237964	     50065 ns/op
BenchmarkDeepContextStack13-12    	  182748	     64797 ns/op
PASS
ok  	command-line-arguments	59.894s

Use btree directly

$ go test -cpuprofile profile.out -benchtime=10s -bench=. ./store/cachekv/benchmark_test.go -run ^$
goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkDeepContextStack1-12     	 2163434	      5485 ns/op
BenchmarkDeepContextStack3-12     	 1741351	      6830 ns/op
BenchmarkDeepContextStack10-12    	  880039	     13731 ns/op
BenchmarkDeepContextStack13-12    	  698671	     17385 ns/op
PASS
ok  	command-line-arguments	61.765s

[tac0turtle]

There is an issue to introduce and rewrite the memdb in favor of tidwall and to stop using the go go routine. This is for cosmos-db. I think we should put this change there and update the repo to cosmos-db like Jacob has done in a pr that is now unblocked. Would this work?

Tidwall has a btreehashmap that is more performant that we may be able to take advantage of as well.

[yihuang]

There is an issue to introduce and rewrite the memdb in favor of tidwall and to stop using the go go routine. This is for cosmos-db. I think we should put this change there and update the repo to cosmos-db like Jacob has done in a pr that is now unblocked. Would this work?

Tidwall has a btreehashmap that is more performant that we may be able to take advantage of as well.

I see, I guess we can do the first part first, caching the valid status.

[yihuang]

Actually I think it's maybe ok to have a separate implementation in cachekv, considering the importance of cachekv, and it don't need to be thread-safe like the MemDB, the MemDB is mainly used for mocking db in unit tests.

[codecov]

Codecov Report

Merging #13881 (4a27e1c) into main (2739f83) will increase coverage by 0.11%.
The diff coverage is 65.51%.

❗ Current head 4a27e1c differs from pull request most recent head b2eeaf8. Consider uploading reports for the commit b2eeaf8 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #13881      +/-   ##
==========================================
+ Coverage   56.25%   56.37%   +0.11%     
==========================================
  Files         667      639      -28     
  Lines       56576    54995    -1581     
==========================================
- Hits        31829    31005     -824     
+ Misses      22165    21466     -699     
+ Partials     2582     2524      -58     

Impacted Files	Coverage Δ
store/cachekv/internal/mergeiterator.go	0.00% <0.00%> (ø)
store/cachekv/internal/memiterator.go	70.93% <70.93%> (ø)
store/cachekv/internal/btree.go	90.00% <90.00%> (ø)
store/cachekv/store.go	89.49% <100.00%> (+1.46%)	⬆️
x/distribution/simulation/operations.go	80.64% <0.00%> (-9.68%)	⬇️
x/gov/abci.go	90.29% <0.00%> (-6.86%)	⬇️
x/group/internal/math/dec.go	47.14% <0.00%> (-1.39%)	⬇️
x/group/module/module.go	48.05% <0.00%> (-0.10%)	⬇️
server/mock/tx.go	53.33% <0.00%> (ø)
... and 37 more

[yihuang]

it don't need to be thread-safe like the MemDB

take back this, it do need to be thread-safe, I've enabled the lock option.

rerun benchmark after recent changes:

$ go test -cpuprofile profile.out -benchtime=10s -bench=. ./store/cachekv/benchmark_test.go -run ^$
goos: darwin
goarch: amd64
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkDeepContextStack1-12     	 2169144	      5495 ns/op
BenchmarkDeepContextStack3-12     	 1677499	      7228 ns/op
BenchmarkDeepContextStack10-12    	  886461	     13547 ns/op
BenchmarkDeepContextStack13-12    	  693322	     17535 ns/op
PASS
ok  	command-line-arguments	62.829s

[tac0turtle]

utACK. thank you for this PR.

[tac0turtle]

@Mergifyio backport release/v0.46.x

[tac0turtle]

@Mergifyio backport release/v0.47.x

[mergify]

backport release/v0.46.x

✅ Backports have been created

• #14337 perf: optimize iteration on nested cache context (backport #13881) has been created for branch release/v0.46.x

[mergify]

backport release/v0.47.x

✅ Backports have been created

• #14338 perf: optimize iteration on nested cache context (backport #13881) has been created for branch release/v0.47.x

[tac0turtle]

@Mergifyio backport release/v0.45.x

[mergify]

backport release/v0.45.x

✅ Backports have been created

• #14341 perf: optimize iteration on nested cache context (backport #13881) has been created for branch release/v0.45.x