Filter Processor Benchmarking

[winder]

Summary

Add new benchmarks Benchmarks for filters.

The results suggest that we're doing something extremely inefficient when setting up each additional filter.

Applying one filter to 1 or 25000 transactions has an almost negligible difference.
On the other hand there seems to be a fixed 8ms overhead for each filter.

The numbers in each report are as follows:

• number of transactions in the test block
• number of string equality filters
• number of regex filters
• number of numeric filters.

num txn	eq filters	regex filters	numeric filters
25000	1000	0	0
1	1000	0	0
25000	1	0	0
1	1	0	0
25000	0	1000	0
1	0	1000	0
25000	0	1	0
1	0	1	0
25000	0	0	1000
1	0	0	1000
25000	0	0	1
1	0	0	1

BenchmarkProcess2/txn_25000_equal_1000_regex_0_numeric_0
BenchmarkProcess2/txn_25000_equal_1000_regex_0_numeric_0-8         	       1	7791642627 ns/op
BenchmarkProcess2/txn_0_equal_1000_regex_0_numeric_0
BenchmarkProcess2/txn_0_equal_1000_regex_0_numeric_0-8             	       1	8057424687 ns/op
BenchmarkProcess2/txn_25000_equal_1_regex_0_numeric_0
BenchmarkProcess2/txn_25000_equal_1_regex_0_numeric_0-8            	     136	   8644760 ns/op
BenchmarkProcess2/txn_1_equal_1_regex_0_numeric_0
BenchmarkProcess2/txn_1_equal_1_regex_0_numeric_0-8                	     133	   8780231 ns/op
BenchmarkProcess2/txn_25000_equal_0_regex_1000_numeric_0
BenchmarkProcess2/txn_25000_equal_0_regex_1000_numeric_0-8         	       1	2271617844 ns/op
BenchmarkProcess2/txn_1_equal_0_regex_1000_numeric_0
BenchmarkProcess2/txn_1_equal_0_regex_1000_numeric_0-8             	       1	2330539073 ns/op
BenchmarkProcess2/txn_25000_equal_0_regex_1_numeric_0
BenchmarkProcess2/txn_25000_equal_0_regex_1_numeric_0-8            	     384	   2741222 ns/op
BenchmarkProcess2/txn_1_equal_0_regex_1_numeric_0
BenchmarkProcess2/txn_1_equal_0_regex_1_numeric_0-8                	     414	   2810811 ns/op
BenchmarkProcess2/txn_25000_equal_0_regex_0_numeric_1000
BenchmarkProcess2/txn_25000_equal_0_regex_0_numeric_1000-8         	       6	 206244626 ns/op
BenchmarkProcess2/txn_25000_equal_0_regex_0_numeric_1
BenchmarkProcess2/txn_25000_equal_0_regex_0_numeric_1-8            	    2234	    489659 ns/op
BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1000
BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1000-8             	       6	 197892161 ns/op
BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1
BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1-8                	    2068	    508157 ns/op

[codecov]

Codecov Report

Merging #146 (0e01345) into master (442791a) will increase coverage by 4.05%.
Report is 55 commits behind head on master.
The diff coverage is 80.81%.

@@            Coverage Diff             @@
##           master     #146      +/-   ##
==========================================
+ Coverage   67.66%   71.71%   +4.05%     
==========================================
  Files          32       37       +5     
  Lines        1976     2747     +771     
==========================================
+ Hits         1337     1970     +633     
- Misses        570      678     +108     
- Partials       69       99      +30     

Files Changed	Coverage Δ
conduit/data/block_export_data.go	100.00% <ø> (+92.30%)	⬆️
conduit/metrics/metrics.go	100.00% <ø> (ø)
conduit/pipeline/metadata.go	69.11% <ø> (ø)
conduit/plugins/config.go	100.00% <ø> (ø)
...duit/plugins/exporters/filewriter/file_exporter.go	81.63% <ø> (-1.06%)	⬇️
conduit/plugins/importers/algod/metrics.go	100.00% <ø> (ø)
...gins/processors/filterprocessor/fields/searcher.go	77.50% <ø> (ø)
...ins/processors/filterprocessor/filter_processor.go	83.82% <ø> (+3.54%)	⬆️
...plugins/processors/filterprocessor/gen/generate.go	34.28% <ø> (ø)
conduit/plugins/processors/noop/noop_processor.go	64.70% <ø> (+6.81%)	⬆️
... and 21 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[shiqizng]

how do you tell from the output that there's fixed 8ms overhead for each filter?

[tzaffi]

BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1000
BenchmarkProcess2/txn_1_equal_0_regex_0_numeric_1000-8 6 197892161 ns/op

It seems that way because when running 1000 filters, it takes 8 seconds. EG:

BenchmarkProcess2/txn_0_equal_1000_regex_0_numeric_0-8             	       1	8057424687 ns/op

[tzaffi]

197892161

But numeric filters seems to be faster.

[winder]

1 filter / 1 txn takes about 8ms
1 filter / 25000 txn takes about 8ms
1000 filters / 1 txn takes about 8 seconds
1000 filters / 25000 txn takes about 8 seconds

I think it follows that the number of txn seems to be nicely optimized, but there seems to be some per-filter overhead somewhere.

[winder]

197892161

But numeric filters seems to be faster.

Good point, I was definitely looking at the first few results more than the other filter types.