Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[no-release-notes] bump for get field stringer changes #8525

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Nov 4, 2024

dolt-side microbenchmarks for before/after:

goos: darwin
goarch: arm64
pkg: github.com/dolthub/dolt/go/performance/microsysbench
                         │  bench1.txt  │              bench2.txt              │
                         │    sec/op    │    sec/op     vs base                │
OltpPointSelect-12         37.27µ ± ∞ ¹   34.56µ ± ∞ ¹       ~ (p=1.000 n=1) ²
OltpJoinScan-12            404.9µ ± ∞ ¹   401.0µ ± ∞ ¹       ~ (p=1.000 n=1) ²
ProjectionAggregation-12   17.07m ± ∞ ¹   18.04m ± ∞ ¹       ~ (p=1.000 n=1) ²
SelectRandomPoints-12      118.2µ ± ∞ ¹   114.5µ ± ∞ ¹       ~ (p=1.000 n=1) ²
SelectRandomRanges-12      157.6µ ± ∞ ¹   157.2µ ± ∞ ¹       ~ (p=1.000 n=1) ²
geomean                    343.8µ         339.3µ        -1.29%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                         │  bench1.txt   │              bench2.txt               │
                         │     B/op      │     B/op       vs base                │
OltpPointSelect-12         19.89Ki ± ∞ ¹   19.47Ki ± ∞ ¹       ~ (p=1.000 n=1) ²
OltpJoinScan-12            164.6Ki ± ∞ ¹   162.9Ki ± ∞ ¹       ~ (p=1.000 n=1) ²
ProjectionAggregation-12   2.861Mi ± ∞ ¹   3.034Mi ± ∞ ¹       ~ (p=1.000 n=1) ²
SelectRandomPoints-12      68.59Ki ± ∞ ¹   67.82Ki ± ∞ ¹       ~ (p=1.000 n=1) ²
SelectRandomRanges-12      99.84Ki ± ∞ ¹   99.50Ki ± ∞ ¹       ~ (p=1.000 n=1) ²
geomean                    145.7Ki         146.1Ki        +0.23%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

                         │  bench1.txt  │              bench2.txt              │
                         │  allocs/op   │  allocs/op    vs base                │
OltpPointSelect-12          443.0 ± ∞ ¹    416.0 ± ∞ ¹       ~ (p=1.000 n=1) ²
OltpJoinScan-12            6.019k ± ∞ ¹   5.903k ± ∞ ¹       ~ (p=1.000 n=1) ²
ProjectionAggregation-12   68.66k ± ∞ ¹   72.91k ± ∞ ¹       ~ (p=1.000 n=1) ²
SelectRandomPoints-12      1.256k ± ∞ ¹   1.207k ± ∞ ¹       ~ (p=1.000 n=1) ²
SelectRandomRanges-12      2.375k ± ∞ ¹   2.353k ± ∞ ¹       ~ (p=1.000 n=1) ²
geomean                    3.527k         3.477k        -1.42%
¹ need >= 6 samples for confidence interval at level 0.95
² need >= 4 samples to detect a difference at alpha level 0.05

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Nov 4, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
0148336 ok 5937457
version total_tests
0148336 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 58.92 58.92 0.0
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt c2bb78c 40.7 dolt 0148336 41.11 1.01

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.62 0.61 -1.61
groupby_scan 16.71 16.41 -1.8
index_join 2.3 2.26 -1.74
index_join_scan 1.93 1.82 -5.7
index_scan 55.82 54.83 -1.77
oltp_point_select 0.27 0.26 -3.7
oltp_read_only 5.57 5.28 -5.21
select_random_points 0.67 0.64 -4.48
select_random_ranges 0.65 0.63 -3.08
table_scan 55.82 54.83 -1.77
types_table_scan 147.61 144.97 -1.79
write_tests from_latency to_latency percent_change
oltp_delete_insert 5.88 5.77 -1.87
oltp_insert 2.91 2.86 -1.72
oltp_read_write 11.45 11.24 -1.83
oltp_update_index 2.97 2.91 -2.02
oltp_update_non_index 2.91 2.86 -1.72
oltp_write_only 5.88 5.88 0.0
types_delete_insert 6.32 6.21 -1.74

@max-hoffman
Copy link
Contributor Author

#benchmark
second to confirm numbers

Copy link

github-actions bot commented Nov 4, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 58.92 57.87 -1.78
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt c2bb78c 40.31 dolt 0148336 41.12 2.01

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.62 0.62 0.0
groupby_scan 16.71 17.01 1.8
index_join 2.3 2.26 -1.74
index_join_scan 1.93 1.86 -3.63
index_scan 55.82 55.82 0.0
oltp_point_select 0.27 0.27 0.0
oltp_read_only 5.47 5.47 0.0
select_random_points 0.67 0.67 0.0
select_random_ranges 0.65 0.65 0.0
table_scan 55.82 56.84 1.83
types_table_scan 147.61 147.61 0.0
write_tests from_latency to_latency percent_change
oltp_delete_insert 5.88 5.77 -1.87
oltp_insert 2.86 2.86 0.0
oltp_read_write 11.45 11.45 0.0
oltp_update_index 2.91 2.97 2.06
oltp_update_non_index 2.86 2.86 0.0
oltp_write_only 5.88 5.99 1.87
types_delete_insert 6.32 6.21 -1.74

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Nov 4, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 59.99 59.99 0.0
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt 134372c 40.56 dolt 0148336 41.1 1.33

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.62 0.62 0.0
groupby_scan 16.71 16.71 0.0
index_join 2.35 2.26 -3.83
index_join_scan 1.89 1.86 -1.59
index_scan 55.82 55.82 0.0
oltp_point_select 0.27 0.27 0.0
oltp_read_only 5.57 5.47 -1.8
select_random_points 0.67 0.67 0.0
select_random_ranges 0.65 0.65 0.0
table_scan 55.82 55.82 0.0
types_table_scan 147.61 147.61 0.0
write_tests from_latency to_latency percent_change
oltp_delete_insert 5.88 5.88 0.0
oltp_insert 2.91 2.91 0.0
oltp_read_write 11.65 11.45 -1.72
oltp_update_index 2.97 2.97 0.0
oltp_update_non_index 2.91 2.91 0.0
oltp_write_only 5.99 5.99 0.0
types_delete_insert 6.32 6.32 0.0

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Nov 5, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
7bc020e ok 5937457
version total_tests
7bc020e 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 percent_change
tpcc-scale-factor-1 62.19 58.92 -5.26
test_name from_server_name from_server_version from_tps to_server_name to_server_version to_tps percent_change
tpcc-scale-factor-1 dolt 134372c 40.61 dolt 7bc020e 40.92 0.76

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency to_latency percent_change
covering_index_scan 0.62 0.61 -1.61
groupby_scan 16.71 16.71 0.0
index_join 2.35 2.26 -3.83
index_join_scan 1.89 1.82 -3.7
index_scan 55.82 55.82 0.0
oltp_point_select 0.27 0.27 0.0
oltp_read_only 5.47 5.37 -1.83
select_random_points 0.67 0.65 -2.99
select_random_ranges 0.65 0.65 0.0
table_scan 55.82 55.82 0.0
types_table_scan 147.61 147.61 0.0
write_tests from_latency to_latency percent_change
oltp_delete_insert 5.88 5.77 -1.87
oltp_insert 2.86 2.86 0.0
oltp_read_write 11.45 11.45 0.0
oltp_update_index 2.91 2.91 0.0
oltp_update_non_index 2.86 2.86 0.0
oltp_write_only 5.88 5.88 0.0
types_delete_insert 6.32 6.21 -1.74

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants