Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement](cloud) Add concurrency for tablet stats processing #43903

Merged
merged 3 commits into from
Nov 29, 2024
Merged

[Enhancement](cloud) Add concurrency for tablet stats processing #43903

merged 3 commits into from
Nov 29, 2024

Conversation

Yukang-Lian
Copy link
Collaborator

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring
Copy link
Contributor

run buildall

gavinchou
gavinchou requested changes Nov 13, 2024
View reviewed changes
Copy link
Contributor

@gavinchou gavinchou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on MS side, eliminate getting tablet stats one by one
use Transaction::batch_get instead for a bunch of tablets at a time

image

@Yukang-Lian Yukang-Lian force-pushed the Support-Batch-Get-Tablet-Stats branch 2 times, most recently from 4af3bc2 to 624dd80 Compare November 20, 2024 16:44
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

2 similar comments
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40035 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 377adbdc41d5c090b0af14bceb06937c95d609ae, data reload: false

------ Round 1 ----------------------------------
q1	17625	7557	7349	7349
q2	2039	172	164	164
q3	10640	1136	1167	1136
q4	10550	772	703	703
q5	7596	2744	2726	2726
q6	243	152	147	147
q7	998	628	608	608
q8	9247	1821	1911	1821
q9	6459	6393	6402	6393
q10	7003	2326	2331	2326
q11	486	265	253	253
q12	428	221	219	219
q13	17785	3044	3108	3044
q14	243	222	208	208
q15	582	542	509	509
q16	662	591	599	591
q17	985	562	619	562
q18	7247	6703	6633	6633
q19	1332	975	964	964
q20	450	185	183	183
q21	3966	3197	3182	3182
q22	372	321	314	314
Total cold run time: 106938 ms
Total hot run time: 40035 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7342	7307	7306	7306
q2	323	227	230	227
q3	2950	2852	2980	2852
q4	2014	1836	1800	1800
q5	5665	5733	5690	5690
q6	231	151	147	147
q7	2278	1855	1822	1822
q8	3468	3587	3545	3545
q9	8690	8922	8931	8922
q10	3591	3598	3594	3594
q11	605	517	534	517
q12	840	633	619	619
q13	12509	3313	3280	3280
q14	311	278	286	278
q15	577	531	525	525
q16	701	664	653	653
q17	1880	1639	1625	1625
q18	8422	7743	7814	7743
q19	1702	1524	1631	1524
q20	2146	1913	1895	1895
q21	5569	5447	5365	5365
q22	659	583	606	583
Total cold run time: 72473 ms
Total hot run time: 60512 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196907 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 377adbdc41d5c090b0af14bceb06937c95d609ae, data reload: false

query1	1257	941	923	923
query2	6263	2120	2169	2120
query3	10847	3975	3927	3927
query4	67470	26903	23590	23590
query5	4987	454	452	452
query6	416	189	180	180
query7	5549	306	298	298
query8	322	234	221	221
query9	8903	2680	2691	2680
query10	461	260	271	260
query11	17266	15322	15927	15322
query12	152	113	103	103
query13	1515	440	427	427
query14	10237	7486	7904	7486
query15	209	185	173	173
query16	7203	470	514	470
query17	1043	598	580	580
query18	1775	293	298	293
query19	195	155	149	149
query20	115	115	116	115
query21	203	101	109	101
query22	4569	4428	4208	4208
query23	35340	34489	34191	34191
query24	5419	2416	2465	2416
query25	469	395	383	383
query26	645	156	150	150
query27	1691	283	293	283
query28	4145	2427	2442	2427
query29	661	409	404	404
query30	219	145	143	143
query31	1013	790	801	790
query32	66	58	54	54
query33	388	284	299	284
query34	936	494	517	494
query35	833	716	709	709
query36	1069	941	978	941
query37	121	75	76	75
query38	4326	4177	4319	4177
query39	1467	1438	1415	1415
query40	201	98	97	97
query41	45	43	42	42
query42	111	98	102	98
query43	565	533	492	492
query44	1203	823	815	815
query45	190	165	170	165
query46	1117	708	719	708
query47	1944	1842	1839	1839
query48	425	308	317	308
query49	712	404	415	404
query50	815	397	397	397
query51	7316	7194	7113	7113
query52	101	91	93	91
query53	255	180	176	176
query54	529	409	400	400
query55	83	76	81	76
query56	256	240	241	240
query57	1292	1190	1153	1153
query58	220	215	214	214
query59	3357	3313	3387	3313
query60	267	245	252	245
query61	117	117	113	113
query62	788	691	684	684
query63	218	196	188	188
query64	1450	672	632	632
query65	3322	3237	3238	3237
query66	623	317	324	317
query67	15782	15792	15710	15710
query68	3224	589	588	588
query69	403	260	258	258
query70	1179	1140	1129	1129
query71	332	246	258	246
query72	6156	4060	4038	4038
query73	762	360	355	355
query74	9926	9024	9013	9013
query75	3416	2703	2680	2680
query76	1482	1189	1143	1143
query77	572	285	283	283
query78	10443	9406	9552	9406
query79	1429	599	600	599
query80	853	430	437	430
query81	501	247	242	242
query82	1301	118	119	118
query83	268	184	148	148
query84	284	75	76	75
query85	900	350	297	297
query86	343	315	294	294
query87	4656	4562	4582	4562
query88	3538	2193	2177	2177
query89	428	287	291	287
query90	1885	184	187	184
query91	146	103	104	103
query92	66	49	51	49
query93	1768	545	545	545
query94	726	295	296	295
query95	346	251	258	251
query96	612	282	277	277
query97	2871	2647	2685	2647
query98	219	228	200	200
query99	1603	1307	1331	1307
Total cold run time: 315707 ms
Total hot run time: 196907 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.35 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 377adbdc41d5c090b0af14bceb06937c95d609ae, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.11
query5	0.43	0.41	0.42
query6	1.15	0.67	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.50	0.51
query10	0.56	0.54	0.55
query11	0.14	0.11	0.10
query12	0.13	0.11	0.11
query13	0.61	0.61	0.60
query14	2.75	2.88	2.69
query15	0.91	0.83	0.82
query16	0.39	0.37	0.38
query17	1.08	1.05	1.07
query18	0.20	0.20	0.21
query19	2.00	1.77	2.04
query20	0.01	0.01	0.01
query21	15.37	0.59	0.59
query22	2.71	2.33	1.72
query23	17.04	0.90	0.85
query24	2.86	1.36	0.95
query25	0.36	0.06	0.06
query26	0.46	0.14	0.14
query27	0.05	0.06	0.04
query28	10.53	1.09	1.07
query29	12.55	3.19	3.20
query30	0.25	0.06	0.05
query31	2.87	0.39	0.39
query32	3.25	0.47	0.47
query33	3.04	3.07	3.02
query34	17.23	4.46	4.51
query35	4.56	4.43	4.50
query36	0.67	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.26 s
Total hot run time: 32.35 s

@Yukang-Lian Yukang-Lian force-pushed the Support-Batch-Get-Tablet-Stats branch from af07574 to 197581f Compare November 22, 2024 11:33
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39874 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 197581f3522e861eb992cb106db2ae8170e96be2, data reload: false

------ Round 1 ----------------------------------
q1	17593	7449	7237	7237
q2	2053	184	170	170
q3	10624	1091	1179	1091
q4	10545	801	739	739
q5	7619	2676	2695	2676
q6	240	153	147	147
q7	985	637	611	611
q8	9254	1800	1894	1800
q9	6526	6353	6386	6353
q10	7037	2326	2326	2326
q11	465	270	256	256
q12	411	221	224	221
q13	17787	3071	3064	3064
q14	239	209	216	209
q15	587	530	510	510
q16	676	579	601	579
q17	990	564	497	497
q18	7375	6878	6795	6795
q19	1335	1006	1018	1006
q20	470	175	183	175
q21	4028	3101	3187	3101
q22	382	311	313	311
Total cold run time: 107221 ms
Total hot run time: 39874 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7260	7248	7239	7239
q2	327	239	233	233
q3	2895	2809	2951	2809
q4	2056	1831	1789	1789
q5	5760	5686	5687	5686
q6	227	143	138	138
q7	2308	1856	1862	1856
q8	3436	3557	3502	3502
q9	8880	8980	8948	8948
q10	3635	3569	3571	3569
q11	596	513	507	507
q12	811	591	608	591
q13	11495	3273	3217	3217
q14	303	285	289	285
q15	587	521	520	520
q16	699	684	651	651
q17	1869	1631	1630	1630
q18	8429	7900	7809	7809
q19	2382	1608	1650	1608
q20	2110	1858	1907	1858
q21	5592	5491	5414	5414
q22	680	566	550	550
Total cold run time: 72337 ms
Total hot run time: 60409 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198248 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 197581f3522e861eb992cb106db2ae8170e96be2, data reload: false

query1	1250	952	913	913
query2	6245	2205	2178	2178
query3	10778	3952	3981	3952
query4	67972	31450	23640	23640
query5	4775	448	439	439
query6	444	194	183	183
query7	5633	304	286	286
query8	329	231	238	231
query9	9208	2671	2672	2671
query10	474	246	241	241
query11	16515	15500	15891	15500
query12	160	105	108	105
query13	1562	446	420	420
query14	10956	7254	7635	7254
query15	204	190	187	187
query16	7124	483	427	427
query17	1412	578	637	578
query18	1766	311	308	308
query19	191	147	151	147
query20	120	112	113	112
query21	205	138	100	100
query22	4871	4514	4546	4514
query23	34962	34506	34412	34412
query24	5481	2539	2604	2539
query25	533	421	447	421
query26	668	157	149	149
query27	1891	291	301	291
query28	4603	2518	2488	2488
query29	721	452	454	452
query30	221	152	158	152
query31	1041	824	839	824
query32	72	58	62	58
query33	421	307	313	307
query34	958	526	508	508
query35	866	748	750	748
query36	1121	986	970	970
query37	132	75	77	75
query38	4495	4431	4486	4431
query39	1554	1480	1471	1471
query40	209	107	100	100
query41	49	44	48	44
query42	109	99	103	99
query43	555	527	522	522
query44	1251	852	829	829
query45	197	171	168	168
query46	1143	737	724	724
query47	2052	1906	1963	1906
query48	460	331	344	331
query49	734	417	410	410
query50	854	407	410	407
query51	7482	7271	7147	7147
query52	96	91	83	83
query53	257	177	176	176
query54	504	412	397	397
query55	81	71	77	71
query56	286	234	228	228
query57	1305	1202	1172	1172
query58	217	212	222	212
query59	3375	3292	3105	3105
query60	275	242	245	242
query61	114	116	107	107
query62	778	662	684	662
query63	213	194	194	194
query64	1426	671	659	659
query65	3306	3192	3233	3192
query66	690	307	310	307
query67	16186	15792	15892	15792
query68	3939	570	570	570
query69	454	244	261	244
query70	1197	1156	1115	1115
query71	370	247	247	247
query72	6496	4041	4029	4029
query73	854	359	356	356
query74	10384	8987	9051	8987
query75	3464	2714	2674	2674
query76	1816	1064	1086	1064
query77	477	279	274	274
query78	10732	9602	9530	9530
query79	2283	597	603	597
query80	1435	423	443	423
query81	534	225	234	225
query82	1317	114	123	114
query83	184	151	157	151
query84	282	82	70	70
query85	989	306	301	301
query86	400	293	280	280
query87	4772	4681	4607	4607
query88	3606	2253	2219	2219
query89	424	291	292	291
query90	1906	185	189	185
query91	143	112	102	102
query92	57	50	53	50
query93	3030	561	554	554
query94	776	287	295	287
query95	346	245	248	245
query96	638	283	281	281
query97	2933	2697	2722	2697
query98	218	198	195	195
query99	1597	1316	1319	1316
Total cold run time: 324191 ms
Total hot run time: 198248 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.03 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 197581f3522e861eb992cb106db2ae8170e96be2, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.11	0.11
query5	0.46	0.42	0.43
query6	1.20	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.51
query10	0.54	0.55	0.56
query11	0.14	0.09	0.10
query12	0.15	0.12	0.11
query13	0.62	0.61	0.61
query14	2.80	2.84	2.70
query15	0.91	0.83	0.82
query16	0.39	0.39	0.39
query17	1.05	1.07	1.03
query18	0.22	0.20	0.20
query19	1.95	1.90	2.03
query20	0.01	0.01	0.00
query21	15.36	0.60	0.57
query22	2.33	2.78	1.74
query23	17.09	0.89	0.80
query24	3.01	1.58	1.49
query25	0.28	0.08	0.29
query26	0.49	0.12	0.13
query27	0.05	0.04	0.04
query28	10.04	1.10	1.07
query29	12.54	3.27	3.23
query30	0.24	0.06	0.07
query31	2.87	0.40	0.39
query32	3.44	0.46	0.48
query33	3.00	3.08	3.04
query34	16.83	4.44	4.47
query35	4.57	4.52	4.45
query36	0.68	0.50	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.34 s
Total hot run time: 33.03 s

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40034 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 08d0d85f6a0ecd288b32032c124f72b002f9ee28, data reload: false

------ Round 1 ----------------------------------
q1	17659	7523	7365	7365
q2	2061	182	187	182
q3	10601	1089	1172	1089
q4	10564	719	663	663
q5	7619	2772	2751	2751
q6	237	153	150	150
q7	983	642	619	619
q8	9267	1868	1937	1868
q9	6508	6361	6414	6361
q10	7033	2286	2328	2286
q11	466	271	263	263
q12	418	224	218	218
q13	17784	3075	3076	3075
q14	252	212	206	206
q15	581	536	538	536
q16	681	597	588	588
q17	987	576	532	532
q18	7347	6733	6899	6733
q19	1349	1046	934	934
q20	467	190	186	186
q21	4220	3226	3107	3107
q22	378	325	322	322
Total cold run time: 107462 ms
Total hot run time: 40034 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7259	7309	7254	7254
q2	328	226	229	226
q3	2919	2871	2979	2871
q4	2079	1813	1808	1808
q5	5724	5696	5736	5696
q6	231	146	142	142
q7	2298	1829	1791	1791
q8	3431	3569	3506	3506
q9	8869	8879	8905	8879
q10	3587	3542	3581	3542
q11	598	501	511	501
q12	819	613	596	596
q13	10496	3328	3302	3302
q14	318	278	272	272
q15	587	530	518	518
q16	711	660	632	632
q17	1886	1683	1617	1617
q18	8329	7788	7518	7518
q19	1717	1673	1623	1623
q20	2153	1892	1876	1876
q21	5649	5306	5423	5306
q22	631	593	573	573
Total cold run time: 70619 ms
Total hot run time: 60049 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197636 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 08d0d85f6a0ecd288b32032c124f72b002f9ee28, data reload: false

query1	1245	957	970	957
query2	6254	2129	2030	2030
query3	10827	4010	4028	4010
query4	67393	34084	24010	24010
query5	4874	450	442	442
query6	294	183	165	165
query7	4876	298	293	293
query8	278	226	220	220
query9	6626	2757	2735	2735
query10	396	265	255	255
query11	15778	15453	15891	15453
query12	157	107	103	103
query13	990	434	446	434
query14	9901	6721	7441	6721
query15	214	194	208	194
query16	7175	462	476	462
query17	1238	622	584	584
query18	1856	322	310	310
query19	216	159	168	159
query20	123	123	115	115
query21	205	103	107	103
query22	4780	4554	4706	4554
query23	34822	34374	34307	34307
query24	5409	2494	2518	2494
query25	514	410	392	392
query26	644	152	147	147
query27	1954	281	290	281
query28	4336	2511	2472	2472
query29	686	431	414	414
query30	217	146	150	146
query31	979	849	883	849
query32	69	59	58	58
query33	449	295	292	292
query34	942	528	533	528
query35	853	736	729	729
query36	1073	965	982	965
query37	126	75	80	75
query38	4611	4405	4399	4399
query39	1543	1447	1485	1447
query40	205	101	96	96
query41	48	42	47	42
query42	112	103	99	99
query43	540	501	498	498
query44	1202	851	838	838
query45	195	166	167	166
query46	1143	712	711	711
query47	2014	1898	1951	1898
query48	419	326	326	326
query49	752	389	395	389
query50	872	397	402	397
query51	7435	7336	7068	7068
query52	99	87	88	87
query53	253	186	177	177
query54	511	393	396	393
query55	81	73	80	73
query56	250	248	254	248
query57	1293	1191	1171	1171
query58	227	218	208	208
query59	3220	2972	3011	2972
query60	275	242	233	233
query61	113	111	110	110
query62	774	664	676	664
query63	215	195	191	191
query64	1386	656	644	644
query65	3270	3191	3218	3191
query66	717	304	312	304
query67	15987	15954	15925	15925
query68	3481	587	584	584
query69	451	305	250	250
query70	1196	1112	1093	1093
query71	406	243	244	243
query72	6643	3982	4060	3982
query73	772	367	366	366
query74	9793	9098	9183	9098
query75	3431	2733	2659	2659
query76	2066	1151	1118	1118
query77	580	294	287	287
query78	10651	9506	9411	9411
query79	1133	598	601	598
query80	754	431	500	431
query81	519	233	226	226
query82	232	115	121	115
query83	186	151	160	151
query84	283	72	72	72
query85	943	308	302	302
query86	337	288	304	288
query87	4635	4555	4594	4555
query88	3896	2270	2234	2234
query89	425	332	297	297
query90	2196	198	186	186
query91	142	112	107	107
query92	65	51	52	51
query93	1387	535	539	535
query94	768	295	290	290
query95	340	251	253	251
query96	608	284	288	284
query97	2849	2674	2681	2674
query98	216	188	196	188
query99	1702	1309	1287	1287
Total cold run time: 311617 ms
Total hot run time: 197636 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.92 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 08d0d85f6a0ecd288b32032c124f72b002f9ee28, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.11	0.10
query5	0.43	0.42	0.41
query6	1.18	0.65	0.66
query7	0.01	0.02	0.02
query8	0.04	0.04	0.03
query9	0.59	0.52	0.50
query10	0.56	0.56	0.57
query11	0.15	0.11	0.11
query12	0.14	0.12	0.11
query13	0.61	0.59	0.61
query14	2.78	2.70	2.86
query15	0.92	0.85	0.85
query16	0.39	0.39	0.38
query17	1.08	1.08	1.01
query18	0.22	0.20	0.21
query19	1.86	1.85	1.96
query20	0.01	0.01	0.00
query21	15.35	0.61	0.58
query22	2.65	1.74	1.68
query23	16.88	0.91	0.83
query24	3.11	1.39	1.48
query25	0.31	0.22	0.16
query26	0.30	0.13	0.13
query27	0.04	0.05	0.05
query28	10.25	1.11	1.07
query29	12.60	3.23	3.26
query30	0.24	0.06	0.06
query31	2.87	0.40	0.38
query32	3.29	0.47	0.46
query33	3.09	3.01	3.04
query34	16.94	4.44	4.44
query35	4.55	4.54	4.45
query36	0.66	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.52 s
Total hot run time: 32.92 s

@Yukang-Lian
Copy link
Collaborator Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39870 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 90576bcd2ee59e1e1ddae3dcac4b7b6089380296, data reload: false

------ Round 1 ----------------------------------
q1	17598	7468	7339	7339
q2	2041	172	163	163
q3	10612	1076	1217	1076
q4	10557	744	750	744
q5	7622	2740	2725	2725
q6	239	149	145	145
q7	970	622	608	608
q8	9236	1813	1946	1813
q9	6558	6368	6387	6368
q10	7041	2299	2323	2299
q11	453	255	262	255
q12	421	221	216	216
q13	17771	3023	3054	3023
q14	255	216	215	215
q15	576	535	530	530
q16	663	579	594	579
q17	981	552	573	552
q18	7154	6702	6610	6610
q19	1333	982	1003	982
q20	479	181	181	181
q21	3999	3233	3132	3132
q22	373	315	315	315
Total cold run time: 106932 ms
Total hot run time: 39870 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7265	7273	7297	7273
q2	326	226	227	226
q3	2898	2823	3139	2823
q4	2077	1741	1785	1741
q5	5733	5701	5693	5693
q6	227	139	142	139
q7	2282	1841	1834	1834
q8	3396	3856	3540	3540
q9	8849	8899	8857	8857
q10	3609	3551	3564	3551
q11	599	508	516	508
q12	813	611	622	611
q13	9922	3226	3245	3226
q14	304	276	280	276
q15	560	541	529	529
q16	707	656	628	628
q17	1847	1626	1628	1626
q18	8243	7858	7541	7541
q19	1739	1519	1510	1510
q20	2127	1866	1858	1858
q21	5524	5362	5354	5354
q22	665	559	559	559
Total cold run time: 69712 ms
Total hot run time: 59903 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 199171 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 90576bcd2ee59e1e1ddae3dcac4b7b6089380296, data reload: false

query1	1256	948	951	948
query2	6211	2139	2056	2056
query3	10792	4051	3952	3952
query4	67151	28974	23589	23589
query5	4979	483	490	483
query6	408	205	199	199
query7	5544	303	286	286
query8	307	220	217	217
query9	8641	2785	2768	2768
query10	435	259	256	256
query11	17345	15236	15898	15236
query12	153	114	110	110
query13	1494	447	470	447
query14	10378	8160	7921	7921
query15	216	196	190	190
query16	7085	464	483	464
query17	1235	619	612	612
query18	1871	342	319	319
query19	211	170	167	167
query20	137	120	124	120
query21	206	120	113	113
query22	4723	4658	4396	4396
query23	35372	34548	34433	34433
query24	5405	2584	2658	2584
query25	501	394	410	394
query26	649	157	156	156
query27	1722	286	288	286
query28	4676	2532	2487	2487
query29	704	433	440	433
query30	216	157	164	157
query31	1046	853	860	853
query32	70	62	53	53
query33	436	293	309	293
query34	990	552	523	523
query35	888	771	764	764
query36	1159	971	997	971
query37	135	81	80	80
query38	4514	4498	4441	4441
query39	1564	1466	1493	1466
query40	202	102	102	102
query41	45	43	43	43
query42	112	99	101	99
query43	554	503	515	503
query44	1208	824	830	824
query45	196	169	171	169
query46	1244	761	771	761
query47	2107	1902	1898	1898
query48	427	327	329	327
query49	735	408	426	408
query50	896	413	411	411
query51	7485	7403	7188	7188
query52	98	89	96	89
query53	264	185	181	181
query54	531	397	396	396
query55	77	77	86	77
query56	254	247	249	247
query57	1317	1207	1185	1185
query58	238	220	225	220
query59	3275	3105	3064	3064
query60	273	253	244	244
query61	112	124	105	105
query62	831	691	698	691
query63	227	204	192	192
query64	1367	662	691	662
query65	3453	3271	3333	3271
query66	707	317	321	317
query67	15994	15708	15614	15614
query68	4094	586	559	559
query69	435	264	263	263
query70	1209	1174	1142	1142
query71	363	251	253	251
query72	6216	4181	4207	4181
query73	819	376	362	362
query74	10245	9074	9019	9019
query75	3524	2780	2763	2763
query76	1871	1136	1151	1136
query77	493	287	279	279
query78	10610	9566	9415	9415
query79	1637	609	606	606
query80	892	443	436	436
query81	520	244	241	241
query82	1322	127	125	125
query83	271	157	153	153
query84	358	74	72	72
query85	896	292	301	292
query86	347	309	303	303
query87	4761	4659	4548	4548
query88	3730	2266	2236	2236
query89	499	315	314	314
query90	2001	189	189	189
query91	135	104	103	103
query92	63	52	52	52
query93	1953	545	545	545
query94	767	306	282	282
query95	352	250	258	250
query96	675	281	283	281
query97	2928	2737	2723	2723
query98	224	207	204	204
query99	1711	1360	1335	1335
Total cold run time: 320975 ms
Total hot run time: 199171 ms

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39928 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 21a07716629f93dd944780f91a959fbd59351ce0, data reload: false

------ Round 1 ----------------------------------
q1	17581	7823	7361	7361
q2	2039	171	167	167
q3	10590	1098	1153	1098
q4	10466	797	710	710
q5	7619	2729	2680	2680
q6	269	153	149	149
q7	998	634	610	610
q8	9253	1799	1911	1799
q9	6535	6441	6472	6441
q10	7052	2302	2287	2287
q11	458	265	260	260
q12	416	221	228	221
q13	17797	3006	3012	3006
q14	243	223	212	212
q15	585	532	521	521
q16	665	598	601	598
q17	983	554	546	546
q18	7250	6561	6690	6561
q19	1336	1083	1005	1005
q20	465	186	185	185
q21	3971	3335	3194	3194
q22	380	321	317	317
Total cold run time: 106951 ms
Total hot run time: 39928 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7286	7282	7295	7282
q2	328	228	236	228
q3	2892	2795	2940	2795
q4	2024	1862	1851	1851
q5	5700	5689	5684	5684
q6	232	145	149	145
q7	2260	1788	1800	1788
q8	3434	3591	3542	3542
q9	8759	8887	8901	8887
q10	3598	3590	3617	3590
q11	597	500	513	500
q12	797	608	626	608
q13	11407	3214	3203	3203
q14	308	279	292	279
q15	593	524	531	524
q16	699	660	654	654
q17	1861	1640	1626	1626
q18	8141	7848	7420	7420
q19	3552	1619	1580	1580
q20	2127	1867	1875	1867
q21	5646	5286	5503	5286
q22	653	551	555	551
Total cold run time: 72894 ms
Total hot run time: 59890 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196970 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 21a07716629f93dd944780f91a959fbd59351ce0, data reload: false

query1	1241	927	934	927
query2	6233	2120	2052	2052
query3	10784	4030	3880	3880
query4	67183	26846	23562	23562
query5	4914	457	453	453
query6	422	183	180	180
query7	5627	308	296	296
query8	300	234	214	214
query9	9550	2733	2718	2718
query10	447	248	251	248
query11	17214	15208	15976	15208
query12	163	114	117	114
query13	1575	437	445	437
query14	10545	7797	7420	7420
query15	215	197	201	197
query16	7124	484	433	433
query17	1220	612	605	605
query18	1833	315	312	312
query19	208	161	173	161
query20	125	121	121	121
query21	212	113	112	112
query22	4581	4715	4501	4501
query23	35321	34327	34843	34327
query24	5429	2596	2505	2505
query25	517	387	400	387
query26	656	159	148	148
query27	2000	291	298	291
query28	4142	2493	2469	2469
query29	669	438	447	438
query30	211	147	151	147
query31	1015	842	853	842
query32	71	62	55	55
query33	452	305	285	285
query34	960	515	537	515
query35	877	747	743	743
query36	1090	958	979	958
query37	120	74	75	74
query38	4496	4267	4421	4267
query39	1539	1463	1499	1463
query40	209	101	96	96
query41	45	41	43	41
query42	111	106	101	101
query43	563	503	498	498
query44	1190	830	851	830
query45	183	167	166	166
query46	1164	706	737	706
query47	2072	1907	1915	1907
query48	422	323	319	319
query49	723	407	381	381
query50	847	394	401	394
query51	7514	7245	7292	7245
query52	98	84	98	84
query53	257	180	180	180
query54	524	402	387	387
query55	79	75	75	75
query56	265	234	242	234
query57	1294	1168	1163	1163
query58	223	212	206	206
query59	3198	3125	3027	3027
query60	257	237	243	237
query61	140	106	122	106
query62	796	700	668	668
query63	217	190	183	183
query64	1352	657	656	656
query65	3280	3225	3225	3225
query66	697	301	311	301
query67	16002	15682	15547	15547
query68	3941	583	536	536
query69	417	251	249	249
query70	1233	1160	1120	1120
query71	414	241	252	241
query72	6063	4032	3967	3967
query73	767	362	369	362
query74	10222	8990	9092	8990
query75	3471	2680	2666	2666
query76	1744	1107	1125	1107
query77	470	273	279	273
query78	10505	9470	9377	9377
query79	1547	590	591	590
query80	893	435	480	435
query81	508	235	229	229
query82	1308	119	120	119
query83	265	149	156	149
query84	277	86	81	81
query85	916	303	297	297
query86	352	301	303	301
query87	4746	4649	4501	4501
query88	3551	2239	2199	2199
query89	423	288	303	288
query90	2030	193	190	190
query91	137	106	105	105
query92	70	48	51	48
query93	1940	542	534	534
query94	860	303	294	294
query95	362	254	242	242
query96	641	280	278	278
query97	2897	2719	2679	2679
query98	223	201	196	196
query99	1661	1299	1305	1299
Total cold run time: 319857 ms
Total hot run time: 196970 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.65 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 21a07716629f93dd944780f91a959fbd59351ce0, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.08	0.07
query4	1.60	0.10	0.11
query5	0.40	0.41	0.43
query6	1.16	0.67	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.57	0.52	0.50
query10	0.56	0.55	0.55
query11	0.13	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.60	0.60
query14	2.82	2.83	2.71
query15	0.91	0.84	0.84
query16	0.38	0.38	0.38
query17	1.00	1.01	1.05
query18	0.23	0.23	0.21
query19	1.88	1.87	2.06
query20	0.01	0.01	0.01
query21	15.36	0.58	0.57
query22	2.36	2.05	2.45
query23	16.80	0.93	0.90
query24	2.52	1.36	1.77
query25	0.25	0.15	0.20
query26	0.33	0.15	0.13
query27	0.05	0.04	0.05
query28	10.22	1.12	1.09
query29	12.56	3.46	3.46
query30	0.24	0.06	0.06
query31	2.87	0.39	0.38
query32	3.25	0.46	0.46
query33	2.97	3.02	3.05
query34	16.99	4.51	4.51
query35	4.54	4.48	4.46
query36	0.67	0.50	0.48
query37	0.08	0.06	0.05
query38	0.05	0.04	0.03
query39	0.03	0.03	0.03
query40	0.16	0.12	0.12
query41	0.07	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 105.24 s
Total hot run time: 33.65 s

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39924 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9665ce1a3a86bd6ef291a4d4890e2559c790c782, data reload: false

------ Round 1 ----------------------------------
q1	17602	7490	7291	7291
q2	2053	199	191	191
q3	10510	1063	1203	1063
q4	10376	739	679	679
q5	7618	2739	2716	2716
q6	235	149	150	149
q7	979	622	620	620
q8	9241	1867	1961	1867
q9	6525	6424	6425	6424
q10	7005	2279	2319	2279
q11	484	270	268	268
q12	408	213	215	213
q13	17817	3036	3062	3036
q14	248	210	213	210
q15	572	531	508	508
q16	661	587	588	587
q17	994	479	591	479
q18	7205	6698	6690	6690
q19	1348	1013	1000	1000
q20	481	181	188	181
q21	4026	3169	3154	3154
q22	384	319	326	319
Total cold run time: 106772 ms
Total hot run time: 39924 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7277	7258	7238	7238
q2	323	228	229	228
q3	2930	2810	3006	2810
q4	2055	1834	1833	1833
q5	5655	5671	5651	5651
q6	224	146	142	142
q7	2276	1893	1828	1828
q8	3397	3576	3492	3492
q9	8955	8879	8969	8879
q10	3614	3580	3540	3540
q11	586	517	519	517
q12	859	596	622	596
q13	12328	3313	3296	3296
q14	303	264	270	264
q15	570	524	536	524
q16	681	660	633	633
q17	1854	1637	1601	1601
q18	8444	7696	7556	7556
q19	1666	1447	1547	1447
q20	2114	1942	1884	1884
q21	5530	5450	5526	5450
q22	620	560	554	554
Total cold run time: 72261 ms
Total hot run time: 59963 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196371 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9665ce1a3a86bd6ef291a4d4890e2559c790c782, data reload: false

query1	1290	962	969	962
query2	6391	2121	1993	1993
query3	10944	4045	4104	4045
query4	67865	33636	23800	23800
query5	5045	456	431	431
query6	313	210	198	198
query7	4870	314	307	307
query8	292	244	238	238
query9	6472	2775	2762	2762
query10	422	256	245	245
query11	15626	15496	15839	15496
query12	156	110	106	106
query13	1054	423	456	423
query14	10519	7222	6580	6580
query15	222	199	184	184
query16	7078	488	538	488
query17	1033	563	579	563
query18	1914	293	322	293
query19	206	154	154	154
query20	131	110	107	107
query21	199	104	103	103
query22	4786	4483	4700	4483
query23	34599	34152	34484	34152
query24	5486	2490	2590	2490
query25	499	422	395	395
query26	646	160	152	152
query27	1710	303	294	294
query28	4386	2517	2472	2472
query29	697	420	409	409
query30	212	153	145	145
query31	1006	870	821	821
query32	67	58	59	58
query33	448	268	276	268
query34	911	521	514	514
query35	869	755	757	755
query36	1111	959	999	959
query37	124	77	77	77
query38	4483	4331	4350	4331
query39	1500	1462	1468	1462
query40	200	100	100	100
query41	47	49	49	49
query42	111	104	113	104
query43	541	498	496	496
query44	1202	824	806	806
query45	185	166	164	164
query46	1150	688	718	688
query47	2029	1924	1977	1924
query48	430	320	341	320
query49	710	393	407	393
query50	838	409	398	398
query51	7283	7179	7069	7069
query52	99	87	90	87
query53	254	181	179	179
query54	504	415	397	397
query55	79	74	76	74
query56	255	243	237	237
query57	1251	1171	1160	1160
query58	217	220	234	220
query59	3140	2881	2925	2881
query60	262	262	254	254
query61	114	108	111	108
query62	831	664	672	664
query63	214	185	193	185
query64	1372	686	640	640
query65	3300	3286	3227	3227
query66	704	305	317	305
query67	15992	15670	15715	15670
query68	3812	589	558	558
query69	432	258	256	256
query70	1200	1173	1102	1102
query71	385	277	258	258
query72	6637	4089	3947	3947
query73	772	369	358	358
query74	10070	8968	9090	8968
query75	3386	2655	2685	2655
query76	1614	957	1178	957
query77	580	285	281	281
query78	10568	9617	9372	9372
query79	1732	604	601	601
query80	1300	425	434	425
query81	531	236	229	229
query82	1198	117	114	114
query83	177	146	159	146
query84	291	83	78	78
query85	1172	370	298	298
query86	406	302	304	302
query87	4819	4587	4591	4587
query88	3776	2276	2223	2223
query89	404	286	290	286
query90	2040	187	191	187
query91	133	101	102	101
query92	63	49	51	49
query93	2016	544	544	544
query94	849	256	307	256
query95	346	249	252	249
query96	625	278	282	278
query97	2827	2712	2711	2711
query98	216	207	197	197
query99	1807	1306	1284	1284
Total cold run time: 315080 ms
Total hot run time: 196371 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.76 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9665ce1a3a86bd6ef291a4d4890e2559c790c782, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.63	0.10	0.11
query5	0.42	0.43	0.39
query6	1.16	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.53	0.52
query10	0.56	0.55	0.57
query11	0.14	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.61	0.61
query14	2.71	2.73	2.79
query15	0.90	0.83	0.82
query16	0.39	0.39	0.39
query17	1.06	1.04	1.05
query18	0.22	0.21	0.21
query19	1.90	1.77	2.01
query20	0.01	0.01	0.02
query21	15.37	0.61	0.59
query22	3.07	2.98	1.92
query23	17.20	0.88	0.75
query24	3.48	1.58	1.01
query25	0.26	0.15	0.05
query26	0.55	0.14	0.14
query27	0.06	0.04	0.04
query28	9.71	1.12	1.07
query29	12.58	3.21	3.18
query30	0.25	0.06	0.06
query31	2.88	0.38	0.38
query32	3.27	0.48	0.47
query33	2.96	3.09	3.11
query34	16.97	4.55	4.50
query35	4.51	4.53	4.56
query36	0.67	0.49	0.47
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.11 s
Total hot run time: 32.76 s

@Yukang-Lian Yukang-Lian force-pushed the Support-Batch-Get-Tablet-Stats branch from 9665ce1 to 191417f Compare November 27, 2024 02:47
@Yukang-Lian
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39834 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1950219cebf294b52cf65fd5650160ffac757f55, data reload: false

------ Round 1 ----------------------------------
q1	17617	7477	7237	7237
q2	2035	175	168	168
q3	10602	1072	1181	1072
q4	10567	852	765	765
q5	7580	2705	2675	2675
q6	238	146	144	144
q7	999	652	603	603
q8	9240	1845	1938	1845
q9	6495	6366	6342	6342
q10	7051	2294	2336	2294
q11	480	261	259	259
q12	419	211	218	211
q13	17767	3022	3052	3022
q14	247	225	222	222
q15	573	542	514	514
q16	677	589	602	589
q17	969	586	560	560
q18	7204	6798	6699	6699
q19	1327	977	1035	977
q20	470	181	179	179
q21	3939	3307	3148	3148
q22	403	323	309	309
Total cold run time: 106899 ms
Total hot run time: 39834 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7379	7211	7394	7211
q2	322	232	236	232
q3	2936	2792	2961	2792
q4	2062	1732	1810	1732
q5	5655	5693	5624	5624
q6	223	142	138	138
q7	2263	1854	1819	1819
q8	3454	3552	3483	3483
q9	8808	8938	8825	8825
q10	3607	3528	3540	3528
q11	609	506	520	506
q12	835	614	640	614
q13	11925	3177	3219	3177
q14	327	271	269	269
q15	582	527	518	518
q16	699	656	656	656
q17	1834	1623	1636	1623
q18	8427	7845	7572	7572
q19	2747	1551	1682	1551
q20	2147	1842	1840	1840
q21	5398	5372	5273	5273
q22	647	554	615	554
Total cold run time: 72886 ms
Total hot run time: 59537 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196478 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1950219cebf294b52cf65fd5650160ffac757f55, data reload: false

query1	1251	952	954	952
query2	6252	2077	1969	1969
query3	10816	3995	3934	3934
query4	67119	29125	23572	23572
query5	4940	455	459	455
query6	426	198	186	186
query7	5608	298	306	298
query8	316	235	242	235
query9	9047	2777	2769	2769
query10	442	259	253	253
query11	17238	15167	15963	15167
query12	152	105	98	98
query13	1507	418	432	418
query14	10278	7024	6786	6786
query15	212	194	190	190
query16	7184	464	505	464
query17	1073	578	582	578
query18	1931	304	294	294
query19	190	152	167	152
query20	116	108	109	108
query21	200	100	102	100
query22	4545	4696	4438	4438
query23	35001	34291	34641	34291
query24	5691	2455	2445	2445
query25	491	383	386	383
query26	642	148	150	148
query27	2071	290	284	284
query28	4623	2523	2492	2492
query29	689	434	415	415
query30	224	151	150	150
query31	1033	845	872	845
query32	70	60	61	60
query33	424	309	319	309
query34	935	530	533	530
query35	882	738	754	738
query36	1092	1010	988	988
query37	123	83	74	74
query38	4533	4367	4403	4367
query39	1519	1461	1474	1461
query40	202	101	97	97
query41	45	41	41	41
query42	115	99	98	98
query43	534	489	505	489
query44	1174	835	826	826
query45	186	173	170	170
query46	1151	736	744	736
query47	2014	1897	1931	1897
query48	432	333	330	330
query49	725	445	392	392
query50	867	396	404	396
query51	7357	7306	7092	7092
query52	98	92	92	92
query53	245	187	177	177
query54	497	406	392	392
query55	78	80	73	73
query56	244	246	244	244
query57	1281	1178	1147	1147
query58	216	214	214	214
query59	3240	3138	3016	3016
query60	275	249	249	249
query61	109	101	103	101
query62	801	696	669	669
query63	209	183	182	182
query64	1360	673	634	634
query65	3288	3206	3235	3206
query66	638	303	299	299
query67	15968	15713	15687	15687
query68	3790	592	571	571
query69	405	255	252	252
query70	1189	1090	1161	1090
query71	341	253	251	251
query72	6469	4190	4116	4116
query73	762	356	361	356
query74	10079	8890	8987	8890
query75	3397	2659	2692	2659
query76	1778	1127	1166	1127
query77	482	269	280	269
query78	10404	9422	9457	9422
query79	2052	607	611	607
query80	1272	430	431	430
query81	523	238	227	227
query82	472	122	113	113
query83	175	145	154	145
query84	287	69	76	69
query85	1082	297	292	292
query86	389	312	303	303
query87	4742	4655	4602	4602
query88	3686	2258	2219	2219
query89	421	302	290	290
query90	1927	185	182	182
query91	137	99	105	99
query92	68	49	55	49
query93	2596	549	547	547
query94	792	291	294	291
query95	343	254	247	247
query96	638	275	283	275
query97	2896	2679	2710	2679
query98	215	193	202	193
query99	1619	1341	1274	1274
Total cold run time: 319663 ms
Total hot run time: 196478 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.81 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1950219cebf294b52cf65fd5650160ffac757f55, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.43	0.41	0.39
query6	1.14	0.67	0.66
query7	0.01	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.51
query10	0.56	0.56	0.56
query11	0.14	0.10	0.10
query12	0.14	0.12	0.10
query13	0.61	0.60	0.61
query14	2.76	2.84	2.77
query15	0.91	0.84	0.82
query16	0.38	0.38	0.37
query17	1.03	1.07	1.04
query18	0.22	0.21	0.21
query19	1.91	1.91	2.03
query20	0.01	0.01	0.01
query21	15.38	0.60	0.61
query22	2.75	1.99	1.99
query23	17.15	1.07	0.78
query24	3.33	0.98	2.04
query25	0.21	0.21	0.08
query26	0.62	0.14	0.14
query27	0.05	0.07	0.04
query28	9.94	1.10	1.07
query29	12.55	3.21	3.19
query30	0.25	0.06	0.06
query31	2.86	0.39	0.39
query32	3.26	0.47	0.49
query33	2.98	3.09	3.03
query34	17.07	4.42	4.46
query35	4.48	4.46	4.44
query36	0.67	0.50	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.84 s
Total hot run time: 32.81 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 27, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

hust-hhb
hust-hhb approved these changes Nov 28, 2024
View reviewed changes
Copy link
Contributor

@hust-hhb hust-hhb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gavinchou gavinchou merged commit 2a5e992 into apache:master Nov 29, 2024
26 of 28 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 29, 2024
@Yukang-Lian
[Enhancement](cloud) Add concurrency for tablet stats processing (#43903
35652d1 
)
@github-actions github-actions bot mentioned this pull request Nov 29, 2024
Merged
gavinchou pushed a commit that referenced this pull request Dec 24, 2024
@github-actions @Yukang-Lian
branch-3.0: [Enhancement](cloud) Add concurrency for tablet stats pro…
9219689 
…cessing #43903 (#44763)

Cherry-picked from #43903

Co-authored-by: abmdocrt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hust-hhb hust-hhb hust-hhb approved these changes

@luwei16 luwei16 luwei16 approved these changes

@gavinchou gavinchou gavinchou approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@Yukang-Lian @doris-robot @dataroaring @gavinchou @luwei16 @hust-hhb