Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](group commit) Fix Group Commit Selecting Non-Alive Backend Instances #44577

Merged
merged 1 commit into from
Nov 27, 2024

Conversation

Yukang-Lian
Copy link
Collaborator

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39755 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 74f2c90a0a3a23fccadda7ebcd1f6f1d83e9dade, data reload: false

------ Round 1 ----------------------------------
q1	17566	7444	7297	7297
q2	2041	175	171	171
q3	10577	1121	1153	1121
q4	10565	709	719	709
q5	7594	2663	2736	2663
q6	236	147	143	143
q7	989	625	588	588
q8	9246	1826	1920	1826
q9	6508	6386	6386	6386
q10	6947	2295	2325	2295
q11	472	258	258	258
q12	432	217	213	213
q13	17764	3037	3027	3027
q14	239	215	210	210
q15	578	524	512	512
q16	654	588	589	588
q17	973	526	511	511
q18	7319	6708	6771	6708
q19	1348	1024	986	986
q20	488	181	177	177
q21	4075	3195	3059	3059
q22	383	317	307	307
Total cold run time: 106994 ms
Total hot run time: 39755 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7315	7263	7269	7263
q2	332	232	235	232
q3	2883	2774	2966	2774
q4	2076	1851	1863	1851
q5	5700	5648	5687	5648
q6	219	143	145	143
q7	2262	1863	1820	1820
q8	3432	3586	3516	3516
q9	8790	8874	8826	8826
q10	3619	3553	3545	3545
q11	606	508	513	508
q12	824	595	614	595
q13	11220	3272	3237	3237
q14	320	290	280	280
q15	557	520	519	519
q16	710	643	647	643
q17	1851	1635	1613	1613
q18	8358	7751	7627	7627
q19	1671	1628	1603	1603
q20	2134	1913	1888	1888
q21	5610	5522	5463	5463
q22	663	584	556	556
Total cold run time: 71152 ms
Total hot run time: 60150 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196778 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 74f2c90a0a3a23fccadda7ebcd1f6f1d83e9dade, data reload: false

query1	1240	902	940	902
query2	6225	2187	2166	2166
query3	10919	3915	4002	3915
query4	67388	27488	23525	23525
query5	4903	465	456	456
query6	415	177	172	172
query7	5538	299	291	291
query8	315	228	227	227
query9	8873	2758	2735	2735
query10	417	252	259	252
query11	17390	15116	15957	15116
query12	155	104	102	102
query13	1483	436	454	436
query14	10888	7425	7636	7425
query15	203	188	188	188
query16	7083	504	516	504
query17	1079	580	580	580
query18	1859	312	308	308
query19	211	159	160	159
query20	121	123	117	117
query21	200	110	111	110
query22	4633	4548	4678	4548
query23	34822	34261	34346	34261
query24	5895	2596	2598	2596
query25	479	386	389	386
query26	644	148	150	148
query27	1900	288	293	288
query28	4449	2527	2503	2503
query29	695	449	433	433
query30	205	147	155	147
query31	1034	825	841	825
query32	106	53	57	53
query33	410	340	280	280
query34	953	501	511	501
query35	861	750	724	724
query36	1090	965	983	965
query37	127	71	73	71
query38	4544	4428	4346	4346
query39	1553	1475	1477	1475
query40	198	95	100	95
query41	45	42	41	41
query42	110	103	101	101
query43	548	530	535	530
query44	1194	829	848	829
query45	188	166	170	166
query46	1148	727	710	710
query47	2073	1901	1900	1900
query48	422	325	320	320
query49	714	384	409	384
query50	852	391	404	391
query51	7411	7285	7084	7084
query52	101	89	93	89
query53	249	178	182	178
query54	519	392	405	392
query55	80	77	87	77
query56	253	233	237	233
query57	1282	1180	1180	1180
query58	223	227	222	222
query59	3433	3178	3042	3042
query60	282	254	238	238
query61	108	105	116	105
query62	803	650	682	650
query63	216	182	185	182
query64	1350	658	633	633
query65	3281	3218	3198	3198
query66	704	297	303	297
query67	15880	15729	15477	15477
query68	3820	578	576	576
query69	421	263	251	251
query70	1155	1113	1126	1113
query71	382	254	250	250
query72	6335	3984	3808	3808
query73	747	365	367	365
query74	10203	8943	9016	8943
query75	3373	2668	2674	2668
query76	1805	1088	1148	1088
query77	473	268	278	268
query78	10560	9450	9322	9322
query79	1708	602	615	602
query80	1403	454	453	453
query81	524	242	232	232
query82	1281	118	118	118
query83	263	153	155	153
query84	279	73	68	68
query85	1010	297	293	293
query86	453	306	304	304
query87	4667	4621	4522	4522
query88	3709	2240	2224	2224
query89	424	292	305	292
query90	1943	190	185	185
query91	141	102	105	102
query92	69	48	51	48
query93	2372	548	553	548
query94	818	289	301	289
query95	349	252	246	246
query96	633	274	286	274
query97	2866	2704	2704	2704
query98	220	196	192	192
query99	1635	1305	1297	1297
Total cold run time: 321013 ms
Total hot run time: 196778 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 74f2c90a0a3a23fccadda7ebcd1f6f1d83e9dade, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.43	0.42	0.40
query6	1.18	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.60	0.51	0.50
query10	0.57	0.54	0.57
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.82	2.79	2.79
query15	0.91	0.83	0.84
query16	0.39	0.37	0.36
query17	0.99	1.04	1.08
query18	0.22	0.22	0.22
query19	1.80	1.82	1.99
query20	0.01	0.01	0.01
query21	15.36	0.59	0.59
query22	2.47	2.38	1.71
query23	17.00	0.88	0.95
query24	3.19	1.19	1.22
query25	0.24	0.12	0.06
query26	0.45	0.14	0.14
query27	0.04	0.04	0.04
query28	10.55	1.10	1.08
query29	12.58	3.19	3.25
query30	0.25	0.07	0.06
query31	2.85	0.39	0.39
query32	3.26	0.47	0.47
query33	2.98	3.01	3.05
query34	17.15	4.55	4.53
query35	4.64	4.52	4.52
query36	0.66	0.49	0.48
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.03
query40	0.15	0.12	0.12
query41	0.08	0.02	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.96 s
Total hot run time: 32.87 s

@Yukang-Lian
Copy link
Collaborator Author

run p0

@Yukang-Lian
Copy link
Collaborator Author

run cloud_p0

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 26, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@gavinchou gavinchou merged commit 45dc1c9 into apache:master Nov 27, 2024
29 of 31 checks passed
dataroaring pushed a commit that referenced this pull request Nov 28, 2024
…Backend Instances #44577 (#44659)

Cherry-picked from #44577

Co-authored-by: abmdocrt <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants