Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](cluster key) 3.0 forbid mow cluster key #44737

Merged
merged 5 commits into from
Dec 2, 2024
Merged

[fix](cluster key) 3.0 forbid mow cluster key #44737

merged 5 commits into from
Dec 2, 2024

Conversation

mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Nov 28, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

branch-3.0 forbid cluster key in 3.0 temporarily

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 28, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40554 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f003b72eb6098116c824542d9ae7036a0c733bfb, data reload: false

------ Round 1 ----------------------------------
q1	18676	7453	7355	7355
q2	2454	171	158	158
q3	11882	1120	1119	1119
q4	10417	738	788	738
q5	8274	2899	2861	2861
q6	233	144	142	142
q7	976	614	593	593
q8	9353	1866	1925	1866
q9	6633	6375	6380	6375
q10	6996	2254	2312	2254
q11	460	268	265	265
q12	401	215	217	215
q13	17784	2951	2963	2951
q14	238	208	209	208
q15	544	513	526	513
q16	672	620	609	609
q17	955	542	593	542
q18	7384	6573	6534	6534
q19	1919	1055	1049	1049
q20	475	199	199	199
q21	3885	3493	3058	3058
q22	1093	950	1002	950
Total cold run time: 111704 ms
Total hot run time: 40554 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7331	7218	7311	7218
q2	322	225	238	225
q3	3147	2952	2812	2812
q4	2004	1778	1842	1778
q5	5626	5698	5623	5623
q6	218	137	137	137
q7	2202	1747	1736	1736
q8	3311	3491	3441	3441
q9	8820	8832	8850	8832
q10	3519	3479	3507	3479
q11	599	505	503	503
q12	772	594	578	578
q13	17110	3110	2942	2942
q14	283	258	264	258
q15	536	508	511	508
q16	694	644	656	644
q17	1784	1594	1552	1552
q18	7677	7529	7193	7193
q19	2915	1570	1561	1561
q20	1984	1775	1739	1739
q21	5164	5006	5090	5006
q22	1024	984	994	984
Total cold run time: 77042 ms
Total hot run time: 58749 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189800 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f003b72eb6098116c824542d9ae7036a0c733bfb, data reload: false

query1	968	363	349	349
query2	6515	2055	1998	1998
query3	6700	208	204	204
query4	33999	23368	23399	23368
query5	4297	453	440	440
query6	260	168	172	168
query7	4633	311	323	311
query8	280	223	228	223
query9	9675	2687	2674	2674
query10	475	255	255	255
query11	18012	15223	15176	15176
query12	154	104	101	101
query13	1680	432	408	408
query14	9887	6747	7146	6747
query15	212	177	175	175
query16	7808	470	466	466
query17	1627	563	556	556
query18	1985	302	296	296
query19	211	150	150	150
query20	114	107	108	107
query21	204	103	104	103
query22	4298	4019	4168	4019
query23	34394	33714	34471	33714
query24	12337	2847	2880	2847
query25	615	408	402	402
query26	1381	164	167	164
query27	2927	309	299	299
query28	8238	2502	2482	2482
query29	721	461	439	439
query30	336	159	153	153
query31	1022	789	796	789
query32	96	58	57	57
query33	766	286	280	280
query34	997	488	512	488
query35	827	693	731	693
query36	1108	934	956	934
query37	210	79	73	73
query38	3935	3849	3855	3849
query39	1506	1418	1444	1418
query40	291	100	101	100
query41	55	49	50	49
query42	111	98	99	98
query43	527	480	484	480
query44	1254	801	792	792
query45	178	165	166	165
query46	1120	715	708	708
query47	1903	1836	1814	1814
query48	449	365	378	365
query49	1277	402	409	402
query50	785	403	421	403
query51	6998	7065	7075	7065
query52	106	93	91	91
query53	259	183	185	183
query54	1124	467	455	455
query55	78	77	78	77
query56	262	252	239	239
query57	1202	1085	1099	1085
query58	230	233	220	220
query59	3153	3011	2953	2953
query60	290	264	278	264
query61	106	101	106	101
query62	862	661	660	660
query63	212	183	182	182
query64	5165	639	597	597
query65	3232	3169	3177	3169
query66	1371	302	298	298
query67	15885	15334	15237	15237
query68	4803	543	557	543
query69	425	254	256	254
query70	1157	1136	1139	1136
query71	397	249	244	244
query72	6378	3962	3903	3903
query73	759	348	344	344
query74	10254	8911	8823	8823
query75	3338	2612	2601	2601
query76	2912	1123	1073	1073
query77	427	286	279	279
query78	10653	9702	9567	9567
query79	9263	590	598	590
query80	2263	422	413	413
query81	567	238	236	236
query82	1405	126	119	119
query83	260	138	134	134
query84	287	80	76	76
query85	2261	303	294	294
query86	504	266	299	266
query87	4376	4287	4260	4260
query88	5756	2389	2401	2389
query89	560	294	289	289
query90	2037	184	187	184
query91	177	142	142	142
query92	63	49	49	49
query93	6394	562	548	548
query94	961	276	279	276
query95	349	255	248	248
query96	637	280	274	274
query97	3337	3120	3130	3120
query98	224	208	196	196
query99	1874	1303	1304	1303
Total cold run time: 318953 ms
Total hot run time: 189800 ms

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 28, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40859 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7ba9185a2e901adf4f3229a63157fb7011b7bb23, data reload: false

------ Round 1 ----------------------------------
q1	17893	7501	7379	7379
q2	2046	162	159	159
q3	10573	1127	1113	1113
q4	10221	773	761	761
q5	7755	3042	2847	2847
q6	231	143	146	143
q7	980	612	605	605
q8	9368	1952	2047	1952
q9	6684	6437	6437	6437
q10	7016	2281	2311	2281
q11	457	261	271	261
q12	400	218	213	213
q13	17793	2983	3012	2983
q14	250	211	216	211
q15	557	524	526	524
q16	671	605	596	596
q17	983	648	531	531
q18	7476	6654	6577	6577
q19	3641	1059	900	900
q20	487	194	194	194
q21	3939	3212	3218	3212
q22	1123	980	999	980
Total cold run time: 110544 ms
Total hot run time: 40859 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7380	7246	7259	7246
q2	327	230	228	228
q3	3149	2959	3056	2959
q4	1974	1810	1786	1786
q5	5692	5752	5672	5672
q6	222	143	144	143
q7	2186	1772	1767	1767
q8	3328	3531	3475	3475
q9	8889	8952	8920	8920
q10	3607	3521	3539	3521
q11	599	529	492	492
q12	808	590	587	587
q13	17061	3126	2969	2969
q14	285	251	254	251
q15	544	500	497	497
q16	694	653	652	652
q17	1788	1577	1552	1552
q18	7818	7566	7300	7300
q19	4816	1607	1653	1607
q20	2006	1815	1797	1797
q21	5148	5184	5110	5110
q22	1074	939	1011	939
Total cold run time: 79395 ms
Total hot run time: 59470 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189847 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7ba9185a2e901adf4f3229a63157fb7011b7bb23, data reload: false

query1	970	379	383	379
query2	6520	2074	2039	2039
query3	6699	207	218	207
query4	34047	23583	23332	23332
query5	4325	453	448	448
query6	273	175	167	167
query7	4615	327	314	314
query8	297	238	228	228
query9	9679	2688	2677	2677
query10	490	278	265	265
query11	18083	15261	15095	15095
query12	157	105	102	102
query13	1633	419	410	410
query14	9967	6510	6768	6510
query15	213	169	174	169
query16	7778	480	477	477
query17	1525	553	538	538
query18	1952	290	289	289
query19	191	145	141	141
query20	119	110	107	107
query21	201	102	99	99
query22	4599	4335	4301	4301
query23	34637	33897	33549	33549
query24	12331	2861	2843	2843
query25	706	380	383	380
query26	1899	166	166	166
query27	3134	285	289	285
query28	8492	2486	2471	2471
query29	1179	427	416	416
query30	320	157	152	152
query31	1021	784	798	784
query32	100	58	56	56
query33	756	273	259	259
query34	1043	491	516	491
query35	923	743	703	703
query36	1125	985	1004	985
query37	273	71	72	71
query38	4021	3926	3829	3829
query39	1476	1402	1421	1402
query40	286	96	94	94
query41	50	47	44	44
query42	104	96	97	96
query43	540	484	488	484
query44	1271	785	801	785
query45	182	166	167	166
query46	1131	729	710	710
query47	1929	1852	1816	1816
query48	460	391	375	375
query49	1288	395	387	387
query50	823	420	420	420
query51	7222	6888	7119	6888
query52	97	97	90	90
query53	258	182	183	182
query54	1161	458	461	458
query55	78	75	76	75
query56	256	251	243	243
query57	1208	1110	1112	1110
query58	233	208	208	208
query59	3136	2878	2922	2878
query60	291	263	254	254
query61	128	121	122	121
query62	890	669	676	669
query63	218	191	185	185
query64	5296	729	702	702
query65	3290	3171	3197	3171
query66	1435	344	324	324
query67	16058	15454	15517	15454
query68	4761	552	545	545
query69	449	259	258	258
query70	1211	1180	1084	1084
query71	376	257	254	254
query72	6520	3941	3951	3941
query73	748	347	332	332
query74	10166	8979	8905	8905
query75	3379	2654	2647	2647
query76	2936	1083	1057	1057
query77	387	260	262	260
query78	10629	9889	9586	9586
query79	8627	577	585	577
query80	2136	424	423	423
query81	602	245	237	237
query82	1311	114	112	112
query83	257	137	141	137
query84	296	78	73	73
query85	2255	305	293	293
query86	501	294	292	292
query87	4467	4393	4213	4213
query88	5686	2402	2400	2400
query89	508	292	292	292
query90	2043	183	189	183
query91	179	139	142	139
query92	61	47	50	47
query93	6140	558	552	552
query94	924	284	290	284
query95	361	254	251	251
query96	652	278	287	278
query97	3310	3185	3199	3185
query98	224	203	196	196
query99	1671	1289	1304	1289
Total cold run time: 320861 ms
Total hot run time: 189847 ms

@mymeiyi mymeiyi force-pushed the 3.0-forbid-cluster-key branch from 7ba9185 to e4d8ba3 Compare November 29, 2024 02:23
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40378 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4d8ba3c8137926265b4093cbe9e763c650388f7, data reload: false

------ Round 1 ----------------------------------
q1	17592	7343	7403	7343
q2	2069	162	156	156
q3	10698	1062	1177	1062
q4	10551	744	701	701
q5	7729	2787	2759	2759
q6	239	145	144	144
q7	967	620	601	601
q8	9571	1912	1918	1912
q9	6780	6416	6418	6416
q10	6940	2287	2330	2287
q11	461	266	271	266
q12	412	215	208	208
q13	17770	2952	2982	2952
q14	229	207	209	207
q15	555	523	519	519
q16	680	603	593	593
q17	970	558	584	558
q18	7107	6445	6558	6445
q19	1379	1058	978	978
q20	502	194	218	194
q21	4251	3218	3099	3099
q22	1045	978	979	978
Total cold run time: 108497 ms
Total hot run time: 40378 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7319	7340	7252	7252
q2	322	228	232	228
q3	2886	2881	2826	2826
q4	2005	1765	1781	1765
q5	5627	5683	5713	5683
q6	226	140	145	140
q7	2199	1744	1816	1744
q8	3316	3525	3502	3502
q9	8777	8822	8831	8822
q10	3530	3532	3486	3486
q11	611	524	494	494
q12	805	605	576	576
q13	16498	3157	3095	3095
q14	303	267	262	262
q15	551	520	513	513
q16	704	666	674	666
q17	1823	1627	1591	1591
q18	8049	7825	7568	7568
q19	4183	1626	1618	1618
q20	2048	1872	1821	1821
q21	5232	5283	5238	5238
q22	1162	1054	1013	1013
Total cold run time: 78176 ms
Total hot run time: 59903 ms

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 29, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40382 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9e2266c905ca5c883b7e8a08dfcfd8c200a537b3, data reload: false

------ Round 1 ----------------------------------
q1	17587	7394	7277	7277
q2	2048	181	163	163
q3	10540	1105	1088	1088
q4	10228	729	722	722
q5	7729	2849	2760	2760
q6	232	147	146	146
q7	979	609	614	609
q8	9341	1899	1946	1899
q9	6587	6366	6431	6366
q10	6938	2225	2316	2225
q11	461	279	261	261
q12	400	223	218	218
q13	17760	2970	2956	2956
q14	244	205	207	205
q15	557	515	534	515
q16	669	591	591	591
q17	958	579	525	525
q18	7481	6547	6784	6547
q19	1379	1042	1031	1031
q20	464	198	208	198
q21	4025	3137	3122	3122
q22	1098	967	958	958
Total cold run time: 107705 ms
Total hot run time: 40382 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7252	7253	7193	7193
q2	329	242	223	223
q3	2856	2850	2887	2850
q4	1987	1809	1802	1802
q5	5702	5702	5672	5672
q6	214	135	139	135
q7	2094	1778	1803	1778
q8	3301	3517	3465	3465
q9	8822	8857	8887	8857
q10	3594	3508	3502	3502
q11	624	515	489	489
q12	793	609	637	609
q13	17714	3135	3003	3003
q14	275	241	270	241
q15	553	509	498	498
q16	694	654	656	654
q17	1771	1562	1520	1520
q18	7606	7488	7157	7157
q19	1691	1624	1491	1491
q20	1999	1769	1758	1758
q21	5101	4926	4998	4926
q22	1084	974	985	974
Total cold run time: 76056 ms
Total hot run time: 58797 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189822 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9e2266c905ca5c883b7e8a08dfcfd8c200a537b3, data reload: false

query1	983	367	366	366
query2	6507	2053	2044	2044
query3	6709	217	206	206
query4	33569	23440	23458	23440
query5	4288	436	440	436
query6	259	161	166	161
query7	4621	314	321	314
query8	280	215	218	215
query9	9385	2669	2662	2662
query10	462	265	256	256
query11	17947	15156	15265	15156
query12	148	98	98	98
query13	1639	435	401	401
query14	9170	6302	7058	6302
query15	232	175	177	175
query16	7965	502	481	481
query17	1623	576	565	565
query18	2079	302	290	290
query19	189	146	147	146
query20	110	113	105	105
query21	201	102	101	101
query22	4655	4215	4274	4215
query23	34706	34039	33911	33911
query24	12127	2937	2861	2861
query25	725	374	375	374
query26	1872	159	163	159
query27	3095	293	288	288
query28	8520	2487	2465	2465
query29	1076	437	414	414
query30	328	158	154	154
query31	1046	823	790	790
query32	92	57	53	53
query33	755	275	265	265
query34	1038	492	512	492
query35	849	724	693	693
query36	1093	926	934	926
query37	276	71	72	71
query38	3979	3819	3839	3819
query39	1492	1421	1416	1416
query40	285	96	97	96
query41	49	46	47	46
query42	108	105	94	94
query43	522	479	490	479
query44	1219	788	779	779
query45	178	160	165	160
query46	1148	696	716	696
query47	1917	1816	1819	1816
query48	454	359	366	359
query49	1274	370	386	370
query50	813	414	415	414
query51	7106	6956	7113	6956
query52	105	90	91	90
query53	258	185	188	185
query54	1270	462	467	462
query55	76	76	77	76
query56	265	263	244	244
query57	1234	1104	1113	1104
query58	240	216	207	207
query59	3150	2885	2983	2885
query60	298	262	263	262
query61	130	125	123	123
query62	876	759	664	664
query63	218	192	185	185
query64	5304	744	693	693
query65	3233	3213	3195	3195
query66	1435	319	317	317
query67	16132	15395	15081	15081
query68	4595	546	558	546
query69	433	260	259	259
query70	1121	1096	1134	1096
query71	418	261	254	254
query72	6817	4075	3930	3930
query73	776	334	334	334
query74	10366	8990	8966	8966
query75	3413	2778	2645	2645
query76	2890	1083	1079	1079
query77	393	266	273	266
query78	10695	9685	9649	9649
query79	8912	600	598	598
query80	2074	425	413	413
query81	567	237	238	237
query82	1418	120	111	111
query83	258	139	139	139
query84	289	79	78	78
query85	2257	307	289	289
query86	487	308	302	302
query87	4527	4247	4251	4247
query88	5696	2409	2407	2407
query89	548	292	299	292
query90	2079	183	181	181
query91	175	144	142	142
query92	65	48	49	48
query93	6118	557	549	549
query94	905	299	287	287
query95	345	250	254	250
query96	629	285	282	282
query97	3321	3128	3142	3128
query98	209	196	190	190
query99	1716	1304	1325	1304
Total cold run time: 319899 ms
Total hot run time: 189822 ms

dataroaring
dataroaring approved these changes Nov 29, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring merged commit 8e881ce into apache:branch-3.0 Dec 2, 2024
19 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mymeiyi @doris-robot @dataroaring