Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](cluster key) fix cluster key schema change #45203

Merged
merged 1 commit into from
Dec 12, 2024
Merged

[fix](cluster key) fix cluster key schema change #45203

merged 1 commit into from
Dec 12, 2024

Conversation

mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Dec 9, 2024 •
edited by zhannngchen
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
for schema change: add a key in the tail of all keys, may get:

F20241209 14:52:09.513403 3166619 segment_writer.cpp:1309] Check failed: key.compare(last_key) >= 0 key is not sorted! current key: ^A^B06olTqiL2rEn48nTpmP8, last key: ^B^@^OЭ^BxsmIHehvUnv
*** Check failure stack trace: ***
    @     0x563674f2dcc6  google::LogMessage::SendToLog()
    @     0x563674f2a710  google::LogMessage::Flush()
    @     0x563674f2e509  google::LogMessageFatal::~LogMessageFatal()
    @     0x56366a8a72c0  doris::segment_v2::SegmentWriter::_generate_short_key_index()
    @     0x56366a8a6032  doris::segment_v2::SegmentWriter::append_block()
    @     0x56366a7624c9  doris::SegmentCreator::add_block()
    @     0x56366a731d35  doris::BaseBetaRowsetWriter::add_block()
    @     0x56366a94a233  doris::VSchemaChangeDirectly::_inner_process()
    @     0x56366a95d558  doris::SchemaChange::process()
    @     0x56366a9511e5  doris::SchemaChangeJob::_convert_historical_rowsets()
    @     0x56366a94e5e2  doris::SchemaChangeJob::_do_process_alter_tablet()
    @     0x56366a94cf22  doris::SchemaChangeJob::process_alter_tablet()
    @     0x563669e9e857  doris::alter_tablet_callback()

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024

clang-tidy review says "All clean, LGTM! 👍"

@dataroaring
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39823 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bf758d3c2a10d5f77f3d3c04a30b3fa2df9cfdfe, data reload: false

------ Round 1 ----------------------------------
q1	17553	7725	7346	7346
q2	2047	178	165	165
q3	10608	1067	1161	1067
q4	10585	792	725	725
q5	7621	2720	2733	2720
q6	235	148	147	147
q7	1001	626	601	601
q8	9236	1831	1918	1831
q9	6602	6448	6443	6443
q10	7002	2354	2311	2311
q11	463	254	256	254
q12	414	219	221	219
q13	17767	3023	2998	2998
q14	245	209	218	209
q15	591	527	530	527
q16	658	595	575	575
q17	973	531	636	531
q18	7188	6600	6729	6600
q19	1332	1003	910	910
q20	470	183	179	179
q21	4030	3271	3148	3148
q22	363	317	329	317
Total cold run time: 106984 ms
Total hot run time: 39823 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7286	7229	7292	7229
q2	321	227	225	225
q3	2918	2805	2944	2805
q4	2036	1815	1859	1815
q5	5770	5690	5653	5653
q6	222	137	142	137
q7	2251	1783	1785	1783
q8	3360	3519	3559	3519
q9	9045	9110	8990	8990
q10	3612	3551	3548	3548
q11	610	498	513	498
q12	832	618	588	588
q13	10958	3222	3199	3199
q14	300	266	268	266
q15	584	530	521	521
q16	710	670	662	662
q17	1826	1649	1607	1607
q18	8452	7893	7442	7442
q19	1724	1595	1647	1595
q20	2117	1855	1849	1849
q21	5645	5591	5429	5429
q22	652	558	584	558
Total cold run time: 71231 ms
Total hot run time: 59918 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.80% (10103/26037)
Line Coverage: 29.69% (84689/285199)
Region Coverage: 28.77% (43484/151148)
Branch Coverage: 25.33% (22097/87230)
Coverage Report: http://coverage.selectdb-in.cc/coverage/bf758d3c2a10d5f77f3d3c04a30b3fa2df9cfdfe_bf758d3c2a10d5f77f3d3c04a30b3fa2df9cfdfe/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 197234 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bf758d3c2a10d5f77f3d3c04a30b3fa2df9cfdfe, data reload: false

query1	1236	969	927	927
query2	6228	2083	2085	2083
query3	11111	4376	4470	4376
query4	67271	28887	23664	23664
query5	4889	473	466	466
query6	427	190	189	189
query7	5660	312	309	309
query8	328	243	251	243
query9	9421	2751	2742	2742
query10	480	250	251	250
query11	17626	15350	15963	15350
query12	163	109	107	107
query13	1568	445	454	445
query14	10330	6890	7634	6890
query15	220	193	200	193
query16	7255	496	457	457
query17	1209	567	600	567
query18	1800	291	309	291
query19	239	178	149	149
query20	115	110	113	110
query21	210	106	101	101
query22	4727	4536	4429	4429
query23	34989	34286	34628	34286
query24	5789	2580	2508	2508
query25	495	391	389	389
query26	636	149	152	149
query27	1906	298	291	291
query28	4264	2496	2462	2462
query29	681	437	423	423
query30	207	159	154	154
query31	987	847	877	847
query32	66	56	57	56
query33	470	304	280	280
query34	923	516	547	516
query35	891	766	771	766
query36	1067	969	1006	969
query37	120	76	74	74
query38	4651	4382	4481	4382
query39	1562	1486	1483	1483
query40	198	104	108	104
query41	46	43	44	43
query42	121	105	107	105
query43	550	506	496	496
query44	1199	832	853	832
query45	190	168	168	168
query46	1196	735	724	724
query47	2031	1924	1905	1905
query48	419	332	326	326
query49	747	395	393	393
query50	896	401	402	401
query51	7484	7226	7014	7014
query52	104	87	89	87
query53	255	189	178	178
query54	513	398	390	390
query55	83	72	77	72
query56	245	243	231	231
query57	1247	1124	1114	1114
query58	231	210	233	210
query59	3345	3156	3014	3014
query60	276	249	249	249
query61	107	109	109	109
query62	783	671	685	671
query63	211	185	189	185
query64	1397	711	629	629
query65	3355	3224	3216	3216
query66	706	300	305	300
query67	15852	15517	15474	15474
query68	4261	553	559	553
query69	437	257	249	249
query70	1194	1149	1143	1143
query71	378	259	240	240
query72	6503	4068	4150	4068
query73	768	358	361	358
query74	10234	9071	9252	9071
query75	3433	2658	2691	2658
query76	1803	1109	1161	1109
query77	457	270	269	269
query78	10371	9496	9410	9410
query79	1430	601	588	588
query80	894	442	440	440
query81	484	234	227	227
query82	1307	122	117	117
query83	256	147	141	141
query84	284	71	71	71
query85	891	348	305	305
query86	342	291	272	272
query87	4733	4551	4605	4551
query88	3773	2203	2180	2180
query89	412	291	291	291
query90	2036	194	188	188
query91	134	99	103	99
query92	65	51	53	51
query93	1925	544	544	544
query94	864	307	296	296
query95	369	254	255	254
query96	636	282	277	277
query97	2845	2685	2722	2685
query98	219	195	189	189
query99	1601	1327	1338	1327
Total cold run time: 321343 ms
Total hot run time: 197234 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.31 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bf758d3c2a10d5f77f3d3c04a30b3fa2df9cfdfe, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.04
query3	0.23	0.07	0.07
query4	1.62	0.10	0.11
query5	0.42	0.42	0.42
query6	1.16	0.66	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.50	0.50
query10	0.55	0.56	0.58
query11	0.15	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.62	0.59
query14	2.75	2.80	2.74
query15	0.92	0.83	0.82
query16	0.39	0.37	0.38
query17	1.08	1.05	1.07
query18	0.22	0.21	0.21
query19	1.94	1.88	2.02
query20	0.02	0.01	0.01
query21	15.38	0.60	0.58
query22	2.58	2.31	1.84
query23	17.16	0.98	0.86
query24	3.29	0.32	0.82
query25	0.30	0.18	0.05
query26	0.37	0.13	0.13
query27	0.04	0.04	0.04
query28	11.46	1.09	1.08
query29	12.57	3.27	3.27
query30	0.26	0.06	0.07
query31	2.85	0.39	0.38
query32	3.28	0.47	0.46
query33	2.98	3.01	3.08
query34	16.70	4.53	4.57
query35	4.55	4.61	4.61
query36	0.68	0.48	0.48
query37	0.10	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.9 s
Total hot run time: 32.31 s

@mymeiyi mymeiyi force-pushed the fix-ck-schema-change branch from b66dc6e to cc03363 Compare December 10, 2024 03:35
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

1 similar comment
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@mymeiyi mymeiyi force-pushed the fix-ck-schema-change branch from cc03363 to 72c9f64 Compare December 10, 2024 04:05
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Dec 10, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 41192 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 72c9f643c9f954ae64049c49688a5c6c43928073, data reload: false

------ Round 1 ----------------------------------
q1	17731	7639	7554	7554
q2	2046	173	165	165
q3	10637	1193	1229	1193
q4	10263	828	802	802
q5	7617	2856	2821	2821
q6	258	153	153	153
q7	1073	649	613	613
q8	9241	1979	1995	1979
q9	6964	6828	6748	6748
q10	7002	2364	2340	2340
q11	461	252	265	252
q12	440	227	222	222
q13	17771	3015	3041	3015
q14	246	204	206	204
q15	569	519	543	519
q16	673	603	595	595
q17	1029	576	550	550
q18	7627	6727	6602	6602
q19	1362	1097	1080	1080
q20	501	173	182	173
q21	4177	3413	3298	3298
q22	391	314	316	314
Total cold run time: 108079 ms
Total hot run time: 41192 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7659	7679	7722	7679
q2	342	224	240	224
q3	3047	2847	2885	2847
q4	2038	1755	1732	1732
q5	5482	5506	5508	5506
q6	242	140	140	140
q7	2221	1715	1706	1706
q8	3460	3623	3694	3623
q9	8814	8752	8836	8752
q10	3590	3486	3534	3486
q11	595	489	502	489
q12	797	570	606	570
q13	11794	3024	3021	3021
q14	293	263	273	263
q15	583	522	516	516
q16	673	641	650	641
q17	1983	1701	1679	1679
q18	7852	7445	7386	7386
q19	1838	1695	1543	1543
q20	2126	1820	1810	1810
q21	5533	5427	5341	5341
q22	640	582	570	570
Total cold run time: 71602 ms
Total hot run time: 59524 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.81% (10102/26031)
Line Coverage: 29.70% (84685/285165)
Region Coverage: 28.77% (43494/151153)
Branch Coverage: 25.33% (22091/87224)
Coverage Report: http://coverage.selectdb-in.cc/coverage/72c9f643c9f954ae64049c49688a5c6c43928073_72c9f643c9f954ae64049c49688a5c6c43928073/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 191649 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 72c9f643c9f954ae64049c49688a5c6c43928073, data reload: false

query1	968	385	380	380
query2	6523	2118	2044	2044
query3	6697	219	211	211
query4	33665	23534	23509	23509
query5	4366	488	472	472
query6	293	187	176	176
query7	4620	302	309	302
query8	291	240	234	234
query9	9883	2743	2750	2743
query10	474	261	265	261
query11	18089	15258	15440	15258
query12	164	103	104	103
query13	1655	424	435	424
query14	9776	6705	8068	6705
query15	285	194	204	194
query16	8490	462	478	462
query17	1813	588	580	580
query18	2136	307	305	305
query19	365	193	158	158
query20	117	107	112	107
query21	214	100	102	100
query22	4589	4408	4192	4192
query23	34930	34181	34458	34181
query24	11501	2512	2599	2512
query25	670	376	383	376
query26	1851	162	150	150
query27	2885	286	284	284
query28	7957	2450	2471	2450
query29	1037	429	418	418
query30	300	150	147	147
query31	1072	829	843	829
query32	98	57	60	57
query33	772	297	297	297
query34	1004	526	565	526
query35	882	742	746	742
query36	1121	956	981	956
query37	293	83	77	77
query38	4602	4282	4207	4207
query39	1595	1407	1427	1407
query40	290	101	99	99
query41	47	44	45	44
query42	111	96	97	96
query43	555	489	501	489
query44	1283	828	822	822
query45	191	159	163	159
query46	1127	716	721	716
query47	1903	1814	1806	1806
query48	415	311	311	311
query49	1287	401	390	390
query50	806	403	383	383
query51	7215	7138	7108	7108
query52	101	89	89	89
query53	263	189	187	187
query54	1191	396	411	396
query55	89	77	79	77
query56	257	245	249	245
query57	1224	1125	1104	1104
query58	229	209	219	209
query59	3232	3211	3251	3211
query60	276	243	251	243
query61	111	104	111	104
query62	895	675	676	675
query63	218	186	195	186
query64	5084	657	615	615
query65	3339	3200	3261	3200
query66	1404	311	314	311
query67	15840	15644	15645	15644
query68	4875	561	551	551
query69	407	259	269	259
query70	1201	1179	1158	1158
query71	326	249	260	249
query72	6340	4090	4213	4090
query73	844	362	361	361
query74	10457	9109	8930	8930
query75	3482	2674	2692	2674
query76	2974	1122	1089	1089
query77	503	283	294	283
query78	10406	9488	9390	9390
query79	2457	607	620	607
query80	1134	429	424	424
query81	533	241	240	240
query82	836	116	116	116
query83	250	152	148	148
query84	231	75	70	70
query85	1322	305	306	305
query86	438	310	295	295
query87	4810	4519	4610	4519
query88	3999	2221	2200	2200
query89	413	306	305	305
query90	2182	193	192	192
query91	137	105	110	105
query92	69	54	49	49
query93	1505	548	554	548
query94	1092	284	291	284
query95	371	251	254	251
query96	624	318	283	283
query97	2820	2701	2728	2701
query98	217	195	202	195
query99	1552	1345	1323	1323
Total cold run time: 305833 ms
Total hot run time: 191649 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 72c9f643c9f954ae64049c49688a5c6c43928073, data reload: false

query1	0.04	0.03	0.03
query2	0.10	0.03	0.03
query3	0.23	0.08	0.08
query4	1.60	0.10	0.11
query5	0.41	0.43	0.40
query6	1.16	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.52
query10	0.53	0.57	0.54
query11	0.14	0.10	0.11
query12	0.14	0.12	0.12
query13	0.61	0.62	0.59
query14	2.86	2.75	2.88
query15	0.89	0.82	0.83
query16	0.38	0.39	0.39
query17	1.07	1.00	1.05
query18	0.23	0.21	0.22
query19	1.87	1.74	1.97
query20	0.02	0.01	0.02
query21	15.37	0.60	0.59
query22	2.69	2.50	1.25
query23	17.33	0.89	0.81
query24	3.02	1.51	1.68
query25	0.28	0.32	0.12
query26	0.44	0.15	0.14
query27	0.04	0.04	0.04
query28	9.72	1.11	1.08
query29	12.58	3.41	3.40
query30	0.25	0.06	0.07
query31	2.86	0.36	0.38
query32	3.31	0.46	0.45
query33	3.01	3.02	3.00
query34	16.67	4.42	4.43
query35	4.53	4.47	4.41
query36	0.67	0.48	0.49
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.17 s
Total hot run time: 32.54 s

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@mymeiyi mymeiyi force-pushed the fix-ck-schema-change branch from f7ae8bd to 0da45c7 Compare December 11, 2024 06:31
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Dec 11, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.75% (10103/26069)
Line Coverage: 29.69% (84728/285369)
Region Coverage: 28.77% (43507/151230)
Branch Coverage: 25.32% (22097/87270)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0da45c74fb564123aa50fd7a3d4c539006b41ec2_0da45c74fb564123aa50fd7a3d4c539006b41ec2/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39696 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0da45c74fb564123aa50fd7a3d4c539006b41ec2, data reload: false

------ Round 1 ----------------------------------
q1	17573	7395	7205	7205
q2	2043	170	172	170
q3	10653	1059	1200	1059
q4	10555	778	725	725
q5	7590	2740	2651	2651
q6	238	148	148	148
q7	1000	625	601	601
q8	9263	1855	1906	1855
q9	6731	6525	6528	6525
q10	7026	2250	2316	2250
q11	473	257	267	257
q12	435	229	227	227
q13	17795	2993	2971	2971
q14	244	212	213	212
q15	560	535	528	528
q16	671	606	582	582
q17	981	573	531	531
q18	7196	6719	6663	6663
q19	1375	937	905	905
q20	471	193	186	186
q21	4048	3206	3134	3134
q22	381	315	311	311
Total cold run time: 107302 ms
Total hot run time: 39696 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7239	7210	7206	7206
q2	329	232	227	227
q3	2847	2768	2913	2768
q4	2076	1809	1818	1809
q5	5695	5656	5630	5630
q6	230	142	143	142
q7	2230	1823	1803	1803
q8	3416	3556	3490	3490
q9	8933	9113	9045	9045
q10	3590	3526	3526	3526
q11	597	518	502	502
q12	835	610	637	610
q13	12664	3276	3249	3249
q14	304	269	270	269
q15	574	526	510	510
q16	699	669	641	641
q17	1845	1636	1602	1602
q18	8289	7703	7650	7650
q19	1786	1521	1598	1521
q20	2104	1858	1862	1858
q21	5645	5473	5396	5396
q22	674	567	614	567
Total cold run time: 72601 ms
Total hot run time: 60021 ms

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Dec 11, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40426 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3ee006acab5e0740488ed22abb2035395947b83b, data reload: false

------ Round 1 ----------------------------------
q1	17594	7454	7272	7272
q2	2046	178	181	178
q3	10534	1170	1161	1161
q4	10226	785	728	728
q5	7617	2789	2731	2731
q6	236	150	150	150
q7	987	620	613	613
q8	9265	1920	1907	1907
q9	6782	6537	6560	6537
q10	7099	2324	2341	2324
q11	475	259	261	259
q12	523	232	227	227
q13	17820	2983	3025	2983
q14	237	208	214	208
q15	562	534	507	507
q16	674	582	581	581
q17	982	583	583	583
q18	7537	6755	6744	6744
q19	1341	1028	993	993
q20	496	189	184	184
q21	3979	3312	3241	3241
q22	384	315	317	315
Total cold run time: 107396 ms
Total hot run time: 40426 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7294	7228	7198	7198
q2	330	236	243	236
q3	3007	2736	2823	2736
q4	1999	1761	1709	1709
q5	5411	5428	5449	5428
q6	219	138	143	138
q7	2130	1736	1748	1736
q8	3277	3452	3415	3415
q9	8696	8691	8659	8659
q10	3510	3409	3456	3409
q11	590	489	509	489
q12	786	570	574	570
q13	8891	3024	3003	3003
q14	284	250	261	250
q15	568	515	520	515
q16	674	629	628	628
q17	1814	1592	1561	1561
q18	7907	7555	7459	7459
q19	1698	1495	1549	1495
q20	2114	1823	1811	1811
q21	5517	5299	5259	5259
q22	648	567	552	552
Total cold run time: 67364 ms
Total hot run time: 58256 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.75% (10102/26069)
Line Coverage: 29.69% (84733/285378)
Region Coverage: 28.76% (43489/151233)
Branch Coverage: 25.31% (22092/87274)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3ee006acab5e0740488ed22abb2035395947b83b_3ee006acab5e0740488ed22abb2035395947b83b/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 189296 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3ee006acab5e0740488ed22abb2035395947b83b, data reload: false

query1	972	391	389	389
query2	6516	2110	2077	2077
query3	6566	222	220	220
query4	33961	23568	23354	23354
query5	4337	483	464	464
query6	272	194	182	182
query7	4586	301	311	301
query8	307	241	230	230
query9	9442	2740	2721	2721
query10	457	256	273	256
query11	18071	15175	15218	15175
query12	148	98	100	98
query13	1645	431	433	431
query14	9308	7064	7601	7064
query15	285	187	184	184
query16	7406	424	443	424
query17	1722	552	543	543
query18	1922	286	321	286
query19	351	152	147	147
query20	119	113	116	113
query21	200	102	110	102
query22	4491	4369	4329	4329
query23	34513	33870	34210	33870
query24	10395	2478	2517	2478
query25	626	376	383	376
query26	1417	151	144	144
query27	2669	279	282	279
query28	7489	2454	2433	2433
query29	868	426	395	395
query30	290	154	151	151
query31	1063	821	797	797
query32	93	55	56	55
query33	750	298	291	291
query34	934	511	520	511
query35	896	746	774	746
query36	1099	944	912	912
query37	129	77	72	72
query38	4455	4216	4217	4216
query39	1442	1427	1534	1427
query40	263	97	97	97
query41	46	43	44	43
query42	111	100	99	99
query43	519	474	487	474
query44	1257	833	816	816
query45	183	161	162	161
query46	1165	695	682	682
query47	1974	1833	1855	1833
query48	418	307	329	307
query49	1149	422	387	387
query50	806	376	377	376
query51	7310	7095	7133	7095
query52	98	90	87	87
query53	255	188	185	185
query54	1027	398	409	398
query55	84	77	79	77
query56	261	230	237	230
query57	1253	1107	1140	1107
query58	232	206	215	206
query59	3113	3037	3021	3021
query60	271	236	267	236
query61	110	110	108	108
query62	892	662	693	662
query63	214	185	182	182
query64	4521	814	781	781
query65	3246	3163	3214	3163
query66	1065	316	303	303
query67	16001	15651	15578	15578
query68	4795	537	543	537
query69	428	253	249	249
query70	1212	1137	1158	1137
query71	347	249	251	249
query72	6462	4044	2605	2605
query73	768	349	361	349
query74	10268	8857	8955	8857
query75	3425	2647	2652	2647
query76	2702	1136	1058	1058
query77	440	283	293	283
query78	10279	9558	9378	9378
query79	1303	599	588	588
query80	1093	431	441	431
query81	524	228	224	224
query82	1167	126	120	120
query83	238	151	139	139
query84	233	71	71	71
query85	1318	299	299	299
query86	386	299	293	293
query87	4676	4548	4513	4513
query88	3582	2241	2193	2193
query89	412	298	296	296
query90	2041	188	188	188
query91	140	106	103	103
query92	58	53	51	51
query93	1080	528	523	523
query94	1053	292	292	292
query95	354	248	260	248
query96	614	281	286	281
query97	2839	2630	2651	2630
query98	222	201	197	197
query99	1542	1330	1335	1330
Total cold run time: 296057 ms
Total hot run time: 189296 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.65 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3ee006acab5e0740488ed22abb2035395947b83b, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.08	0.06
query4	1.62	0.10	0.10
query5	0.45	0.40	0.40
query6	1.17	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.49
query10	0.57	0.58	0.55
query11	0.15	0.12	0.11
query12	0.14	0.11	0.11
query13	0.61	0.60	0.61
query14	2.87	2.88	2.86
query15	0.89	0.83	0.83
query16	0.40	0.36	0.37
query17	1.00	1.07	1.03
query18	0.22	0.21	0.21
query19	1.89	1.83	2.07
query20	0.01	0.00	0.02
query21	15.36	0.60	0.58
query22	2.49	2.21	2.02
query23	17.10	0.84	0.74
query24	2.80	2.33	0.95
query25	0.24	0.07	0.05
query26	0.49	0.14	0.13
query27	0.05	0.05	0.05
query28	9.94	1.11	1.09
query29	12.56	3.20	3.25
query30	0.25	0.06	0.06
query31	2.85	0.38	0.38
query32	3.30	0.46	0.46
query33	3.00	3.02	3.05
query34	16.93	4.43	4.42
query35	4.51	4.44	4.46
query36	0.70	0.47	0.48
query37	0.08	0.06	0.05
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.98 s
Total hot run time: 32.65 s

@mymeiyi mymeiyi force-pushed the fix-ck-schema-change branch from 3ee006a to d875a50 Compare December 11, 2024 08:32
@mymeiyi
Copy link
Contributor Author

mymeiyi commented Dec 11, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 39969 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d875a5027a3c5d3815f4008600fed7c32b63a47f, data reload: false

------ Round 1 ----------------------------------
q1	17614	8327	7289	7289
q2	2042	168	168	168
q3	10558	1133	1232	1133
q4	10216	790	787	787
q5	7630	2753	2683	2683
q6	238	152	151	151
q7	1016	621	611	611
q8	9241	1834	1919	1834
q9	6724	6592	6511	6511
q10	7037	2282	2339	2282
q11	469	259	264	259
q12	408	229	238	229
q13	17813	2977	3000	2977
q14	236	210	212	210
q15	564	512	519	512
q16	653	597	580	580
q17	1036	524	607	524
q18	7438	6758	6713	6713
q19	1349	1027	969	969
q20	461	184	186	184
q21	4019	3276	3051	3051
q22	371	315	312	312
Total cold run time: 107133 ms
Total hot run time: 39969 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7229	7250	7203	7203
q2	332	233	233	233
q3	2921	2902	3057	2902
q4	2132	1792	1776	1776
q5	5589	5723	5662	5662
q6	219	138	137	137
q7	2191	1782	1837	1782
q8	3466	3570	3526	3526
q9	9018	8984	9103	8984
q10	3598	3588	3556	3556
q11	607	513	529	513
q12	823	654	648	648
q13	12105	3161	3166	3161
q14	283	251	256	251
q15	562	510	503	503
q16	679	615	614	614
q17	1813	1577	1580	1577
q18	7878	7411	7441	7411
q19	1700	1525	1514	1514
q20	2080	1812	1802	1802
q21	5419	5356	5264	5264
q22	636	539	572	539
Total cold run time: 71280 ms
Total hot run time: 59558 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190557 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d875a5027a3c5d3815f4008600fed7c32b63a47f, data reload: false

query1	972	377	372	372
query2	6512	2062	2055	2055
query3	6708	210	221	210
query4	34007	23466	23876	23466
query5	4386	485	466	466
query6	279	178	180	178
query7	4632	304	304	304
query8	332	249	239	239
query9	9654	2728	2712	2712
query10	487	261	251	251
query11	17905	15204	15017	15017
query12	168	100	104	100
query13	1650	407	410	407
query14	11101	6983	7656	6983
query15	302	179	192	179
query16	8027	449	427	427
query17	1760	553	549	549
query18	2135	290	290	290
query19	352	155	146	146
query20	140	111	113	111
query21	203	110	102	102
query22	4529	4383	4228	4228
query23	34398	33819	33917	33819
query24	10328	2489	2404	2404
query25	626	377	391	377
query26	1234	146	149	146
query27	2765	275	273	273
query28	7870	2462	2437	2437
query29	826	407	399	399
query30	292	152	156	152
query31	1026	798	810	798
query32	96	57	59	57
query33	772	296	282	282
query34	980	508	504	504
query35	877	741	745	741
query36	1109	946	949	946
query37	143	76	80	76
query38	4306	4176	4165	4165
query39	1464	1438	1402	1402
query40	207	108	103	103
query41	50	46	48	46
query42	112	101	99	99
query43	546	496	512	496
query44	1219	826	819	819
query45	194	168	173	168
query46	1167	704	728	704
query47	1949	1838	1874	1838
query48	427	322	333	322
query49	1167	406	403	403
query50	825	379	385	379
query51	7118	7098	6985	6985
query52	105	93	86	86
query53	249	174	177	174
query54	1114	400	403	400
query55	78	80	79	79
query56	254	235	229	229
query57	1251	1082	1080	1080
query58	224	228	211	211
query59	3272	3166	3180	3166
query60	270	254	243	243
query61	111	109	112	109
query62	892	683	692	683
query63	212	185	185	185
query64	4160	651	640	640
query65	3271	3174	3198	3174
query66	856	309	319	309
query67	16248	15739	15772	15739
query68	4893	539	589	539
query69	439	247	259	247
query70	1202	1118	1055	1055
query71	318	246	247	246
query72	6456	4090	4059	4059
query73	753	363	365	363
query74	10387	8965	8916	8916
query75	3424	2623	2641	2623
query76	2906	1034	1044	1034
query77	462	276	300	276
query78	10435	9522	9397	9397
query79	2431	591	600	591
query80	1223	427	426	426
query81	553	237	232	232
query82	745	126	121	121
query83	242	148	143	143
query84	235	68	73	68
query85	1345	300	304	300
query86	427	305	290	290
query87	4736	4632	4640	4632
query88	3924	2216	2184	2184
query89	404	297	294	294
query90	1978	182	184	182
query91	137	105	111	105
query92	72	54	53	53
query93	1704	525	531	525
query94	807	289	275	275
query95	387	252	244	244
query96	617	278	276	276
query97	2816	2749	2712	2712
query98	216	211	199	199
query99	1563	1313	1311	1311
Total cold run time: 301138 ms
Total hot run time: 190557 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.97 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d875a5027a3c5d3815f4008600fed7c32b63a47f, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.40	0.39	0.40
query6	1.14	0.65	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.53	0.50
query10	0.54	0.58	0.55
query11	0.15	0.11	0.11
query12	0.13	0.11	0.10
query13	0.62	0.59	0.59
query14	2.79	2.84	2.84
query15	0.91	0.83	0.82
query16	0.37	0.38	0.38
query17	1.08	1.05	1.05
query18	0.23	0.21	0.22
query19	1.98	1.76	2.02
query20	0.02	0.01	0.02
query21	15.36	0.60	0.58
query22	2.48	2.03	2.03
query23	17.09	1.00	0.76
query24	3.25	1.55	1.12
query25	0.26	0.14	0.07
query26	0.57	0.14	0.14
query27	0.05	0.05	0.04
query28	10.10	1.10	1.08
query29	12.55	3.27	3.25
query30	0.24	0.06	0.06
query31	2.88	0.38	0.39
query32	3.27	0.47	0.47
query33	2.95	3.02	3.01
query34	16.88	4.46	4.50
query35	4.50	4.54	4.50
query36	0.68	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.54 s
Total hot run time: 32.97 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.75% (10102/26069)
Line Coverage: 29.69% (84733/285378)
Region Coverage: 28.76% (43495/151233)
Branch Coverage: 25.31% (22093/87274)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d875a5027a3c5d3815f4008600fed7c32b63a47f_d875a5027a3c5d3815f4008600fed7c32b63a47f/report/index.html

dataroaring
dataroaring approved these changes Dec 12, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

zhannngchen
zhannngchen approved these changes Dec 12, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhannngchen zhannngchen merged commit e2c24e3 into apache:master Dec 12, 2024
25 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dataroaring dataroaring dataroaring approved these changes

@zhannngchen zhannngchen zhannngchen approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mymeiyi @doris-robot @dataroaring @zhannngchen