Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[chore](conf) Set enable_advance_next_id=true by default #44790

Merged
merged 5 commits into from
Jan 3, 2025
Merged

[chore](conf) Set enable_advance_next_id=true by default #44790

merged 5 commits into from
Jan 3, 2025

Conversation

gavinchou
Copy link
Contributor

Bind the generation of next_id to physical time to ensure its generation is monotonically increasing, even if we directly overwrite the FE metadata and then restart. This way, it supports a lossy cluster rollback in scenarios where major version upgrades are incompatible.

What problem does this PR solve?

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • [x No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@gavinchou
Copy link
Contributor Author

run buildall

zhannngchen
zhannngchen previously approved these changes Nov 29, 2024
View reviewed changes
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

dataroaring
dataroaring previously approved these changes Nov 29, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@gavinchou
Copy link
Contributor Author

run feut

@gavinchou gavinchou force-pushed the gavin-advance-id-by-default branch from 1a5eaf1 to 83249b0 Compare December 31, 2024 09:12
@gavinchou
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32466 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 83249b0079280144085e8e9c5a27165567e0b8fd, data reload: false

------ Round 1 ----------------------------------
q1	17633	6070	6073	6070
q2	2048	301	179	179
q3	10533	1257	701	701
q4	10218	866	439	439
q5	7623	2179	1920	1920
q6	212	182	147	147
q7	891	750	606	606
q8	9236	1286	1119	1119
q9	5354	5001	4914	4914
q10	6766	2332	1871	1871
q11	477	272	262	262
q12	339	354	214	214
q13	17784	3604	3022	3022
q14	227	224	212	212
q15	556	509	496	496
q16	634	608	591	591
q17	565	837	327	327
q18	7123	6386	6524	6386
q19	1816	974	550	550
q20	303	324	187	187
q21	2859	2181	1946	1946
q22	358	320	307	307
Total cold run time: 103555 ms
Total hot run time: 32466 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6219	6198	6230	6198
q2	232	332	228	228
q3	2196	2602	2292	2292
q4	1427	1796	1431	1431
q5	4362	4740	4804	4740
q6	185	180	146	146
q7	2066	1984	1875	1875
q8	2623	2787	2667	2667
q9	7290	7165	7310	7165
q10	3034	3315	2800	2800
q11	607	508	513	508
q12	661	753	619	619
q13	3346	3677	3157	3157
q14	279	301	278	278
q15	557	514	510	510
q16	644	684	649	649
q17	1221	1733	1229	1229
q18	7829	7554	6973	6973
q19	780	1098	1041	1041
q20	1929	1966	1814	1814
q21	5439	5013	4856	4856
q22	606	608	592	592
Total cold run time: 53532 ms
Total hot run time: 51768 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190868 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 83249b0079280144085e8e9c5a27165567e0b8fd, data reload: false

query1	968	389	406	389
query2	6516	2412	2439	2412
query3	6706	213	214	213
query4	33840	23916	23374	23374
query5	4374	636	493	493
query6	278	201	195	195
query7	4630	501	306	306
query8	289	241	234	234
query9	9365	2722	2723	2722
query10	474	319	257	257
query11	18034	15871	15173	15173
query12	163	106	104	104
query13	1649	536	451	451
query14	10992	7024	7662	7024
query15	238	196	194	194
query16	8132	622	442	442
query17	1601	772	608	608
query18	2117	407	313	313
query19	241	184	167	167
query20	123	114	116	114
query21	207	127	112	112
query22	4347	4426	4257	4257
query23	34408	33330	33363	33330
query24	6441	2309	2286	2286
query25	496	463	385	385
query26	1196	279	163	163
query27	1982	481	335	335
query28	5327	2441	2396	2396
query29	763	550	419	419
query30	228	196	153	153
query31	987	913	803	803
query32	87	60	58	58
query33	513	366	312	312
query34	767	852	519	519
query35	804	853	771	771
query36	1062	1030	949	949
query37	121	101	77	77
query38	4154	4145	4009	4009
query39	1513	1439	1471	1439
query40	207	129	106	106
query41	50	47	44	44
query42	121	99	102	99
query43	527	545	501	501
query44	1321	806	802	802
query45	180	182	174	174
query46	907	1046	676	676
query47	1897	1911	1860	1860
query48	385	408	322	322
query49	785	490	399	399
query50	677	647	398	398
query51	7322	7136	7191	7136
query52	103	103	92	92
query53	229	271	189	189
query54	482	500	412	412
query55	82	77	82	77
query56	259	273	249	249
query57	1233	1167	1120	1120
query58	240	254	244	244
query59	3108	3181	3056	3056
query60	269	266	262	262
query61	115	109	146	109
query62	872	803	740	740
query63	234	202	202	202
query64	4418	1005	688	688
query65	3335	3238	3266	3238
query66	1060	426	323	323
query67	15901	15803	15534	15534
query68	9104	777	520	520
query69	477	295	256	256
query70	1292	1191	1168	1168
query71	427	295	257	257
query72	5916	3934	3865	3865
query73	683	797	376	376
query74	10071	9065	8868	8868
query75	4610	3204	2656	2656
query76	5100	1199	803	803
query77	925	382	284	284
query78	11039	10758	9467	9467
query79	6752	999	591	591
query80	700	554	436	436
query81	489	291	244	244
query82	665	166	127	127
query83	203	170	152	152
query84	285	101	77	77
query85	780	361	307	307
query86	355	333	308	308
query87	4667	4597	4534	4534
query88	5015	2242	2213	2213
query89	449	336	308	308
query90	1964	192	191	191
query91	139	141	104	104
query92	73	59	56	56
query93	4127	889	528	528
query94	665	404	310	310
query95	338	279	259	259
query96	495	651	276	276
query97	2745	2833	2659	2659
query98	242	208	199	199
query99	1680	1517	1459	1459
Total cold run time: 304924 ms
Total hot run time: 190868 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.12 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 83249b0079280144085e8e9c5a27165567e0b8fd, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.10
query5	0.45	0.42	0.42
query6	1.16	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.53	0.52
query10	0.57	0.58	0.56
query11	0.15	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.61	0.60
query14	2.86	2.72	2.77
query15	0.89	0.82	0.82
query16	0.37	0.36	0.38
query17	1.05	1.06	1.06
query18	0.23	0.20	0.20
query19	1.83	1.77	2.00
query20	0.02	0.02	0.01
query21	15.38	0.94	0.59
query22	0.75	0.75	0.69
query23	15.34	1.46	0.56
query24	3.28	1.40	0.64
query25	0.25	0.10	0.15
query26	0.35	0.14	0.13
query27	0.05	0.05	0.05
query28	13.81	1.53	1.04
query29	12.63	3.99	3.28
query30	0.26	0.08	0.06
query31	2.81	0.59	0.38
query32	3.22	0.53	0.46
query33	3.08	3.14	3.14
query34	16.77	5.12	4.50
query35	4.46	4.50	4.54
query36	0.66	0.49	0.48
query37	0.10	0.06	0.05
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.07	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.5 s
Total hot run time: 31.12 s

@gavinchou gavinchou dismissed stale reviews from zhannngchen and dataroaring via b1471f5 December 31, 2024 11:59
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 31, 2024
@gavinchou
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32515 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b1471f56cadad949401fc864428060347932afec, data reload: false

------ Round 1 ----------------------------------
q1	17611	6180	6053	6053
q2	2054	302	171	171
q3	10417	1253	700	700
q4	10221	866	428	428
q5	7512	2115	1978	1978
q6	199	181	149	149
q7	914	759	608	608
q8	9229	1332	1163	1163
q9	5258	5005	4926	4926
q10	6787	2334	1854	1854
q11	481	279	268	268
q12	349	364	215	215
q13	17770	3538	2964	2964
q14	230	231	212	212
q15	560	498	495	495
q16	626	612	582	582
q17	564	836	339	339
q18	7126	6480	6410	6410
q19	4503	974	551	551
q20	285	310	180	180
q21	2683	2104	1939	1939
q22	362	333	330	330
Total cold run time: 105741 ms
Total hot run time: 32515 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6459	6329	6290	6290
q2	235	330	230	230
q3	2210	2578	2269	2269
q4	1450	1859	1373	1373
q5	4353	4745	5007	4745
q6	187	179	142	142
q7	2093	1975	1810	1810
q8	2585	2791	2695	2695
q9	7355	7297	7197	7197
q10	3055	3339	2867	2867
q11	606	523	501	501
q12	647	763	639	639
q13	3327	3830	3124	3124
q14	285	290	288	288
q15	551	516	504	504
q16	677	698	672	672
q17	1212	1720	1246	1246
q18	7549	7332	6828	6828
q19	784	1130	1171	1130
q20	1871	1986	1791	1791
q21	5505	5213	4831	4831
q22	606	602	563	563
Total cold run time: 53602 ms
Total hot run time: 51735 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191130 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b1471f56cadad949401fc864428060347932afec, data reload: false

query1	974	398	361	361
query2	6530	2412	2430	2412
query3	6709	215	215	215
query4	33910	23553	23827	23553
query5	4380	611	473	473
query6	282	206	213	206
query7	4617	491	302	302
query8	303	251	247	247
query9	9568	2736	2720	2720
query10	471	314	254	254
query11	18140	15449	15154	15154
query12	157	104	103	103
query13	1649	526	385	385
query14	10523	8033	7036	7036
query15	259	201	199	199
query16	8274	584	426	426
query17	1550	715	551	551
query18	2117	408	309	309
query19	217	183	152	152
query20	116	112	111	111
query21	207	124	103	103
query22	4262	4536	4391	4391
query23	34177	33507	33527	33507
query24	6472	2250	2255	2250
query25	522	481	389	389
query26	1207	267	152	152
query27	2042	465	357	357
query28	5131	2419	2398	2398
query29	730	535	413	413
query30	230	178	152	152
query31	967	889	821	821
query32	89	63	66	63
query33	500	397	289	289
query34	746	836	507	507
query35	779	813	740	740
query36	1048	1063	950	950
query37	104	100	75	75
query38	4204	4213	4278	4213
query39	1512	1420	1423	1420
query40	216	134	103	103
query41	47	48	48	48
query42	118	100	107	100
query43	528	536	499	499
query44	1272	799	803	799
query45	180	171	166	166
query46	868	1040	633	633
query47	1913	1941	1874	1874
query48	376	402	323	323
query49	775	478	378	378
query50	632	653	386	386
query51	7046	7122	7058	7058
query52	99	102	90	90
query53	219	257	186	186
query54	471	486	394	394
query55	77	76	81	76
query56	256	266	235	235
query57	1201	1178	1100	1100
query58	238	221	242	221
query59	3123	3292	3087	3087
query60	284	278	281	278
query61	151	131	140	131
query62	872	818	744	744
query63	227	194	196	194
query64	4596	1035	648	648
query65	3268	3190	3211	3190
query66	1045	420	315	315
query67	15941	15755	15682	15682
query68	8901	743	517	517
query69	461	340	248	248
query70	1249	1163	1182	1163
query71	426	289	252	252
query72	6097	3847	3852	3847
query73	651	747	361	361
query74	10440	9067	8944	8944
query75	4348	3179	2651	2651
query76	5044	1175	835	835
query77	950	375	267	267
query78	10319	10200	9386	9386
query79	3124	885	613	613
query80	724	519	437	437
query81	466	287	237	237
query82	593	158	122	122
query83	192	162	145	145
query84	280	89	82	82
query85	764	369	373	369
query86	344	326	300	300
query87	4751	4487	4635	4487
query88	3267	2255	2233	2233
query89	424	323	307	307
query90	1957	189	191	189
query91	132	132	107	107
query92	68	57	53	53
query93	1666	837	533	533
query94	670	412	298	298
query95	328	265	250	250
query96	494	608	279	279
query97	2716	2804	2675	2675
query98	221	200	216	200
query99	1645	1596	1446	1446
Total cold run time: 295426 ms
Total hot run time: 191130 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.02 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b1471f56cadad949401fc864428060347932afec, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.08	0.07
query4	1.62	0.10	0.10
query5	0.44	0.40	0.40
query6	1.16	0.64	0.65
query7	0.02	0.02	0.02
query8	0.04	0.02	0.03
query9	0.55	0.52	0.52
query10	0.55	0.56	0.57
query11	0.14	0.10	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.59
query14	2.83	2.90	2.75
query15	0.88	0.82	0.81
query16	0.38	0.37	0.38
query17	1.02	1.06	1.03
query18	0.22	0.20	0.20
query19	1.96	1.87	1.95
query20	0.01	0.02	0.01
query21	15.38	0.90	0.57
query22	0.75	0.71	0.62
query23	15.41	1.44	0.55
query24	3.42	1.60	2.17
query25	0.15	0.17	0.06
query26	0.23	0.16	0.14
query27	0.05	0.05	0.06
query28	14.96	1.44	1.05
query29	12.66	4.04	3.31
query30	0.25	0.09	0.06
query31	2.82	0.61	0.38
query32	3.23	0.55	0.46
query33	3.13	3.06	3.08
query34	16.97	5.03	4.55
query35	4.52	4.53	4.56
query36	0.64	0.48	0.50
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.03	0.02	0.02
Total cold run time: 107.97 s
Total hot run time: 32.02 s

@gavinchou
Copy link
Contributor Author

run feut

@gavinchou
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32474 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 22ad903d48cf9d041257953ccc2c66f0035c5141, data reload: false

------ Round 1 ----------------------------------
q1	17612	6168	6074	6074
q2	2058	312	166	166
q3	10408	1229	712	712
q4	10198	866	441	441
q5	7514	2160	1962	1962
q6	205	177	148	148
q7	916	752	607	607
q8	9223	1311	1173	1173
q9	5198	4854	4889	4854
q10	6723	2298	1866	1866
q11	469	272	267	267
q12	344	355	221	221
q13	17773	3532	3029	3029
q14	240	230	218	218
q15	557	511	500	500
q16	626	602	591	591
q17	556	847	337	337
q18	7278	6512	6298	6298
q19	2864	954	536	536
q20	290	310	197	197
q21	2812	2174	1969	1969
q22	355	323	308	308
Total cold run time: 104219 ms
Total hot run time: 32474 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6322	6204	6274	6204
q2	237	327	231	231
q3	2207	2618	2298	2298
q4	1377	1815	1317	1317
q5	4332	4698	4818	4698
q6	183	170	141	141
q7	2070	1975	1862	1862
q8	2599	2785	2636	2636
q9	7281	7169	7112	7112
q10	3069	3305	2813	2813
q11	582	505	515	505
q12	676	753	635	635
q13	3313	3846	3173	3173
q14	281	296	289	289
q15	570	509	497	497
q16	642	674	634	634
q17	1197	1743	1245	1245
q18	7749	7344	6944	6944
q19	775	1111	1025	1025
q20	1885	1941	1876	1876
q21	5356	5012	4883	4883
q22	590	616	606	606
Total cold run time: 53293 ms
Total hot run time: 51624 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189872 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 22ad903d48cf9d041257953ccc2c66f0035c5141, data reload: false

query1	979	375	366	366
query2	6515	2365	2317	2317
query3	6725	221	210	210
query4	33618	23674	23924	23674
query5	4781	633	448	448
query6	285	207	194	194
query7	4640	490	311	311
query8	307	244	243	243
query9	9399	2661	2615	2615
query10	487	302	243	243
query11	18238	15436	15244	15244
query12	164	104	103	103
query13	1649	515	395	395
query14	10472	6768	7007	6768
query15	235	197	183	183
query16	8300	599	445	445
query17	1509	713	540	540
query18	2119	394	293	293
query19	216	187	164	164
query20	118	112	111	111
query21	212	124	108	108
query22	4326	4514	4163	4163
query23	34902	33332	33192	33192
query24	6466	2298	2376	2298
query25	537	467	408	408
query26	1222	268	156	156
query27	1995	453	353	353
query28	5461	2420	2403	2403
query29	762	564	492	492
query30	225	179	147	147
query31	1002	950	837	837
query32	89	59	57	57
query33	528	347	297	297
query34	742	842	538	538
query35	788	815	735	735
query36	976	1041	949	949
query37	110	98	77	77
query38	4170	4211	4303	4211
query39	1479	1448	1431	1431
query40	207	109	95	95
query41	49	102	45	45
query42	117	101	104	101
query43	514	524	482	482
query44	1287	814	820	814
query45	177	169	170	169
query46	849	1041	651	651
query47	1921	1939	1846	1846
query48	375	407	323	323
query49	770	472	401	401
query50	607	649	390	390
query51	7183	7188	6930	6930
query52	99	96	91	91
query53	220	254	182	182
query54	471	481	406	406
query55	85	75	75	75
query56	247	267	246	246
query57	1199	1172	1123	1123
query58	240	220	257	220
query59	3014	3222	2894	2894
query60	279	261	254	254
query61	109	103	105	103
query62	856	788	736	736
query63	234	186	193	186
query64	4573	973	636	636
query65	3265	3183	3233	3183
query66	914	414	305	305
query67	16034	15874	15564	15564
query68	9650	755	527	527
query69	484	285	252	252
query70	1213	1118	1137	1118
query71	449	273	267	267
query72	5836	3771	3842	3771
query73	950	759	360	360
query74	10416	9284	9227	9227
query75	4641	3155	2633	2633
query76	5661	1192	776	776
query77	965	373	270	270
query78	10163	10174	9323	9323
query79	6133	864	582	582
query80	681	508	433	433
query81	462	264	228	228
query82	211	152	179	152
query83	287	175	147	147
query84	283	91	72	72
query85	732	357	300	300
query86	362	294	308	294
query87	4607	4454	4390	4390
query88	3997	2252	2214	2214
query89	428	334	302	302
query90	2082	192	193	192
query91	130	132	105	105
query92	72	56	114	56
query93	3368	859	533	533
query94	676	380	281	281
query95	335	266	254	254
query96	487	602	287	287
query97	2713	2830	2727	2727
query98	222	212	195	195
query99	1669	1559	1434	1434
Total cold run time: 303008 ms
Total hot run time: 189872 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.58 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 22ad903d48cf9d041257953ccc2c66f0035c5141, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.10	0.11
query5	0.42	0.41	0.41
query6	1.19	0.65	0.65
query7	0.02	0.02	0.01
query8	0.04	0.04	0.03
query9	0.59	0.50	0.50
query10	0.55	0.57	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.60	0.59
query14	2.70	2.86	2.75
query15	0.89	0.83	0.82
query16	0.39	0.41	0.38
query17	1.06	1.08	1.03
query18	0.23	0.21	0.22
query19	1.99	1.82	1.97
query20	0.01	0.01	0.02
query21	15.36	0.91	0.57
query22	0.75	0.72	0.65
query23	15.41	1.35	0.57
query24	3.15	1.36	1.48
query25	0.21	0.12	0.14
query26	0.24	0.15	0.14
query27	0.06	0.04	0.04
query28	14.25	1.50	1.04
query29	12.57	3.96	3.21
query30	0.24	0.08	0.06
query31	2.82	0.59	0.38
query32	3.22	0.53	0.46
query33	3.10	3.16	3.11
query34	16.64	5.07	4.45
query35	4.44	4.43	4.46
query36	0.63	0.50	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.03	0.02
query40	0.18	0.13	0.14
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.49 s
Total hot run time: 31.58 s

@gavinchou
Copy link
Contributor Author

run buildall

@gavinchou
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32733 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d1fd50c56f1a39902b3616a33edde28de3a4352d, data reload: false

------ Round 1 ----------------------------------
q1	17620	6186	6017	6017
q2	2044	301	166	166
q3	10423	1290	731	731
q4	10216	873	443	443
q5	7522	2190	1968	1968
q6	204	181	153	153
q7	889	739	639	639
q8	9250	1385	1173	1173
q9	5300	4954	4891	4891
q10	6764	2314	1849	1849
q11	473	281	266	266
q12	344	354	220	220
q13	17762	3640	3084	3084
q14	228	241	224	224
q15	563	509	499	499
q16	625	626	592	592
q17	573	863	338	338
q18	7210	6367	6372	6367
q19	4501	977	556	556
q20	310	313	184	184
q21	2804	2260	2055	2055
q22	378	334	318	318
Total cold run time: 106003 ms
Total hot run time: 32733 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6448	6236	6237	6236
q2	238	327	234	234
q3	2234	2652	2309	2309
q4	1386	1807	1341	1341
q5	4389	4748	4939	4748
q6	194	189	150	150
q7	2105	1995	1787	1787
q8	2608	2804	2697	2697
q9	7295	7159	7210	7159
q10	3041	3316	2829	2829
q11	597	535	501	501
q12	651	775	591	591
q13	3497	3810	3232	3232
q14	289	328	314	314
q15	561	516	516	516
q16	663	685	628	628
q17	1208	1720	1247	1247
q18	7764	7467	6964	6964
q19	831	1154	1026	1026
q20	1884	1980	1873	1873
q21	5524	5098	4861	4861
q22	614	605	596	596
Total cold run time: 54021 ms
Total hot run time: 51839 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190741 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d1fd50c56f1a39902b3616a33edde28de3a4352d, data reload: false

query1	987	372	359	359
query2	6516	2441	2286	2286
query3	6710	213	214	213
query4	34238	23694	23618	23618
query5	4362	607	446	446
query6	272	201	199	199
query7	4619	497	306	306
query8	301	255	243	243
query9	9692	2658	2640	2640
query10	452	318	246	246
query11	18467	15440	15341	15341
query12	157	106	103	103
query13	1643	532	383	383
query14	11748	7132	7791	7132
query15	266	208	205	205
query16	8749	604	424	424
query17	1611	743	556	556
query18	2132	393	289	289
query19	219	172	147	147
query20	115	114	111	111
query21	210	120	102	102
query22	4351	4479	4246	4246
query23	34932	33315	33424	33315
query24	6388	2333	2292	2292
query25	462	435	384	384
query26	1080	250	153	153
query27	2037	434	332	332
query28	5000	2450	2409	2409
query29	537	516	415	415
query30	227	179	148	148
query31	1012	908	847	847
query32	86	61	61	61
query33	501	346	295	295
query34	784	811	505	505
query35	805	834	745	745
query36	1017	1025	957	957
query37	117	95	73	73
query38	4143	4144	4240	4144
query39	1474	1423	1432	1423
query40	206	118	100	100
query41	48	45	50	45
query42	124	105	105	105
query43	537	565	511	511
query44	1299	804	824	804
query45	182	175	181	175
query46	862	1049	656	656
query47	1909	1909	1892	1892
query48	390	430	315	315
query49	809	492	434	434
query50	610	644	399	399
query51	7168	7168	7146	7146
query52	107	102	96	96
query53	224	264	187	187
query54	484	488	407	407
query55	85	79	85	79
query56	257	301	253	253
query57	1172	1173	1131	1131
query58	242	233	226	226
query59	3114	3199	2914	2914
query60	286	296	255	255
query61	112	106	106	106
query62	869	786	771	771
query63	223	190	198	190
query64	4246	988	632	632
query65	3268	3216	3285	3216
query66	1040	432	306	306
query67	16027	15912	15517	15517
query68	7647	692	520	520
query69	477	283	263	263
query70	1205	1162	1119	1119
query71	441	289	257	257
query72	6199	3939	3797	3797
query73	663	749	360	360
query74	10335	9181	9022	9022
query75	4021	3172	2677	2677
query76	3693	1196	783	783
query77	765	362	284	284
query78	10127	10534	9437	9437
query79	3191	818	605	605
query80	728	519	436	436
query81	475	278	229	229
query82	628	162	123	123
query83	162	163	144	144
query84	238	90	76	76
query85	784	360	295	295
query86	361	320	297	297
query87	4411	4550	4394	4394
query88	4553	2182	2164	2164
query89	410	341	302	302
query90	1943	186	196	186
query91	134	138	102	102
query92	73	63	52	52
query93	1445	882	544	544
query94	653	394	298	298
query95	341	257	242	242
query96	493	618	280	280
query97	2876	2952	2851	2851
query98	235	249	198	198
query99	1684	1560	1443	1443
Total cold run time: 295710 ms
Total hot run time: 190741 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.5 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d1fd50c56f1a39902b3616a33edde28de3a4352d, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.07
query4	1.62	0.11	0.10
query5	0.40	0.41	0.42
query6	1.17	0.65	0.65
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.50	0.52
query10	0.55	0.57	0.56
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.60	0.61	0.59
query14	2.85	2.83	2.80
query15	0.89	0.83	0.83
query16	0.38	0.38	0.37
query17	1.06	1.02	1.06
query18	0.24	0.20	0.21
query19	1.99	1.82	2.00
query20	0.02	0.01	0.01
query21	15.35	0.94	0.59
query22	0.74	0.78	0.68
query23	15.32	1.46	0.59
query24	2.96	1.12	1.14
query25	0.28	0.08	0.06
query26	0.28	0.15	0.13
query27	0.06	0.07	0.05
query28	13.46	1.53	1.04
query29	12.58	3.97	3.33
query30	0.24	0.08	0.06
query31	2.82	0.60	0.38
query32	3.23	0.55	0.47
query33	3.09	3.12	3.07
query34	17.19	5.07	4.48
query35	4.48	4.45	4.50
query36	0.67	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.18	0.13	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 106.27 s
Total hot run time: 31.5 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 3, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2025

PR approved by at least one committer and no changes requested.

@gavinchou gavinchou merged commit 10eac22 into apache:master Jan 3, 2025
25 of 28 checks passed
gavinchou added a commit to gavinchou/doris that referenced this pull request Jan 5, 2025
@gavinchou
[chore](conf) Set enable_advance_next_id=true by default (apache#44790)
6eaeedd 
Bind the generation of next_id to physical time to ensure its generation
is monotonically increasing, even if we directly overwrite the FE
metadata and then restart. This way, it supports a lossy cluster
rollback in scenarios where major version upgrades are incompatible.
@gavinchou gavinchou mentioned this pull request Jan 5, 2025
Merged
16 tasks
gavinchou added a commit that referenced this pull request Jan 5, 2025
@gavinchou
[chore](conf) Set enable_advance_next_id=true by default (#44790) (#4…
71a3cc7 
…6425)

pick #44790

Bind the generation of next_id to physical time to ensure its generation
is monotonically increasing, even if we directly overwrite the FE
metadata and then restart. This way, it supports a lossy cluster
rollback in scenarios where major version upgrades are incompatible.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bobhan1 bobhan1 bobhan1 approved these changes

@zhannngchen zhannngchen zhannngchen approved these changes

@liaoxin01 liaoxin01 liaoxin01 approved these changes

@dataroaring dataroaring dataroaring left review comments

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.3-merged kind/behavior-changed reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@gavinchou @doris-robot @liaoxin01 @zhannngchen @bobhan1 @dataroaring