Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature](backup) backup and restore privileges catalogs workloadgroup #44905

Merged
merged 4 commits into from
Dec 25, 2024

Conversation

justfortaste
Copy link
Contributor

@justfortaste justfortaste commented Dec 3, 2024

What problem does this PR solve?

Issue Number: close #43952

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@justfortaste justfortaste force-pushed the feature_backup_user_master branch from a1b6b8c to cd44780 Compare December 3, 2024 08:19
@justfortaste
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40176 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit cd4478055df98f21c5eb2e8a11a889d669eb7aa0, data reload: false

------ Round 1 ----------------------------------
q1	17968	8058	7289	7289
q2	2049	185	174	174
q3	10839	1083	1213	1083
q4	10544	747	728	728
q5	7648	2729	2762	2729
q6	239	149	148	148
q7	1004	642	604	604
q8	9265	1921	1915	1915
q9	6708	6556	6537	6537
q10	7066	2301	2307	2301
q11	474	264	246	246
q12	419	221	225	221
q13	17775	3099	3028	3028
q14	248	216	215	215
q15	584	546	518	518
q16	663	583	600	583
q17	976	576	577	576
q18	7500	6589	6596	6589
q19	1370	1090	1013	1013
q20	471	181	181	181
q21	4002	3232	3179	3179
q22	380	322	319	319
Total cold run time: 108192 ms
Total hot run time: 40176 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7251	7208	7227	7208
q2	334	232	237	232
q3	2891	2917	2893	2893
q4	2121	1812	1903	1812
q5	5675	5658	5644	5644
q6	229	146	143	143
q7	2228	1840	1815	1815
q8	3422	3566	3527	3527
q9	8910	9052	9052	9052
q10	3616	3574	3543	3543
q11	614	513	523	513
q12	828	595	633	595
q13	9978	3252	3263	3252
q14	299	262	270	262
q15	606	524	525	524
q16	696	656	661	656
q17	1896	1655	1597	1597
q18	8284	7743	7654	7654
q19	1691	1544	1608	1544
q20	2125	1893	1909	1893
q21	5601	5445	5522	5445
q22	663	566	601	566
Total cold run time: 69958 ms
Total hot run time: 60370 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197345 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit cd4478055df98f21c5eb2e8a11a889d669eb7aa0, data reload: false

query1	1248	966	991	966
query2	6233	2031	1998	1998
query3	11120	4495	4676	4495
query4	67319	29634	23599	23599
query5	4984	495	471	471
query6	420	188	179	179
query7	5656	310	299	299
query8	328	242	243	242
query9	9159	2707	2691	2691
query10	468	249	257	249
query11	17880	15498	15811	15498
query12	159	107	112	107
query13	1548	425	435	425
query14	10371	7024	6719	6719
query15	208	187	192	187
query16	7099	496	439	439
query17	1212	577	561	561
query18	1787	307	308	307
query19	212	155	153	153
query20	119	108	111	108
query21	207	100	101	100
query22	4866	4688	4486	4486
query23	34997	34400	34687	34400
query24	5504	2606	2490	2490
query25	485	380	395	380
query26	650	154	154	154
query27	1824	285	295	285
query28	4511	2506	2481	2481
query29	682	425	449	425
query30	205	153	150	150
query31	1039	854	844	844
query32	67	62	56	56
query33	437	317	298	298
query34	940	519	532	519
query35	898	756	759	756
query36	1102	967	962	962
query37	123	71	79	71
query38	4565	4419	4392	4392
query39	1529	1503	1517	1503
query40	199	100	102	100
query41	45	41	40	40
query42	113	98	98	98
query43	535	485	505	485
query44	1200	839	821	821
query45	191	176	167	167
query46	1173	739	731	731
query47	2055	1949	1926	1926
query48	421	319	318	318
query49	727	395	430	395
query50	857	403	399	399
query51	7339	7199	7139	7139
query52	95	85	89	85
query53	256	182	182	182
query54	516	409	391	391
query55	74	78	76	76
query56	253	234	236	234
query57	1301	1156	1125	1125
query58	221	219	211	211
query59	3096	3093	2938	2938
query60	271	280	268	268
query61	110	106	107	106
query62	806	674	645	645
query63	214	187	190	187
query64	1382	668	639	639
query65	3297	3171	3212	3171
query66	692	327	286	286
query67	15930	15795	15544	15544
query68	3548	569	561	561
query69	418	257	257	257
query70	1109	1160	1145	1145
query71	351	249	249	249
query72	6408	4355	4156	4156
query73	754	358	364	358
query74	10096	9006	9035	9006
query75	3406	2649	2719	2649
query76	1769	1099	1127	1099
query77	495	279	294	279
query78	10466	9535	9405	9405
query79	1124	596	593	593
query80	812	429	443	429
query81	504	231	232	231
query82	202	128	118	118
query83	167	144	144	144
query84	281	158	66	66
query85	855	299	315	299
query86	365	270	301	270
query87	4631	4583	4621	4583
query88	3561	2204	2175	2175
query89	418	306	294	294
query90	2015	192	185	185
query91	138	103	104	103
query92	66	48	49	48
query93	1137	532	540	532
query94	783	292	299	292
query95	342	265	245	245
query96	618	289	276	276
query97	2846	2666	2694	2666
query98	212	193	205	193
query99	1618	1329	1287	1287
Total cold run time: 317045 ms
Total hot run time: 197345 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.88 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit cd4478055df98f21c5eb2e8a11a889d669eb7aa0, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.05	0.05
query3	0.23	0.05	0.06
query4	1.65	0.08	0.08
query5	0.43	0.40	0.40
query6	1.16	0.68	0.66
query7	0.02	0.02	0.01
query8	0.05	0.05	0.05
query9	0.56	0.49	0.49
query10	0.56	0.55	0.57
query11	0.16	0.12	0.13
query12	0.15	0.12	0.12
query13	0.62	0.60	0.60
query14	2.73	2.74	2.77
query15	0.93	0.86	0.84
query16	0.38	0.39	0.39
query17	1.07	1.06	1.06
query18	0.19	0.19	0.20
query19	1.84	1.79	2.04
query20	0.02	0.02	0.01
query21	15.36	0.69	0.68
query22	4.64	5.82	2.39
query23	18.24	1.46	1.36
query24	2.09	0.23	0.23
query25	0.14	0.08	0.08
query26	0.26	0.18	0.18
query27	0.09	0.08	0.08
query28	13.26	1.19	1.15
query29	12.63	3.40	3.32
query30	0.24	0.06	0.05
query31	2.85	0.40	0.40
query32	3.26	0.48	0.49
query33	3.10	3.07	3.12
query34	16.97	4.53	4.53
query35	4.71	4.57	4.63
query36	0.68	0.49	0.47
query37	0.20	0.16	0.15
query38	0.15	0.17	0.15
query39	0.05	0.04	0.03
query40	0.16	0.13	0.14
query41	0.10	0.05	0.05
query42	0.06	0.06	0.05
query43	0.05	0.05	0.05
Total cold run time: 112.15 s
Total hot run time: 33.88 s

lide-reed
lide-reed previously approved these changes Dec 4, 2024
Copy link
Contributor

@lide-reed lide-reed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Dec 4, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Dec 4, 2024
Copy link
Contributor

github-actions bot commented Dec 4, 2024

PR approved by anyone and no changes requested.

@justfortaste
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 4, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 39965 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c03b5179e3beaae38ade25d1e89298d73708775b, data reload: false

------ Round 1 ----------------------------------
q1	17983	7721	7287	7287
q2	2063	174	169	169
q3	10894	1110	1202	1110
q4	10318	691	737	691
q5	7593	2767	2652	2652
q6	238	150	147	147
q7	1012	645	602	602
q8	9247	1859	1897	1859
q9	6718	6532	6477	6477
q10	7001	2306	2338	2306
q11	470	260	259	259
q12	438	229	220	220
q13	17797	3067	3075	3067
q14	257	218	212	212
q15	581	540	521	521
q16	687	588	585	585
q17	989	582	522	522
q18	7331	6672	6693	6672
q19	1362	1031	953	953
q20	459	177	177	177
q21	4121	3192	3169	3169
q22	374	308	316	308
Total cold run time: 107933 ms
Total hot run time: 39965 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7308	7243	7261	7243
q2	333	231	229	229
q3	2904	2908	2999	2908
q4	2107	1893	1898	1893
q5	5655	5666	5698	5666
q6	224	141	141	141
q7	2249	1844	1813	1813
q8	3436	3530	3572	3530
q9	8890	9129	9048	9048
q10	3635	3578	3568	3568
q11	630	503	523	503
q12	858	637	620	620
q13	11554	3298	3238	3238
q14	313	272	270	270
q15	582	544	528	528
q16	715	636	642	636
q17	1855	1618	1589	1589
q18	8507	7767	7694	7694
q19	1691	1517	1637	1517
q20	2148	1869	1894	1869
q21	5669	5596	5543	5543
q22	644	567	643	567
Total cold run time: 71907 ms
Total hot run time: 60613 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198747 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c03b5179e3beaae38ade25d1e89298d73708775b, data reload: false

query1	1305	973	923	923
query2	6244	2055	1987	1987
query3	11113	4698	4695	4695
query4	33204	23545	23463	23463
query5	3524	475	449	449
query6	272	185	187	185
query7	3991	309	306	306
query8	297	239	238	238
query9	9555	2726	2721	2721
query10	473	246	248	246
query11	17986	15449	15336	15336
query12	162	102	107	102
query13	1554	414	412	412
query14	9700	7254	7958	7254
query15	265	220	198	198
query16	7832	486	510	486
query17	1640	585	573	573
query18	1543	310	305	305
query19	363	154	150	150
query20	128	111	113	111
query21	204	106	105	105
query22	4786	4421	4399	4399
query23	36428	34822	34840	34822
query24	11124	2712	2551	2551
query25	619	400	393	393
query26	1444	157	153	153
query27	2557	281	284	281
query28	7461	2514	2489	2489
query29	885	442	417	417
query30	239	158	155	155
query31	1051	858	834	834
query32	97	57	57	57
query33	775	309	294	294
query34	1253	526	539	526
query35	899	760	745	745
query36	1139	982	990	982
query37	140	77	74	74
query38	4428	4459	4412	4412
query39	1554	1496	1469	1469
query40	276	105	105	105
query41	48	42	45	42
query42	109	103	103	103
query43	536	512	505	505
query44	1395	852	826	826
query45	204	174	172	172
query46	1185	744	744	744
query47	2055	1953	1918	1918
query48	436	331	337	331
query49	998	431	420	420
query50	844	417	410	410
query51	7433	7329	7297	7297
query52	106	90	93	90
query53	265	183	186	183
query54	1435	424	448	424
query55	87	82	87	82
query56	271	239	238	238
query57	1330	1162	1125	1125
query58	233	229	213	213
query59	3292	3018	3109	3018
query60	271	240	245	240
query61	121	113	110	110
query62	850	669	672	669
query63	213	199	198	198
query64	4928	660	706	660
query65	3280	3198	3214	3198
query66	1099	314	329	314
query67	16109	15801	15706	15706
query68	5464	571	551	551
query69	406	256	262	256
query70	1200	1139	1063	1063
query71	319	247	252	247
query72	6213	4124	4067	4067
query73	756	366	366	366
query74	10256	8973	9061	8973
query75	3417	2674	2716	2674
query76	3243	1110	1106	1106
query77	378	280	281	280
query78	10510	9493	9475	9475
query79	1693	632	605	605
query80	1074	433	443	433
query81	543	237	238	237
query82	938	128	119	119
query83	243	146	143	143
query84	235	67	72	67
query85	1373	297	309	297
query86	391	303	297	297
query87	4669	4492	4656	4492
query88	3535	2228	2189	2189
query89	436	293	286	286
query90	1960	191	232	191
query91	144	101	108	101
query92	61	50	53	50
query93	1887	551	554	551
query94	761	296	301	296
query95	356	258	250	250
query96	619	281	275	275
query97	2850	2710	2659	2659
query98	221	205	196	196
query99	1567	1323	1316	1316
Total cold run time: 305361 ms
Total hot run time: 198747 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.54 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c03b5179e3beaae38ade25d1e89298d73708775b, data reload: false

query1	0.03	0.03	0.02
query2	0.08	0.05	0.05
query3	0.23	0.05	0.05
query4	1.65	0.08	0.08
query5	0.41	0.42	0.41
query6	1.14	0.69	0.67
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.55	0.51	0.50
query10	0.56	0.56	0.55
query11	0.17	0.12	0.12
query12	0.15	0.13	0.12
query13	0.61	0.61	0.60
query14	2.72	2.76	2.71
query15	0.93	0.86	0.83
query16	0.37	0.37	0.37
query17	1.06	1.00	1.06
query18	0.19	0.19	0.19
query19	1.99	1.90	2.11
query20	0.02	0.02	0.02
query21	15.37	0.66	0.67
query22	4.57	6.35	2.05
query23	18.21	1.38	1.28
query24	2.19	0.22	0.22
query25	0.16	0.08	0.08
query26	0.26	0.18	0.18
query27	0.08	0.07	0.08
query28	13.21	1.17	1.16
query29	12.66	3.44	3.43
query30	0.24	0.05	0.06
query31	2.88	0.41	0.40
query32	3.22	0.48	0.48
query33	3.02	2.98	3.05
query34	16.73	4.57	4.55
query35	4.62	4.59	4.61
query36	0.68	0.52	0.49
query37	0.19	0.16	0.16
query38	0.15	0.16	0.15
query39	0.05	0.04	0.03
query40	0.17	0.13	0.13
query41	0.09	0.06	0.05
query42	0.06	0.05	0.06
query43	0.05	0.05	0.05
Total cold run time: 111.79 s
Total hot run time: 33.54 s

@justfortaste
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40581 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9b3b17ab5cf3f87b1eb2d138231bba5470bcef21, data reload: false

------ Round 1 ----------------------------------
q1	17996	8816	7588	7588
q2	2078	181	175	175
q3	10652	1146	1184	1146
q4	10496	775	760	760
q5	7592	2847	2755	2755
q6	235	150	151	150
q7	1010	640	610	610
q8	9469	1905	1920	1905
q9	6623	6469	6499	6469
q10	6999	2327	2334	2327
q11	475	266	262	262
q12	410	228	225	225
q13	17780	3036	3027	3027
q14	252	214	210	210
q15	569	543	531	531
q16	669	605	575	575
q17	979	579	598	579
q18	7241	6728	6631	6631
q19	1357	1062	981	981
q20	471	181	187	181
q21	3976	3245	3186	3186
q22	392	322	308	308
Total cold run time: 107721 ms
Total hot run time: 40581 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7286	7214	7197	7197
q2	327	227	231	227
q3	2833	2977	2931	2931
q4	2118	1888	1860	1860
q5	5612	5672	5644	5644
q6	229	144	149	144
q7	2229	1785	1829	1785
q8	3344	3504	3537	3504
q9	8819	9000	8969	8969
q10	3577	3595	3544	3544
q11	598	512	505	505
q12	796	607	616	607
q13	12257	3272	3218	3218
q14	302	289	268	268
q15	576	519	519	519
q16	673	660	650	650
q17	1831	1638	1626	1626
q18	8355	7635	7793	7635
q19	1668	1476	1467	1467
q20	2147	1874	1870	1870
q21	5659	5553	5333	5333
q22	651	576	572	572
Total cold run time: 71887 ms
Total hot run time: 60075 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198024 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9b3b17ab5cf3f87b1eb2d138231bba5470bcef21, data reload: false

query1	1535	963	946	946
query2	6210	2071	2035	2035
query3	10988	4523	4437	4437
query4	66601	29096	23759	23759
query5	5021	472	441	441
query6	400	178	172	172
query7	5474	303	306	303
query8	324	236	238	236
query9	8930	2731	2709	2709
query10	439	238	242	238
query11	17233	15383	15983	15383
query12	157	97	103	97
query13	1447	422	386	386
query14	9742	8322	7620	7620
query15	209	187	196	187
query16	7148	496	481	481
query17	1208	581	606	581
query18	1843	323	328	323
query19	202	154	162	154
query20	122	115	116	115
query21	211	108	105	105
query22	4871	4636	4397	4397
query23	35089	34544	34506	34506
query24	5498	2517	2516	2516
query25	479	390	390	390
query26	636	148	148	148
query27	1896	291	282	282
query28	4286	2514	2480	2480
query29	674	412	433	412
query30	201	157	150	150
query31	1015	835	853	835
query32	65	52	50	50
query33	433	293	300	293
query34	936	545	544	544
query35	876	753	777	753
query36	1104	1004	959	959
query37	123	71	72	71
query38	4532	4401	4543	4401
query39	1542	1613	1454	1454
query40	211	100	98	98
query41	44	41	42	41
query42	110	95	100	95
query43	536	489	506	489
query44	1196	824	846	824
query45	183	175	165	165
query46	1178	722	746	722
query47	2005	1915	1924	1915
query48	415	327	316	316
query49	735	386	379	379
query50	833	389	391	389
query51	7290	7254	7116	7116
query52	94	87	89	87
query53	258	183	175	175
query54	499	395	409	395
query55	76	75	77	75
query56	244	234	238	234
query57	1220	1109	1110	1109
query58	212	202	204	202
query59	3219	2879	2858	2858
query60	256	237	236	236
query61	104	109	109	109
query62	755	673	661	661
query63	209	185	198	185
query64	1357	689	628	628
query65	3252	3239	3232	3232
query66	635	299	295	295
query67	16024	15761	15647	15647
query68	4079	569	560	560
query69	436	271	245	245
query70	1248	1165	1134	1134
query71	320	245	253	245
query72	6398	3976	4078	3976
query73	757	370	372	370
query74	10224	9035	8994	8994
query75	3396	2671	2649	2649
query76	2026	1090	999	999
query77	480	273	266	266
query78	10467	9500	9502	9500
query79	1451	610	593	593
query80	860	428	429	428
query81	495	224	232	224
query82	1053	127	120	120
query83	162	152	148	148
query84	282	72	70	70
query85	817	289	290	289
query86	319	302	300	300
query87	4700	4533	4577	4533
query88	3554	2241	2199	2199
query89	422	299	289	289
query90	2035	185	187	185
query91	132	103	103	103
query92	66	49	54	49
query93	1654	549	552	549
query94	769	289	296	289
query95	344	251	252	251
query96	613	277	284	277
query97	2829	2683	2700	2683
query98	213	194	196	194
query99	1586	1317	1287	1287
Total cold run time: 317037 ms
Total hot run time: 198024 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.85 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9b3b17ab5cf3f87b1eb2d138231bba5470bcef21, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.05	0.04
query3	0.23	0.06	0.06
query4	1.64	0.08	0.09
query5	0.42	0.41	0.40
query6	1.15	0.67	0.65
query7	0.02	0.01	0.02
query8	0.06	0.04	0.04
query9	0.55	0.50	0.51
query10	0.54	0.56	0.54
query11	0.16	0.12	0.11
query12	0.16	0.13	0.13
query13	0.60	0.60	0.59
query14	2.74	2.74	2.78
query15	0.94	0.84	0.83
query16	0.38	0.38	0.37
query17	1.06	1.09	1.06
query18	0.20	0.19	0.19
query19	1.87	1.79	2.00
query20	0.01	0.01	0.01
query21	15.36	0.66	0.65
query22	3.72	7.08	2.56
query23	18.27	1.39	1.38
query24	2.16	0.22	0.22
query25	0.16	0.09	0.10
query26	0.28	0.18	0.18
query27	0.09	0.08	0.07
query28	13.28	1.17	1.15
query29	12.61	3.42	3.37
query30	0.24	0.06	0.06
query31	2.87	0.39	0.40
query32	3.24	0.49	0.47
query33	3.02	3.04	3.06
query34	16.84	4.48	4.45
query35	4.56	4.54	4.60
query36	0.66	0.47	0.48
query37	0.19	0.15	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.13	0.13
query41	0.09	0.04	0.05
query42	0.06	0.04	0.05
query43	0.05	0.04	0.04
Total cold run time: 110.97 s
Total hot run time: 33.85 s

@justfortaste justfortaste force-pushed the feature_backup_user_master branch from 9b3b17a to 4deab8c Compare December 5, 2024 04:46
@justfortaste
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39926 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4deab8c3de336d6a7a5b03bf8742684af2a04833, data reload: false

------ Round 1 ----------------------------------
q1	18018	8185	7340	7340
q2	2080	177	168	168
q3	10734	1088	1191	1088
q4	10326	753	752	752
q5	7578	2696	2578	2578
q6	242	149	146	146
q7	994	658	602	602
q8	9240	1828	1861	1828
q9	6600	6516	6502	6502
q10	6966	2311	2330	2311
q11	456	246	252	246
q12	428	221	218	218
q13	17759	3018	3026	3018
q14	236	211	217	211
q15	581	528	526	526
q16	633	582	587	582
q17	967	571	650	571
q18	7334	6643	6703	6643
q19	1344	1005	948	948
q20	470	182	177	177
q21	4017	3363	3157	3157
q22	397	314	329	314
Total cold run time: 107400 ms
Total hot run time: 39926 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7321	7200	7274	7200
q2	323	224	221	221
q3	2924	2850	2904	2850
q4	2022	1802	1803	1802
q5	5614	5616	5583	5583
q6	226	140	142	140
q7	2256	1847	1811	1811
q8	3394	3494	3449	3449
q9	8912	8924	8964	8924
q10	3558	3499	3516	3499
q11	602	513	495	495
q12	808	609	624	609
q13	11252	3191	3218	3191
q14	317	265	264	264
q15	587	536	520	520
q16	690	642	647	642
q17	1840	1635	1606	1606
q18	8135	7713	7791	7713
q19	1643	1551	1507	1507
q20	2115	1888	1872	1872
q21	5644	5458	5349	5349
q22	623	595	568	568
Total cold run time: 70806 ms
Total hot run time: 59815 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196804 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4deab8c3de336d6a7a5b03bf8742684af2a04833, data reload: false

query1	1535	971	955	955
query2	6252	2080	2047	2047
query3	10980	4270	4312	4270
query4	67348	28150	23576	23576
query5	4986	462	455	455
query6	390	186	178	178
query7	5507	315	300	300
query8	330	239	240	239
query9	8325	2736	2736	2736
query10	415	258	250	250
query11	16839	15313	15923	15313
query12	158	106	108	106
query13	1454	432	410	410
query14	10808	7016	7281	7016
query15	218	187	190	187
query16	6779	492	444	444
query17	1088	640	577	577
query18	1010	298	297	297
query19	200	159	161	159
query20	122	119	112	112
query21	210	122	104	104
query22	4867	4550	4511	4511
query23	35231	34362	34624	34362
query24	5499	2556	2641	2556
query25	483	408	385	385
query26	649	146	155	146
query27	2083	288	293	288
query28	4603	2500	2490	2490
query29	670	417	420	417
query30	211	156	153	153
query31	997	844	850	844
query32	68	54	84	54
query33	443	317	298	298
query34	940	524	552	524
query35	880	762	785	762
query36	1098	969	973	969
query37	128	77	70	70
query38	4469	4416	4352	4352
query39	1549	1495	1489	1489
query40	202	98	98	98
query41	43	44	44	44
query42	116	97	99	97
query43	541	477	510	477
query44	1196	831	829	829
query45	186	170	172	170
query46	1194	707	777	707
query47	2049	1918	1900	1900
query48	411	313	312	312
query49	741	382	402	382
query50	834	394	400	394
query51	7379	7135	7002	7002
query52	100	84	88	84
query53	267	177	176	176
query54	511	410	391	391
query55	77	77	79	77
query56	256	235	232	232
query57	1230	1119	1097	1097
query58	218	204	213	204
query59	3243	2991	2882	2882
query60	268	240	264	240
query61	108	134	101	101
query62	775	698	659	659
query63	208	184	189	184
query64	1427	664	633	633
query65	3282	3211	3218	3211
query66	710	305	295	295
query67	15997	15514	15769	15514
query68	3870	587	570	570
query69	434	258	252	252
query70	1220	1128	1188	1128
query71	373	249	245	245
query72	6163	4095	4005	4005
query73	773	369	360	360
query74	10293	8976	9155	8976
query75	3408	2658	2711	2658
query76	1881	1040	1119	1040
query77	487	268	366	268
query78	10537	9467	9426	9426
query79	1171	598	600	598
query80	814	419	450	419
query81	485	295	236	236
query82	1357	121	120	120
query83	253	146	156	146
query84	278	72	68	68
query85	863	296	310	296
query86	328	301	309	301
query87	4718	4511	4519	4511
query88	3395	2232	2218	2218
query89	420	296	290	290
query90	1990	187	188	187
query91	137	100	104	100
query92	66	54	52	52
query93	1278	543	556	543
query94	744	287	282	282
query95	353	254	251	251
query96	616	296	279	279
query97	2841	2680	2701	2680
query98	221	199	192	192
query99	1894	1314	1334	1314
Total cold run time: 317052 ms
Total hot run time: 196804 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.32 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4deab8c3de336d6a7a5b03bf8742684af2a04833, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.05	0.04
query3	0.23	0.05	0.05
query4	1.66	0.09	0.08
query5	0.42	0.40	0.40
query6	1.13	0.68	0.66
query7	0.02	0.02	0.02
query8	0.06	0.04	0.05
query9	0.55	0.49	0.50
query10	0.54	0.56	0.56
query11	0.16	0.12	0.13
query12	0.16	0.13	0.13
query13	0.62	0.60	0.60
query14	2.81	2.72	2.73
query15	0.90	0.85	0.83
query16	0.38	0.40	0.38
query17	1.00	1.03	1.07
query18	0.20	0.18	0.20
query19	1.86	1.72	1.93
query20	0.02	0.02	0.01
query21	15.35	0.66	0.66
query22	4.34	6.78	2.11
query23	18.20	1.38	1.30
query24	2.12	0.24	0.22
query25	0.15	0.08	0.09
query26	0.28	0.18	0.19
query27	0.08	0.08	0.08
query28	13.24	1.16	1.15
query29	12.65	3.37	3.38
query30	0.24	0.06	0.06
query31	2.88	0.40	0.40
query32	3.23	0.48	0.50
query33	2.97	3.07	3.06
query34	17.04	4.52	4.48
query35	4.52	4.52	4.49
query36	0.66	0.49	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.04	0.04
query40	0.17	0.13	0.13
query41	0.10	0.05	0.06
query42	0.06	0.06	0.05
query43	0.05	0.05	0.04
Total cold run time: 111.55 s
Total hot run time: 33.32 s

@justfortaste
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 24, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 40072 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 424bbf8f5cf625d8f59ad13089bb0c6d37a38f2e, data reload: false

------ Round 1 ----------------------------------
q1	17600	7511	7295	7295
q2	2059	181	175	175
q3	10624	1074	1209	1074
q4	10556	767	780	767
q5	7632	2735	2738	2735
q6	244	146	151	146
q7	992	627	616	616
q8	9247	1880	1901	1880
q9	6614	6401	6409	6401
q10	7039	2301	2335	2301
q11	494	272	272	272
q12	414	216	227	216
q13	17797	2938	2956	2938
q14	245	213	217	213
q15	569	519	492	492
q16	664	582	589	582
q17	972	547	548	547
q18	7324	6748	6757	6748
q19	1415	1051	1023	1023
q20	485	187	191	187
q21	4100	3149	3448	3149
q22	381	323	315	315
Total cold run time: 107467 ms
Total hot run time: 40072 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7344	7303	7242	7242
q2	331	232	229	229
q3	2872	2788	2935	2788
q4	2038	1824	1795	1795
q5	5672	5685	5616	5616
q6	223	145	149	145
q7	2233	1769	1852	1769
q8	3437	3549	3493	3493
q9	8862	9001	8919	8919
q10	3600	3548	3529	3529
q11	614	513	512	512
q12	832	616	608	608
q13	12609	3126	3067	3067
q14	304	284	290	284
q15	547	506	506	506
q16	713	659	677	659
q17	1867	1608	1631	1608
q18	8266	7746	7488	7488
q19	1720	1647	1466	1466
q20	2090	1847	1858	1847
q21	5691	5530	5494	5494
q22	650	596	585	585
Total cold run time: 72515 ms
Total hot run time: 59649 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196134 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 424bbf8f5cf625d8f59ad13089bb0c6d37a38f2e, data reload: false

query1	1305	944	930	930
query2	6258	2325	2251	2251
query3	10962	4652	4793	4652
query4	33632	23309	23406	23309
query5	4736	463	437	437
query6	292	232	193	193
query7	4002	300	304	300
query8	311	238	239	238
query9	9572	2767	2760	2760
query10	504	259	258	258
query11	17906	15101	15123	15101
query12	164	106	103	103
query13	1584	429	443	429
query14	10119	6869	6801	6801
query15	266	180	213	180
query16	7977	472	426	426
query17	1592	589	591	589
query18	2087	306	317	306
query19	368	167	161	161
query20	136	113	118	113
query21	203	114	109	109
query22	4854	4714	4471	4471
query23	34441	33908	33857	33857
query24	10436	2523	2555	2523
query25	641	388	390	388
query26	1195	160	165	160
query27	2657	336	336	336
query28	7746	2507	2498	2498
query29	820	427	439	427
query30	238	150	151	150
query31	1055	848	852	848
query32	94	58	56	56
query33	766	295	304	295
query34	1146	537	518	518
query35	913	745	749	745
query36	1121	971	961	961
query37	154	79	74	74
query38	4346	4161	4334	4161
query39	1507	1494	1412	1412
query40	211	104	103	103
query41	45	44	44	44
query42	120	108	104	104
query43	543	507	502	502
query44	1317	864	847	847
query45	189	197	172	172
query46	1187	763	750	750
query47	1983	1903	1908	1903
query48	444	336	338	336
query49	933	395	392	392
query50	843	392	383	383
query51	7317	7103	7240	7103
query52	99	94	91	91
query53	266	185	183	183
query54	1172	415	421	415
query55	78	88	79	79
query56	258	242	246	242
query57	1276	1163	1171	1163
query58	245	226	244	226
query59	3439	3410	3137	3137
query60	288	285	270	270
query61	137	153	135	135
query62	906	685	695	685
query63	217	180	184	180
query64	4068	699	655	655
query65	3274	3204	3224	3204
query66	1139	304	321	304
query67	15883	15653	15447	15447
query68	4842	570	567	567
query69	445	263	251	251
query70	1196	1123	1136	1123
query71	409	252	247	247
query72	6975	4118	4020	4020
query73	784	359	363	359
query74	10105	8772	8898	8772
query75	3422	2671	2668	2668
query76	2874	1189	1067	1067
query77	378	269	272	269
query78	10367	9351	9435	9351
query79	1108	598	674	598
query80	851	439	439	439
query81	532	226	231	226
query82	373	117	119	117
query83	266	154	151	151
query84	237	73	66	66
query85	1188	312	346	312
query86	373	303	312	303
query87	4486	4261	4326	4261
query88	3651	2235	2211	2211
query89	414	286	292	286
query90	1988	193	189	189
query91	141	104	108	104
query92	60	51	51	51
query93	1592	554	550	550
query94	850	283	285	283
query95	352	250	249	249
query96	621	274	283	274
query97	2830	2702	2678	2678
query98	212	196	197	196
query99	1526	1315	1315	1315
Total cold run time: 301431 ms
Total hot run time: 196134 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 424bbf8f5cf625d8f59ad13089bb0c6d37a38f2e, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.11	0.11
query5	0.41	0.39	0.41
query6	1.16	0.66	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.52	0.52
query10	0.55	0.55	0.54
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.59
query14	2.70	2.73	2.71
query15	0.90	0.83	0.83
query16	0.39	0.39	0.37
query17	1.08	1.08	1.02
query18	0.23	0.23	0.22
query19	1.85	1.89	2.00
query20	0.01	0.01	0.01
query21	15.37	0.59	0.58
query22	2.97	2.16	1.63
query23	16.95	1.16	0.84
query24	3.02	1.50	1.99
query25	0.26	0.24	0.06
query26	0.51	0.13	0.14
query27	0.05	0.05	0.04
query28	9.51	1.11	1.08
query29	12.65	3.27	3.23
query30	0.24	0.06	0.06
query31	2.87	0.39	0.39
query32	3.24	0.46	0.47
query33	3.23	3.16	3.02
query34	16.87	4.45	4.54
query35	4.48	4.48	4.49
query36	0.70	0.48	0.49
query37	0.10	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.03	0.02
query40	0.17	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 106.17 s
Total hot run time: 32.95 s

@justfortaste
Copy link
Contributor Author

runbuild all

Copy link
Contributor

@lide-reed lide-reed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 24, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@lide-reed lide-reed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@liutang123
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32776 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bb3c295d2aee0e08c15186bc3163bd3ad1988481, data reload: false

------ Round 1 ----------------------------------
q1	17587	6157	6060	6060
q2	2056	293	159	159
q3	10441	1261	732	732
q4	10207	864	423	423
q5	7525	2188	1958	1958
q6	205	183	144	144
q7	897	778	609	609
q8	9247	1347	1146	1146
q9	5225	5003	4991	4991
q10	6770	2306	1902	1902
q11	478	281	254	254
q12	350	359	219	219
q13	18109	3674	3017	3017
q14	244	240	220	220
q15	584	511	516	511
q16	617	621	583	583
q17	556	842	314	314
q18	7026	6434	6826	6434
q19	1644	980	559	559
q20	305	324	198	198
q21	2901	2309	2025	2025
q22	383	352	318	318
Total cold run time: 103357 ms
Total hot run time: 32776 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6345	6312	6388	6312
q2	244	324	237	237
q3	2257	2811	2488	2488
q4	1436	1864	1410	1410
q5	4562	4908	5106	4908
q6	191	180	143	143
q7	2145	2095	1927	1927
q8	2708	2841	2758	2758
q9	7380	7375	7177	7177
q10	3020	3407	2854	2854
q11	597	506	492	492
q12	706	753	567	567
q13	3356	3761	3054	3054
q14	289	298	280	280
q15	574	546	530	530
q16	676	686	636	636
q17	1186	1736	1261	1261
q18	7552	7525	7433	7433
q19	803	1178	1070	1070
q20	1938	2035	1908	1908
q21	5718	5411	4980	4980
q22	614	604	559	559
Total cold run time: 54297 ms
Total hot run time: 52984 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197667 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bb3c295d2aee0e08c15186bc3163bd3ad1988481, data reload: false

query1	1300	1011	917	917
query2	6331	2374	2310	2310
query3	11086	4856	5049	4856
query4	33492	24125	23368	23368
query5	5362	632	456	456
query6	281	201	211	201
query7	3992	489	305	305
query8	291	245	235	235
query9	9742	2731	2731	2731
query10	491	297	238	238
query11	18036	15521	15184	15184
query12	158	103	106	103
query13	1629	541	394	394
query14	11105	7479	6947	6947
query15	238	204	193	193
query16	7916	658	443	443
query17	1491	754	586	586
query18	2055	410	321	321
query19	197	182	177	177
query20	130	122	114	114
query21	206	130	109	109
query22	4759	4560	4524	4524
query23	34268	33612	33496	33496
query24	6302	2359	2338	2338
query25	474	462	382	382
query26	874	266	160	160
query27	2097	469	341	341
query28	5696	2534	2488	2488
query29	607	558	422	422
query30	209	181	150	150
query31	980	958	844	844
query32	72	63	58	58
query33	481	359	304	304
query34	778	853	513	513
query35	828	832	761	761
query36	1024	1060	974	974
query37	117	105	76	76
query38	4463	4363	4216	4216
query39	1585	1471	1435	1435
query40	221	123	113	113
query41	48	49	45	45
query42	121	103	102	102
query43	533	523	494	494
query44	1382	829	843	829
query45	189	186	183	183
query46	921	1062	651	651
query47	2036	2029	1927	1927
query48	396	421	323	323
query49	728	468	397	397
query50	632	679	404	404
query51	7135	7339	7190	7190
query52	101	101	95	95
query53	237	252	182	182
query54	475	495	452	452
query55	87	81	80	80
query56	255	280	253	253
query57	1226	1215	1195	1195
query58	230	235	226	226
query59	3124	3298	3219	3219
query60	285	267	248	248
query61	102	114	114	114
query62	896	798	758	758
query63	230	186	196	186
query64	3289	1038	684	684
query65	3332	3272	3245	3245
query66	1051	411	317	317
query67	16985	15828	15530	15530
query68	9780	763	526	526
query69	497	287	250	250
query70	1209	1141	1143	1141
query71	462	285	252	252
query72	5934	3862	3821	3821
query73	863	750	360	360
query74	10319	9324	9298	9298
query75	4811	3138	2693	2693
query76	5620	1187	768	768
query77	1044	356	269	269
query78	10155	10197	9955	9955
query79	3571	888	589	589
query80	686	509	441	441
query81	492	260	224	224
query82	460	144	121	121
query83	196	167	152	152
query84	280	95	66	66
query85	754	362	294	294
query86	441	304	305	304
query87	4671	4509	4503	4503
query88	3536	2239	2207	2207
query89	411	331	310	310
query90	2036	187	181	181
query91	134	140	104	104
query92	71	57	50	50
query93	1361	886	545	545
query94	661	400	289	289
query95	331	265	247	247
query96	487	607	279	279
query97	2765	2853	2651	2651
query98	233	203	197	197
query99	1680	1555	1468	1468
Total cold run time: 303341 ms
Total hot run time: 197667 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.46 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bb3c295d2aee0e08c15186bc3163bd3ad1988481, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.08	0.08
query4	1.61	0.11	0.11
query5	0.41	0.44	0.43
query6	1.16	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.50	0.50
query10	0.55	0.58	0.56
query11	0.15	0.10	0.10
query12	0.13	0.11	0.10
query13	0.61	0.61	0.61
query14	2.74	2.77	2.84
query15	0.88	0.83	0.83
query16	0.38	0.38	0.38
query17	1.06	1.06	1.05
query18	0.22	0.20	0.21
query19	1.87	1.88	2.00
query20	0.01	0.00	0.01
query21	15.38	0.92	0.57
query22	0.75	0.86	0.70
query23	15.22	1.45	0.58
query24	3.30	0.86	1.25
query25	0.17	0.11	0.08
query26	0.29	0.15	0.14
query27	0.05	0.06	0.05
query28	14.01	1.55	1.06
query29	12.57	3.97	3.26
query30	0.25	0.09	0.07
query31	2.82	0.60	0.39
query32	3.23	0.56	0.48
query33	3.00	3.05	3.08
query34	16.65	5.15	4.48
query35	4.51	4.54	4.52
query36	0.65	0.49	0.51
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.03	0.04	0.03
Total cold run time: 106.09 s
Total hot run time: 31.46 s

@lide-reed lide-reed merged commit 8e57386 into apache:master Dec 25, 2024
24 of 27 checks passed
@w41ter
Copy link
Contributor

w41ter commented Dec 25, 2024

@justfortaste Please update the doris_website.

Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Dec 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Support backup and restore privileges
9 participants