Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement](Log) Reduce usage of log fatal(PART II) #47249

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

linrrzqqq
Copy link
Contributor

Issue Number: close #42445
Related PR: #45671

Replace the InternalError in these two PRs( #38863 and #38996 ) and the DCHECK in the execution layer with FatalError

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@linrrzqqq
Copy link
Contributor Author

run buildall

@linrrzqqq
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 30789 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b87038bd4ba61c8f242223f2b57229379b08713, data reload: false

------ Round 1 ----------------------------------
partsupp	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:44:20	NULL	utf-8	NULL	NULL	
region	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:24	NULL	utf-8	NULL	NULL	
revenue0	View	NULL	NULL	-1	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:24	NULL	NULL	utf-8	NULL	NULL	
supplier	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:25	NULL	utf-8	NULL	NULL	
============================================
q1	17593	5406	5114	5114
q2	2060	323	182	182
q3	10458	1217	743	743
q4	10267	966	540	540
q5	8184	2371	2152	2152
q6	q7	1011	731	623	623
q8	9259	1380	1229	1229
q9	5323	4945	4883	4883
q10	6842	2338	1903	1903
q11	q12	350	365	238	238
q13	17779	3669	3130	3130
q14	q15	541	489	478	478
q16	651	604	593	593
q17	q18	7436	6437	6540	6437
q19	q20	322	341	192	192
q21	2766	2265	2002	2002
q22	404	373	350	350
Total cold run time: 101246 ms
Total hot run time: 30789 ms

----- Round 2, with runtime_filter_mode=off -----
partsupp	Doris	NULL	NULL	80000000	56	4534120086	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:44:20	NULL	utf-8	NULL	NULL	
region	Doris	NULL	NULL	5	240	1201	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:24	NULL	utf-8	NULL	NULL	
revenue0	View	NULL	NULL	-1	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:24	NULL	NULL	utf-8	NULL	NULL	
supplier	Doris	NULL	NULL	1000000	87	87519212	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:25	NULL	utf-8	NULL	NULL	
============================================
q1	5182	5152	5117	5117
q2	248	344	251	251
q3	2263	2671	2291	2291
q4	1482	1866	1466	1466
q5	4303	4784	4627	4627
q6	q7	1990	1798	1810	1798
q8	2670	2868	2720	2720
q9	7311	7312	7336	7312
q10	3056	3271	2698	2698
q11	q12	688	787	618	618
q13	3502	3992	3350	3350
q14	q15	513	478	477	477
q16	664	673	636	636
q17	q18	7998	7404	7395	7395
q19	q20	2072	2049	1940	1940
q21	5867	5140	5052	5052
q22	671	641	643	641
Total cold run time: 50480 ms
Total hot run time: 48389 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196169 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b87038bd4ba61c8f242223f2b57229379b08713, data reload: false

============================================
query1	994	439	413	413
query2	6528	3235	3301	3235
query3	6797	218	217	217
query4	32348	23930	23835	23835
query5	4395	617	486	486
query6	294	201	189	189
query7	4604	510	335	335
query8	301	258	249	249
query9	9641	2862	2827	2827
query10	481	324	260	260
query11	18342	15818	15411	15411
query12	155	108	109	108
query13	1677	537	378	378
query14	9936	7974	7143	7143
query15	269	214	187	187
query16	8070	647	497	497
query17	1608	744	580	580
query18	2132	425	315	315
query19	242	194	166	166
query20	126	129	130	129
query21	217	125	106	106
query22	5670	5941	5615	5615
query23	34578	33809	34105	33809
query24	6687	2452	2421	2421
query25	512	489	425	425
query26	1186	289	168	168
query27	1997	499	362	362
query28	5284	2518	2485	2485
query29	718	568	428	428
query30	203	195	161	161
query31	1029	923	855	855
query32	82	67	60	60
query33	511	390	363	363
query34	773	906	571	571
query35	819	850	803	803
query36	1293	1326	1290	1290
query37	119	108	82	82
query38	4599	4587	4479	4479
query39	1480	1450	1487	1450
query40	206	121	116	116
query41	52	52	50	50
query42	134	114	112	112
query43	641	619	616	616
query44	1384	846	851	846
query45	188	175	172	172
query46	934	1078	676	676
query47	2115	2162	2076	2076
query48	393	426	336	336
query49	781	494	411	411
query50	676	691	405	405
query51	7256	7241	7090	7090
query52	105	103	92	92
query53	233	254	190	190
query54	477	507	413	413
query55	80	122	75	75
query56	252	263	249	249
query57	1269	1290	1228	1228
query58	283	231	243	231
query59	3152	3095	3140	3095
query60	284	273	260	260
query61	141	112	117	112
query62	795	728	681	681
query63	230	200	193	193
query64	4195	1039	727	727
query65	3416	3348	3328	3328
query66	1002	430	318	318
query67	18512	17500	17209	17209
query68	10104	838	553	553
query69	466	325	282	282
query70	1607	1420	1526	1420
query71	399	281	261	261
query72	6783	3868	4026	3868
query73	2060	804	383	383
query74	10232	9591	8875	8875
query75	4011	3294	2777	2777
query76	3794	1243	802	802
query77	567	375	287	287
query78	9827	10046	9278	9278
query79	2122	895	637	637
query80	807	534	475	475
query81	557	289	260	260
query82	517	160	131	131
query83	195	194	165	165
query84	274	96	79	79
query85	789	356	305	305
query86	420	364	393	364
query87	4740	4887	4994	4887
query88	3605	2292	2209	2209
query89	436	366	336	336
query90	1829	193	197	193
query91	138	154	128	128
query92	60	60	56	56
query93	1373	944	568	568
query94	677	442	272	272
query95	348	286	272	272
query96	502	634	298	298
query97	query98	251	213	209	209
query99	1424	1441	1339	1339
Total cold run time: 296797 ms
Total hot run time: 196169 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.52% (10819/26060)
Line Coverage: 31.86% (91453/287069)
Region Coverage: 31.09% (47104/151527)
Branch Coverage: 27.13% (23849/87912)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3b87038bd4ba61c8f242223f2b57229379b08713_3b87038bd4ba61c8f242223f2b57229379b08713/report/index.html

@linrrzqqq linrrzqqq marked this pull request as ready for review January 20, 2025 20:09
@linrrzqqq
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3b87038bd4ba61c8f242223f2b57229379b08713, data reload: false

------ Round 1 ----------------------------------
partsupp	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:44:20	NULL	utf-8	NULL	NULL	
region	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:24	NULL	utf-8	NULL	NULL	
revenue0	View	NULL	NULL	-1	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:24	NULL	NULL	utf-8	NULL	NULL	
supplier	Doris	NULL	NULL	0	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:25	NULL	utf-8	NULL	NULL	
============================================
q1	17737	5437	5185	5185
q2	2055	305	164	164
q3	10500	1266	740	740
q4	10225	1000	574	574
q5	7774	2456	2190	2190
q6	q7	1029	724	615	615
q8	9257	1432	1229	1229
q9	5268	4921	5074	4921
q10	6876	2346	1919	1919
q11	q12	351	366	245	245
q13	17784	3720	3097	3097
q14	q15	518	458	478	458
q16	642	632	595	595
q17	q18	7632	6479	6496	6479
q19	q20	339	361	196	196
q21	2980	2174	2080	2080
q22	402	384	361	361
Total cold run time: 101369 ms
Total hot run time: 31048 ms

----- Round 2, with runtime_filter_mode=off -----
partsupp	Doris	NULL	NULL	80000000	56	4534120086	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:44:20	NULL	utf-8	NULL	NULL	
region	Doris	NULL	NULL	5	240	1201	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:24	NULL	utf-8	NULL	NULL	
revenue0	View	NULL	NULL	-1	0	0	NULL	NULL	NULL	NULL	2023-12-26 18:27:24	NULL	NULL	utf-8	NULL	NULL	
supplier	Doris	NULL	NULL	1000000	87	87519212	NULL	NULL	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:27:25	NULL	utf-8	NULL	NULL	
============================================
q1	5266	5136	5169	5136
q2	250	342	248	248
q3	2274	2670	2325	2325
q4	1539	1845	1443	1443
q5	4314	4798	4681	4681
q6	q7	2015	1902	1813	1813
q8	2581	2727	2704	2704
q9	6927	6850	6934	6850
q10	2906	3246	2685	2685
q11	q12	679	710	614	614
q13	3324	3756	3077	3077
q14	q15	504	475	458	458
q16	660	690	650	650
q17	q18	7596	7133	6939	6939
q19	q20	1931	2031	1845	1845
q21	5648	5330	5047	5047
q22	666	682	634	634
Total cold run time: 49080 ms
Total hot run time: 47149 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193652 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3b87038bd4ba61c8f242223f2b57229379b08713, data reload: false

============================================
query1	998	391	406	391
query2	6549	2287	2153	2153
query3	6804	218	215	215
query4	34647	24582	23873	23873
query5	4325	597	487	487
query6	304	219	213	213
query7	4596	513	323	323
query8	305	275	247	247
query9	9648	2836	2823	2823
query10	456	320	260	260
query11	18635	15764	15592	15592
query12	152	113	105	105
query13	1656	550	384	384
query14	10370	7634	7474	7474
query15	241	202	205	202
query16	7984	667	494	494
query17	1598	758	580	580
query18	2020	407	331	331
query19	234	190	174	174
query20	132	119	112	112
query21	213	130	108	108
query22	4752	5207	4990	4990
query23	34635	33456	33427	33427
query24	6623	2510	2407	2407
query25	493	460	392	392
query26	1201	276	161	161
query27	2012	479	360	360
query28	5397	2446	2423	2423
query29	744	558	436	436
query30	236	194	164	164
query31	971	900	863	863
query32	83	70	62	62
query33	560	360	293	293
query34	766	867	543	543
query35	839	837	769	769
query36	1279	1232	1220	1220
query37	132	106	84	84
query38	4325	4272	4239	4239
query39	1490	1415	1432	1415
query40	214	115	103	103
query41	54	52	58	52
query42	125	110	109	109
query43	605	634	578	578
query44	1345	813	833	813
query45	204	175	164	164
query46	910	1046	663	663
query47	2015	2030	1949	1949
query48	384	419	334	334
query49	787	496	414	414
query50	641	689	387	387
query51	7111	7171	6931	6931
query52	107	103	99	99
query53	233	258	196	196
query54	483	513	422	422
query55	80	82	80	80
query56	270	274	263	263
query57	1260	1238	1187	1187
query58	264	243	246	243
query59	3133	3184	3156	3156
query60	277	273	263	263
query61	121	123	125	123
query62	813	719	675	675
query63	225	203	199	199
query64	4526	1051	689	689
query65	3339	3290	3285	3285
query66	864	411	334	334
query67	17512	17189	17217	17189
query68	3868	880	560	560
query69	450	308	278	278
query70	1491	1474	1521	1474
query71	388	293	260	260
query72	5135	3953	4012	3953
query73	666	763	377	377
query74	9963	9293	8906	8906
query75	3268	3302	2777	2777
query76	3104	1179	815	815
query77	476	397	290	290
query78	10155	10027	9431	9431
query79	2334	844	662	662
query80	692	547	466	466
query81	506	296	241	241
query82	220	156	123	123
query83	171	185	158	158
query84	242	97	76	76
query85	771	360	306	306
query86	424	391	380	380
query87	4635	4664	4685	4664
query88	3776	2188	2147	2147
query89	417	355	333	333
query90	1830	193	196	193
query91	165	143	121	121
query92	61	60	53	53
query93	1308	881	554	554
query94	621	416	305	305
query95	335	273	274	273
query96	486	611	291	291
query97	query98	238	209	208	208
query99	1318	1390	1288	1288
Total cold run time: 285821 ms
Total hot run time: 193652 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.57% (10834/26060)
Line Coverage: 31.92% (91622/287069)
Region Coverage: 31.14% (47180/151532)
Branch Coverage: 27.16% (23883/87922)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3b87038bd4ba61c8f242223f2b57229379b08713_3b87038bd4ba61c8f242223f2b57229379b08713/report/index.html

@linrrzqqq
Copy link
Contributor Author

run external

@linrrzqqq
Copy link
Contributor Author

run p0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants