Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[minor](profile) Add intermediate projection time in do_projections #44727

Closed

Conversation

Mryange
Copy link
Contributor

@Mryange Mryange commented Nov 28, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange
Copy link
Contributor Author

Mryange commented Nov 28, 2024

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.33% (9976/26025)
Line Coverage: 29.43% (83513/283770)
Region Coverage: 28.58% (42995/150442)
Branch Coverage: 25.19% (21836/86672)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3db1dd934461d0f8c00ca8ca86446848433a23b8_3db1dd934461d0f8c00ca8ca86446848433a23b8/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39921 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3db1dd934461d0f8c00ca8ca86446848433a23b8, data reload: false

------ Round 1 ----------------------------------
q1	17598	7509	7267	7267
q2	2042	172	169	169
q3	10604	1113	1200	1113
q4	10552	735	754	735
q5	7607	2693	2683	2683
q6	240	150	145	145
q7	993	652	592	592
q8	9245	1861	1971	1861
q9	6551	6424	6459	6424
q10	7001	2311	2317	2311
q11	477	258	252	252
q12	430	219	217	217
q13	17779	3012	3014	3012
q14	254	215	209	209
q15	580	537	515	515
q16	672	586	602	586
q17	978	519	522	519
q18	7210	6785	6735	6735
q19	1329	971	970	970
q20	474	181	179	179
q21	3994	3271	3119	3119
q22	389	321	308	308
Total cold run time: 106999 ms
Total hot run time: 39921 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7277	7297	7281	7281
q2	329	228	232	228
q3	2946	2892	2970	2892
q4	2055	1786	1830	1786
q5	5717	5679	5668	5668
q6	229	140	144	140
q7	2263	1866	1787	1787
q8	3454	3571	3492	3492
q9	8835	8918	8912	8912
q10	3589	3572	3552	3552
q11	597	507	513	507
q12	838	642	640	640
q13	11119	3254	3244	3244
q14	299	270	274	270
q15	582	541	535	535
q16	706	633	653	633
q17	1855	1636	1611	1611
q18	8216	7768	7454	7454
q19	1734	1601	1419	1419
q20	2118	1882	1861	1861
q21	5628	5486	5480	5480
q22	658	564	545	545
Total cold run time: 71044 ms
Total hot run time: 59937 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198358 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3db1dd934461d0f8c00ca8ca86446848433a23b8, data reload: false

query1	1234	958	920	920
query2	6253	2117	2021	2021
query3	10955	4143	4063	4063
query4	68129	28472	23902	23902
query5	4983	487	464	464
query6	411	186	189	186
query7	5527	299	289	289
query8	318	233	226	226
query9	8871	2762	2725	2725
query10	434	240	243	240
query11	17646	15388	15891	15388
query12	153	105	100	100
query13	1470	418	427	418
query14	10652	7563	7588	7563
query15	205	182	185	182
query16	7092	460	450	450
query17	1283	564	571	564
query18	1830	297	301	297
query19	204	146	147	146
query20	121	119	113	113
query21	204	104	111	104
query22	4886	4837	4663	4663
query23	35338	34553	34621	34553
query24	5560	2558	2463	2463
query25	500	399	387	387
query26	644	150	155	150
query27	1725	294	297	294
query28	4177	2502	2482	2482
query29	718	449	442	442
query30	215	153	150	150
query31	1040	848	862	848
query32	70	61	54	54
query33	428	287	305	287
query34	929	543	536	536
query35	879	784	762	762
query36	1077	963	975	963
query37	128	86	78	78
query38	4654	4528	4409	4409
query39	1544	1477	1500	1477
query40	205	103	98	98
query41	45	40	44	40
query42	107	99	96	96
query43	533	490	492	490
query44	1210	855	827	827
query45	180	173	168	168
query46	1167	717	718	717
query47	2037	1988	1976	1976
query48	423	324	314	314
query49	714	378	433	378
query50	858	413	399	399
query51	7455	7275	7075	7075
query52	96	87	89	87
query53	252	186	179	179
query54	508	409	390	390
query55	78	75	78	75
query56	256	234	243	234
query57	1330	1182	1173	1173
query58	212	203	210	203
query59	3243	3114	3096	3096
query60	266	247	232	232
query61	105	106	106	106
query62	793	660	697	660
query63	216	187	192	187
query64	1450	660	644	644
query65	3285	3195	3251	3195
query66	742	296	306	296
query67	16010	15785	15686	15686
query68	3866	552	544	544
query69	438	253	254	253
query70	1164	1152	1140	1140
query71	336	251	255	251
query72	6528	4272	4007	4007
query73	768	360	355	355
query74	10188	9110	9011	9011
query75	3420	2657	2704	2657
query76	2166	1019	1047	1019
query77	475	334	271	271
query78	10583	9526	9341	9341
query79	1496	589	586	586
query80	890	437	422	422
query81	505	241	229	229
query82	1321	120	122	120
query83	258	147	147	147
query84	273	72	66	66
query85	945	300	289	289
query86	343	304	260	260
query87	4728	4623	4621	4621
query88	3724	2248	2186	2186
query89	415	296	290	290
query90	2031	190	186	186
query91	132	105	103	103
query92	70	50	50	50
query93	1943	533	529	529
query94	803	299	298	298
query95	350	255	247	247
query96	630	285	290	285
query97	2916	2736	2704	2704
query98	214	197	199	197
query99	1612	1316	1314	1314
Total cold run time: 321794 ms
Total hot run time: 198358 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3db1dd934461d0f8c00ca8ca86446848433a23b8, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.07	0.07
query4	1.62	0.10	0.11
query5	0.42	0.42	0.42
query6	1.18	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.53	0.52
query10	0.55	0.56	0.56
query11	0.14	0.11	0.10
query12	0.15	0.12	0.10
query13	0.61	0.60	0.59
query14	2.77	2.73	2.85
query15	0.91	0.82	0.81
query16	0.38	0.39	0.37
query17	1.05	0.97	1.04
query18	0.23	0.21	0.21
query19	1.94	1.85	1.96
query20	0.01	0.01	0.01
query21	15.36	0.58	0.56
query22	2.47	2.98	1.62
query23	17.26	0.96	0.86
query24	3.40	1.26	2.00
query25	0.20	0.08	0.08
query26	0.75	0.15	0.14
query27	0.05	0.04	0.05
query28	9.62	1.10	1.09
query29	12.54	3.27	3.23
query30	0.24	0.06	0.06
query31	2.87	0.39	0.39
query32	3.27	0.47	0.47
query33	2.97	2.98	3.02
query34	17.14	4.44	4.44
query35	4.52	4.48	4.53
query36	0.65	0.49	0.48
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.03
query40	0.16	0.14	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.72 s
Total hot run time: 32.67 s

@Mryange Mryange closed this Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants