Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](profile) add profile about filtered rows in analytic operator #44785

Closed
wants to merge 1 commit into from

Conversation

zhangstar333
Copy link
Contributor

What problem does this PR solve?

Problem Summary:
add profile about filtered rows in analytic operator

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TPC-H: Total hot run time: 40035 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4499d065708fa2088dbcc312556e91f27c6746d, data reload: false

------ Round 1 ----------------------------------
q1	17598	7438	7267	7267
q2	2049	173	162	162
q3	10624	1104	1181	1104
q4	10571	750	765	750
q5	7592	2732	2687	2687
q6	244	146	149	146
q7	991	621	606	606
q8	9232	1816	1906	1816
q9	6687	6483	6446	6446
q10	6947	2299	2308	2299
q11	463	257	257	257
q12	426	216	219	216
q13	17763	3001	3017	3001
q14	244	209	222	209
q15	580	537	508	508
q16	617	596	574	574
q17	975	534	545	534
q18	7199	6761	6782	6761
q19	1335	1034	1009	1009
q20	466	175	190	175
q21	4024	3196	3226	3196
q22	392	316	312	312
Total cold run time: 107019 ms
Total hot run time: 40035 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7276	7284	7298	7284
q2	329	232	233	232
q3	2860	2810	2966	2810
q4	2059	1735	1739	1735
q5	5682	5634	5648	5634
q6	231	141	140	140
q7	2288	1795	1845	1795
q8	3373	3615	3511	3511
q9	9038	9080	8999	8999
q10	3606	3536	3560	3536
q11	588	509	503	503
q12	830	630	620	620
q13	10238	3268	3275	3268
q14	328	270	266	266
q15	580	532	516	516
q16	694	637	663	637
q17	1867	1654	1613	1613
q18	8310	7733	7657	7657
q19	1711	1559	1577	1559
q20	2098	1896	1931	1896
q21	5615	5437	5507	5437
q22	663	592	571	571
Total cold run time: 70264 ms
Total hot run time: 60219 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.33% (9976/26028)
Line Coverage: 29.41% (83547/284034)
Region Coverage: 28.52% (42967/150636)
Branch Coverage: 25.14% (21819/86774)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e4499d065708fa2088dbcc312556e91f27c6746d_e4499d065708fa2088dbcc312556e91f27c6746d/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 197854 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e4499d065708fa2088dbcc312556e91f27c6746d, data reload: false

query1	1313	948	971	948
query2	6238	2116	2089	2089
query3	10972	4455	4453	4453
query4	66494	28557	23812	23812
query5	5000	469	447	447
query6	416	181	177	177
query7	5658	314	298	298
query8	316	230	227	227
query9	9206	2634	2638	2634
query10	457	250	241	241
query11	17700	15197	15857	15197
query12	151	108	102	102
query13	1593	418	406	406
query14	9794	7407	7514	7407
query15	221	189	190	189
query16	7167	457	432	432
query17	1278	591	592	591
query18	1845	316	321	316
query19	220	158	160	158
query20	122	124	113	113
query21	212	106	104	104
query22	4731	4639	4539	4539
query23	34741	34245	34236	34236
query24	5304	2677	2493	2493
query25	518	374	378	374
query26	655	147	147	147
query27	2144	273	280	273
query28	4184	2493	2445	2445
query29	673	415	425	415
query30	213	148	156	148
query31	1022	825	806	806
query32	68	55	57	55
query33	391	289	286	286
query34	938	531	509	509
query35	891	775	775	775
query36	1119	956	980	956
query37	123	73	75	73
query38	4489	4355	4359	4355
query39	1547	1456	1466	1456
query40	204	102	99	99
query41	45	42	40	40
query42	104	91	102	91
query43	520	501	496	496
query44	1184	820	837	820
query45	181	171	165	165
query46	1156	715	737	715
query47	2063	1939	1921	1921
query48	419	315	340	315
query49	729	388	385	385
query50	832	390	389	389
query51	7439	7237	7274	7237
query52	97	89	87	87
query53	251	174	172	172
query54	505	374	386	374
query55	76	84	72	72
query56	246	224	228	224
query57	1285	1108	1098	1098
query58	221	211	208	208
query59	3218	3139	3150	3139
query60	255	241	235	235
query61	107	108	104	104
query62	807	651	697	651
query63	212	184	195	184
query64	1407	658	623	623
query65	3294	3227	3184	3184
query66	698	297	303	297
query67	16005	15771	15776	15771
query68	4117	560	561	560
query69	425	250	257	250
query70	1244	1121	1053	1053
query71	352	256	257	256
query72	6424	4088	4100	4088
query73	767	349	357	349
query74	10251	9056	9140	9056
query75	3405	2622	2637	2622
query76	1785	1110	1048	1048
query77	513	286	286	286
query78	10515	9441	9466	9441
query79	1522	591	597	591
query80	872	428	445	428
query81	503	232	234	232
query82	1302	116	126	116
query83	256	148	155	148
query84	282	75	68	68
query85	885	293	302	293
query86	361	297	307	297
query87	4682	4657	4515	4515
query88	3582	2220	2210	2210
query89	435	291	290	290
query90	2001	187	183	183
query91	137	101	102	101
query92	63	50	49	49
query93	1880	536	575	536
query94	793	282	288	282
query95	348	248	249	248
query96	623	283	283	283
query97	2839	2681	2728	2681
query98	225	203	194	194
query99	1714	1335	1322	1322
Total cold run time: 318787 ms
Total hot run time: 197854 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e4499d065708fa2088dbcc312556e91f27c6746d, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.62	0.11	0.11
query5	0.40	0.41	0.42
query6	1.18	0.67	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.57	0.50	0.51
query10	0.55	0.55	0.55
query11	0.14	0.11	0.11
query12	0.15	0.11	0.11
query13	0.61	0.62	0.60
query14	2.71	2.86	2.73
query15	0.90	0.82	0.83
query16	0.40	0.38	0.37
query17	1.07	1.00	1.02
query18	0.23	0.21	0.20
query19	1.96	1.88	2.05
query20	0.01	0.02	0.01
query21	15.36	0.59	0.62
query22	2.87	2.66	1.57
query23	17.06	0.81	0.87
query24	2.74	0.99	1.71
query25	0.35	0.08	0.07
query26	0.41	0.15	0.14
query27	0.05	0.03	0.05
query28	10.79	1.10	1.07
query29	12.52	3.22	3.24
query30	0.25	0.06	0.06
query31	2.85	0.38	0.38
query32	3.27	0.46	0.47
query33	3.02	3.08	3.05
query34	16.81	4.49	4.47
query35	4.53	4.46	4.46
query36	0.69	0.51	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.12	0.13
query41	0.07	0.02	0.03
query42	0.03	0.02	0.03
query43	0.03	0.04	0.03
Total cold run time: 106.94 s
Total hot run time: 32.4 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.82% (10124/26082)
Line Coverage: 29.76% (85079/285875)
Region Coverage: 28.81% (43640/151472)
Branch Coverage: 25.36% (22169/87428)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e4499d065708fa2088dbcc312556e91f27c6746d_e4499d065708fa2088dbcc312556e91f27c6746d/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants