Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](nereids) support one phase DeferMaterializeTopN #45693

Merged
merged 1 commit into from
Dec 20, 2024

Conversation

englefly
Copy link
Contributor

@englefly englefly commented Dec 20, 2024

What problem does this PR solve?

make DeferMaterializeTopN support one phase sort.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 20, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 40062 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6ffb388e98384adf4d2f37476afe6fefae809cc0, data reload: false

------ Round 1 ----------------------------------
q1	17633	7513	7268	7268
q2	2038	181	174	174
q3	10746	1171	1153	1153
q4	10547	692	756	692
q5	7609	2725	2675	2675
q6	239	152	152	152
q7	990	619	624	619
q8	9262	1881	1924	1881
q9	6683	6476	6492	6476
q10	7050	2306	2312	2306
q11	471	264	266	264
q12	411	224	222	222
q13	17794	2969	2973	2969
q14	254	213	216	213
q15	558	505	507	505
q16	683	589	604	589
q17	977	516	545	516
q18	7406	6773	6711	6711
q19	1355	967	1032	967
q20	468	185	188	185
q21	4031	3212	3235	3212
q22	393	320	313	313
Total cold run time: 107598 ms
Total hot run time: 40062 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7234	7262	7266	7262
q2	331	229	233	229
q3	2920	2791	2764	2764
q4	2003	1750	1708	1708
q5	5379	5383	5423	5383
q6	224	139	142	139
q7	2142	1745	1686	1686
q8	3273	3414	3436	3414
q9	8715	8623	8677	8623
q10	3534	3426	3432	3426
q11	594	501	510	501
q12	772	572	577	572
q13	13362	3005	2972	2972
q14	290	264	266	264
q15	548	493	508	493
q16	676	655	647	647
q17	1795	1575	1583	1575
q18	7923	7469	7504	7469
q19	1692	1568	1563	1563
q20	2059	1859	1837	1837
q21	5417	5328	5336	5328
q22	621	567	595	567
Total cold run time: 71504 ms
Total hot run time: 58422 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189687 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6ffb388e98384adf4d2f37476afe6fefae809cc0, data reload: false

query1	976	380	383	380
query2	4160	2417	2324	2324
query3	6547	227	214	214
query4	33932	23395	23530	23395
query5	3603	469	462	462
query6	275	194	194	194
query7	4551	308	296	296
query8	292	229	229	229
query9	9345	2692	2691	2691
query10	447	249	255	249
query11	18333	15185	15262	15185
query12	151	107	117	107
query13	1659	419	414	414
query14	10168	6916	7196	6916
query15	292	177	190	177
query16	7342	478	468	468
query17	1600	601	580	580
query18	1999	314	313	313
query19	388	158	156	156
query20	125	111	112	111
query21	213	107	109	107
query22	4528	4377	4153	4153
query23	37013	34019	33528	33528
query24	11421	2397	2431	2397
query25	546	380	389	380
query26	1105	152	154	152
query27	2683	333	324	324
query28	7792	2422	2422	2422
query29	660	396	405	396
query30	272	147	147	147
query31	1022	819	813	813
query32	76	60	57	57
query33	741	285	288	285
query34	931	505	512	505
query35	866	754	743	743
query36	1110	942	914	914
query37	205	77	72	72
query38	4094	4096	4318	4096
query39	1487	1444	1428	1428
query40	286	100	103	100
query41	49	45	45	45
query42	121	99	98	98
query43	532	508	486	486
query44	1241	816	818	816
query45	181	165	172	165
query46	1169	698	717	698
query47	1979	1823	1838	1823
query48	417	319	332	319
query49	1141	387	387	387
query50	806	380	383	380
query51	7256	7043	6959	6959
query52	101	97	85	85
query53	250	181	182	181
query54	1180	396	423	396
query55	81	81	78	78
query56	239	234	253	234
query57	1210	1130	1141	1130
query58	229	225	233	225
query59	3317	2966	2942	2942
query60	277	268	253	253
query61	114	107	111	107
query62	852	684	695	684
query63	215	189	187	187
query64	3928	689	643	643
query65	3291	3212	3220	3212
query66	1053	366	299	299
query67	15841	15633	15495	15495
query68	6539	542	529	529
query69	445	250	254	250
query70	1220	1145	1143	1143
query71	472	250	246	246
query72	6167	4080	4022	4022
query73	808	363	357	357
query74	9599	8828	8852	8828
query75	3884	2616	2668	2616
query76	4395	1030	1107	1030
query77	718	264	275	264
query78	10200	9455	9370	9370
query79	1102	614	584	584
query80	846	435	453	435
query81	521	227	231	227
query82	350	122	121	121
query83	228	149	157	149
query84	239	72	70	70
query85	1316	300	300	300
query86	357	312	304	304
query87	4463	4330	4317	4317
query88	3958	2266	2165	2165
query89	409	290	290	290
query90	2220	185	186	185
query91	140	106	109	106
query92	64	55	52	52
query93	1502	529	533	529
query94	1110	287	301	287
query95	368	254	245	245
query96	612	291	279	279
query97	2824	2720	2684	2684
query98	229	194	201	194
query99	1543	1332	1302	1302
Total cold run time: 299154 ms
Total hot run time: 189687 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.55 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6ffb388e98384adf4d2f37476afe6fefae809cc0, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.60	0.10	0.10
query5	0.43	0.40	0.42
query6	1.18	0.66	0.66
query7	0.02	0.01	0.02
query8	0.04	0.04	0.02
query9	0.58	0.51	0.49
query10	0.56	0.58	0.56
query11	0.14	0.11	0.11
query12	0.14	0.11	0.11
query13	0.60	0.61	0.61
query14	2.72	2.74	2.82
query15	0.91	0.82	0.82
query16	0.38	0.39	0.38
query17	1.01	1.07	1.07
query18	0.20	0.20	0.20
query19	1.98	1.78	2.00
query20	0.01	0.01	0.01
query21	15.37	0.59	0.58
query22	2.60	1.46	1.64
query23	16.97	1.12	0.76
query24	3.16	1.17	1.59
query25	0.14	0.12	0.13
query26	0.55	0.15	0.14
query27	0.05	0.05	0.05
query28	9.97	1.10	1.09
query29	12.54	3.33	3.24
query30	0.25	0.07	0.06
query31	2.84	0.37	0.38
query32	3.28	0.46	0.45
query33	3.16	3.07	3.13
query34	17.05	4.51	4.53
query35	4.55	4.51	4.51
query36	0.65	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.46 s
Total hot run time: 32.55 s

@englefly englefly merged commit e81ecb5 into apache:master Dec 20, 2024
30 of 31 checks passed
@englefly englefly deleted the one-phase-defer-topn branch December 20, 2024 07:08
github-actions bot pushed a commit that referenced this pull request Dec 20, 2024
### What problem does this PR solve?
make DeferMaterializeTopN support one phase sort.
github-actions bot pushed a commit that referenced this pull request Dec 20, 2024
### What problem does this PR solve?
make DeferMaterializeTopN support one phase sort.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants