Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](Nereids) fix leading with multi level of brace pairs (#34169) #35606

Merged
merged 1 commit into from
May 30, 2024

Conversation

LiBinfeng-01
Copy link
Collaborator

@LiBinfeng-01 LiBinfeng-01 commented May 29, 2024

cherry-pick: #34169
fix leading with multi level of brace pairs
example:
leading(t1 {{t2 t3} {t4 t5}} t6) can be reduced to leading(t1 {t2 t3 {t4 t5}} t6) also update cases which remove project node from explain shape plan

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor

run buildall

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50564 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b538959e638a461106cf76511b864b44cb7263d6, data reload: false

------ Round 1 ----------------------------------
q1	17525	4392	4391	4391
q2	2034	192	152	152
q3	10242	1914	1932	1914
q4	10121	1236	1306	1236
q5	8399	3894	3972	3894
q6	240	127	125	125
q7	2067	1604	1582	1582
q8	9277	2721	2730	2721
q9	10672	10904	10864	10864
q10	8915	3543	3523	3523
q11	437	257	251	251
q12	475	309	300	300
q13	18327	3974	4063	3974
q14	360	334	333	333
q15	514	463	459	459
q16	672	568	588	568
q17	1141	954	960	954
q18	7296	6918	7060	6918
q19	1743	1590	1620	1590
q20	520	325	330	325
q21	4497	4139	4098	4098
q22	494	392	394	392
Total cold run time: 115968 ms
Total hot run time: 50564 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4353	4324	4359	4324
q2	320	225	222	222
q3	4149	4154	4150	4150
q4	2772	2737	2760	2737
q5	7130	7116	7133	7116
q6	239	122	122	122
q7	3263	2799	2806	2799
q8	4391	4470	4472	4470
q9	16908	16814	16905	16814
q10	4262	4265	4264	4264
q11	760	684	717	684
q12	1020	852	838	838
q13	5265	3728	3744	3728
q14	459	421	434	421
q15	533	466	445	445
q16	745	707	676	676
q17	3844	3879	3875	3875
q18	8828	8742	8855	8742
q19	1735	1724	1658	1658
q20	2440	2201	2101	2101
q21	8500	8466	8484	8466
q22	1057	944	943	943
Total cold run time: 82973 ms
Total hot run time: 79595 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203710 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b538959e638a461106cf76511b864b44cb7263d6, data reload: false

query1	943	391	389	389
query2	6600	2622	2476	2476
query3	6918	209	209	209
query4	20029	18063	18047	18047
query5	19742	6544	6519	6519
query6	284	219	223	219
query7	4153	310	335	310
query8	254	269	252	252
query9	3141	2727	2659	2659
query10	414	310	310	310
query11	11383	10633	10709	10633
query12	128	81	76	76
query13	5589	677	683	677
query14	17114	13078	13618	13078
query15	394	229	223	223
query16	6473	297	266	266
query17	1700	1471	880	880
query18	2307	421	408	408
query19	218	152	151	151
query20	83	81	81	81
query21	194	98	98	98
query22	5312	5015	5090	5015
query23	32617	32094	32248	32094
query24	7794	6556	6533	6533
query25	538	430	459	430
query26	957	169	166	166
query27	2210	301	303	301
query28	6187	2342	2310	2310
query29	2947	2882	2757	2757
query30	251	168	168	168
query31	900	766	745	745
query32	71	65	61	61
query33	402	261	249	249
query34	862	475	500	475
query35	1105	926	924	924
query36	1433	1193	1160	1160
query37	89	60	62	60
query38	3032	2912	2909	2909
query39	1416	1310	1317	1310
query40	321	97	98	97
query41	41	38	37	37
query42	89	88	88	88
query43	699	626	717	626
query44	1156	726	726	726
query45	250	233	233	233
query46	1241	972	969	969
query47	1845	1843	1802	1802
query48	1027	736	702	702
query49	658	386	365	365
query50	868	578	619	578
query51	4793	4716	4639	4639
query52	89	90	78	78
query53	455	331	325	325
query54	2661	2453	2452	2452
query55	98	76	84	76
query56	241	211	207	207
query57	1190	1099	1045	1045
query58	226	218	207	207
query59	4504	4030	4227	4030
query60	215	193	212	193
query61	97	94	92	92
query62	867	518	540	518
query63	496	345	343	343
query64	2751	1587	1500	1500
query65	3625	3584	3584	3584
query66	812	378	375	375
query67	16713	15029	15769	15029
query68	10846	654	659	654
query69	572	350	329	329
query70	2030	1403	1268	1268
query71	422	315	313	313
query72	6215	3456	3493	3456
query73	746	326	328	326
query74	6423	5881	5892	5881
query75	5559	3684	3701	3684
query76	6923	1168	1225	1168
query77	1196	261	262	261
query78	12480	11876	12233	11876
query79	10199	638	651	638
query80	968	406	402	402
query81	455	245	229	229
query82	623	102	99	99
query83	169	127	134	127
query84	263	71	75	71
query85	883	318	324	318
query86	342	321	289	289
query87	3253	3050	2997	2997
query88	4469	2350	2340	2340
query89	391	296	297	296
query90	1945	206	218	206
query91	177	140	149	140
query92	60	55	55	55
query93	5315	611	599	599
query94	745	207	209	207
query95	1116	1064	1056	1056
query96	646	329	330	329
query97	6579	6461	6584	6461
query98	195	184	176	176
query99	3035	915	940	915
Total cold run time: 319452 ms
Total hot run time: 203710 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.92 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b538959e638a461106cf76511b864b44cb7263d6, data reload: false

query1	0.03	0.03	0.02
query2	0.06	0.03	0.02
query3	0.26	0.05	0.04
query4	1.77	0.06	0.07
query5	0.53	0.52	0.52
query6	1.30	0.61	0.61
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.50	0.47
query10	0.55	0.53	0.54
query11	0.12	0.09	0.09
query12	0.13	0.09	0.09
query13	0.62	0.62	0.61
query14	0.78	0.79	0.79
query15	0.80	0.76	0.77
query16	0.36	0.36	0.36
query17	1.02	1.02	1.02
query18	0.23	0.24	0.26
query19	1.90	1.86	1.77
query20	0.02	0.01	0.01
query21	15.49	0.57	0.53
query22	2.02	2.59	1.64
query23	17.37	0.96	0.85
query24	5.69	1.49	1.30
query25	0.33	0.09	0.06
query26	0.73	0.17	0.16
query27	0.04	0.04	0.06
query28	6.63	0.78	0.72
query29	12.63	2.22	2.34
query30	0.63	0.53	0.52
query31	2.83	0.41	0.38
query32	3.33	0.50	0.50
query33	3.05	3.07	3.09
query34	15.24	4.80	4.77
query35	4.83	4.84	4.84
query36	1.07	1.02	1.02
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 103.35 s
Total hot run time: 30.92 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit b538959e638a461106cf76511b864b44cb7263d6 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@morrySnow morrySnow merged commit 47716bc into apache:branch-2.0 May 30, 2024
23 of 24 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
)

cherry-pick: apache#34169

fix leading with multi level of brace pairs. example:
leading(t1 {{t2 t3} {t4 t5}} t6)
can be reduced to
leading(t1 {t2 t3 {t4 t5}} t6)

also update cases which remove project node from explain shape plan
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants