Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](nereids) disable join reorder if any table row count is not available #40349 (2.0) #41425

Merged
merged 3 commits into from
Sep 29, 2024

Conversation

englefly
Copy link
Contributor

Proposed changes

pick #40349
Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

run buildall

@englefly
Copy link
Contributor Author

run buildall

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 48793 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9d7530e3e61db7b66c00e00b719794a2424b9e20, data reload: false

------ Round 1 ----------------------------------
q1	17697	4354	4324	4324
q2	2083	157	143	143
q3	10263	1903	1917	1903
q4	10378	1209	1290	1209
q5	8422	3848	3913	3848
q6	227	126	126	126
q7	2085	1601	1609	1601
q8	9518	2723	2695	2695
q9	9988	10045	9636	9636
q10	8634	3539	3496	3496
q11	425	243	246	243
q12	465	303	303	303
q13	18348	3970	4020	3970
q14	352	327	332	327
q15	513	462	468	462
q16	547	447	472	447
q17	1126	939	928	928
q18	7270	6921	6825	6825
q19	1707	1536	1508	1508
q20	508	325	296	296
q21	4371	4166	4111	4111
q22	477	393	392	392
Total cold run time: 115404 ms
Total hot run time: 48793 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4327	4385	4327
q2	329	221	225	221
q3	4206	4123	4160	4123
q4	2769	2774	2739	2739
q5	7197	7149	7111	7111
q6	237	122	120	120
q7	3232	2827	2830	2827
q8	4385	4444	4512	4444
q9	13738	13776	13673	13673
q10	4412	4489	4465	4465
q11	761	698	712	698
q12	1076	901	918	901
q13	7000	3784	3764	3764
q14	463	424	429	424
q15	495	471	469	469
q16	652	585	596	585
q17	3859	3751	3822	3751
q18	8872	8802	8828	8802
q19	1710	1644	1659	1644
q20	2455	2108	2105	2105
q21	8403	8374	8482	8374
q22	1028	908	920	908
Total cold run time: 81604 ms
Total hot run time: 76475 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212552 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9d7530e3e61db7b66c00e00b719794a2424b9e20, data reload: false

query1	922	400	431	400
query2	6551	2258	2035	2035
query3	6926	206	198	198
query4	23640	21447	21611	21447
query5	19747	6525	6559	6525
query6	350	214	232	214
query7	4842	300	298	298
query8	262	274	229	229
query9	3045	2599	2566	2566
query10	426	330	295	295
query11	16108	15335	14946	14946
query12	130	77	75	75
query13	1019	442	434	434
query14	18771	13090	13351	13090
query15	375	214	232	214
query16	6460	272	255	255
query17	1908	920	898	898
query18	898	312	313	312
query19	216	156	145	145
query20	103	95	106	95
query21	195	95	90	90
query22	5205	5062	5050	5050
query23	34208	33406	33533	33406
query24	6679	6348	6331	6331
query25	522	424	415	415
query26	1042	161	160	160
query27	2252	292	292	292
query28	6127	2249	2202	2202
query29	2898	2615	2668	2615
query30	254	172	164	164
query31	996	761	740	740
query32	69	62	61	61
query33	442	247	255	247
query34	858	458	480	458
query35	1163	984	902	902
query36	1390	1224	1203	1203
query37	94	59	62	59
query38	3069	2874	2888	2874
query39	1383	1311	1316	1311
query40	254	93	91	91
query41	43	38	36	36
query42	88	80	84	80
query43	638	548	614	548
query44	1204	706	711	706
query45	243	228	226	226
query46	1216	974	956	956
query47	1919	1827	1672	1672
query48	519	416	409	409
query49	626	384	371	371
query50	863	579	612	579
query51	4730	4652	4639	4639
query52	94	84	98	84
query53	238	184	184	184
query54	2636	2504	2504	2504
query55	95	82	78	78
query56	228	205	198	198
query57	1207	1144	1176	1144
query58	209	196	211	196
query59	3563	3332	3396	3332
query60	232	214	195	195
query61	95	94	96	94
query62	824	488	507	488
query63	201	176	178	176
query64	3422	1592	1525	1525
query65	3605	3537	3551	3537
query66	767	415	393	393
query67	15889	16096	18162	16096
query68	8690	615	621	615
query69	490	275	254	254
query70	1729	1782	1431	1431
query71	395	299	304	299
query72	6860	4832	4732	4732
query73	750	328	323	323
query74	6325	5795	5901	5795
query75	4603	3620	3650	3620
query76	4545	1163	1213	1163
query77	533	243	248	243
query78	12596	11715	11816	11715
query79	7500	638	648	638
query80	2528	394	387	387
query81	524	234	242	234
query82	1689	99	96	96
query83	183	133	131	131
query84	257	70	69	69
query85	1378	303	312	303
query86	328	304	294	294
query87	3246	3022	3048	3022
query88	4744	2258	2283	2258
query89	398	299	302	299
query90	1710	202	210	202
query91	163	123	124	123
query92	54	50	51	50
query93	5562	557	537	537
query94	716	208	209	208
query95	2004	1953	1933	1933
query96	635	320	314	314
query97	6470	6434	6358	6358
query98	222	203	202	202
query99	2914	956	916	916
Total cold run time: 317934 ms
Total hot run time: 212552 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.93 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9d7530e3e61db7b66c00e00b719794a2424b9e20, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.04	0.04
query4	1.80	0.06	0.07
query5	0.54	0.52	0.52
query6	1.26	0.61	0.61
query7	0.02	0.01	0.01
query8	0.03	0.02	0.03
query9	0.51	0.49	0.49
query10	0.54	0.54	0.52
query11	0.11	0.09	0.08
query12	0.11	0.10	0.09
query13	0.61	0.61	0.61
query14	0.77	0.78	0.77
query15	0.77	0.75	0.75
query16	0.36	0.39	0.39
query17	1.03	1.02	1.04
query18	0.23	0.25	0.25
query19	1.92	1.85	1.87
query20	0.02	0.01	0.01
query21	15.48	0.56	0.53
query22	1.93	2.42	1.98
query23	17.25	1.02	0.83
query24	7.13	0.89	2.27
query25	0.37	0.12	0.05
query26	0.86	0.16	0.16
query27	0.04	0.04	0.03
query28	5.39	0.78	0.72
query29	12.80	2.30	2.31
query30	0.57	0.59	0.59
query31	2.82	0.39	0.36
query32	3.38	0.50	0.50
query33	3.08	3.00	3.07
query34	15.26	4.78	4.80
query35	4.86	4.86	4.81
query36	1.08	1.03	1.01
query37	0.06	0.04	0.05
query38	0.03	0.03	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.64 s
Total hot run time: 30.93 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 9d7530e3e61db7b66c00e00b719794a2424b9e20 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

@englefly
Copy link
Contributor Author

run external

@englefly englefly merged commit 203d25d into apache:branch-2.0 Sep 29, 2024
21 of 23 checks passed
@englefly englefly deleted the pick40349-2.0 branch September 29, 2024 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants