Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](nereids)disable join reorder if column stats is invalid #41790 (branch-2.0) #42913

Merged
merged 1 commit into from
Nov 1, 2024

Conversation

englefly
Copy link
Contributor

Proposed changes

pick #41790
Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 48817 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b7f5d0d00ab0aebaeb1f17a85a22947e1cfe25d, data reload: false

------ Round 1 ----------------------------------
q1	18075	4323	4329	4323
q2	2026	151	142	142
q3	10266	1861	1874	1861
q4	10345	1241	1303	1241
q5	8433	3891	3875	3875
q6	228	124	145	124
q7	2014	1579	1577	1577
q8	9279	2745	2718	2718
q9	10029	9900	9742	9742
q10	8620	3488	3488	3488
q11	419	246	249	246
q12	471	297	298	297
q13	18351	3946	4038	3946
q14	355	329	320	320
q15	502	475	454	454
q16	533	467	459	459
q17	1128	973	917	917
q18	7199	6771	6826	6771
q19	1706	1577	1543	1543
q20	543	327	311	311
q21	4471	4124	4062	4062
q22	508	419	400	400
Total cold run time: 115501 ms
Total hot run time: 48817 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4294	4292	4316	4292
q2	319	227	220	220
q3	4152	4165	4163	4163
q4	2751	2723	2729	2723
q5	7092	7028	7020	7020
q6	238	121	119	119
q7	3277	2823	2795	2795
q8	4358	4396	4442	4396
q9	13738	13731	13694	13694
q10	4235	4297	4323	4297
q11	779	698	708	698
q12	1033	835	839	835
q13	7134	3749	3778	3749
q14	463	434	418	418
q15	504	469	465	465
q16	655	569	600	569
q17	3731	3767	3789	3767
q18	8947	8971	9298	8971
q19	1736	1673	1687	1673
q20	2472	2282	2231	2231
q21	8797	8582	8486	8486
q22	1067	952	926	926
Total cold run time: 81772 ms
Total hot run time: 76507 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212524 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b7f5d0d00ab0aebaeb1f17a85a22947e1cfe25d, data reload: false

query1	927	392	435	392
query2	6628	2339	2274	2274
query3	6919	199	201	199
query4	23320	21572	21527	21527
query5	19724	6568	6529	6529
query6	294	218	235	218
query7	4332	303	312	303
query8	261	272	254	254
query9	3062	2627	2557	2557
query10	425	300	294	294
query11	15700	14838	14893	14838
query12	119	78	73	73
query13	1049	441	421	421
query14	18428	13558	13614	13558
query15	367	224	221	221
query16	6446	284	259	259
query17	1907	920	877	877
query18	882	323	318	318
query19	208	154	149	149
query20	104	96	98	96
query21	189	99	92	92
query22	5196	4987	5043	4987
query23	34100	33559	33485	33485
query24	6821	6295	6361	6295
query25	529	413	432	413
query26	795	168	166	166
query27	2254	304	297	297
query28	6136	2245	2197	2197
query29	2870	2808	2841	2808
query30	243	169	167	167
query31	927	746	762	746
query32	64	63	59	59
query33	432	267	259	259
query34	858	481	466	466
query35	1120	898	968	898
query36	1197	989	1170	989
query37	97	59	58	58
query38	3065	2906	2878	2878
query39	1390	1335	1330	1330
query40	202	95	95	95
query41	39	40	39	39
query42	94	87	90	87
query43	787	656	594	594
query44	1167	718	729	718
query45	251	232	236	232
query46	1258	974	965	965
query47	1879	1790	1711	1711
query48	508	409	419	409
query49	627	371	382	371
query50	865	614	595	595
query51	4746	4683	4677	4677
query52	93	91	85	85
query53	230	193	192	192
query54	2668	2500	2500	2500
query55	84	83	81	81
query56	231	202	203	202
query57	1207	1192	1195	1192
query58	231	212	213	212
query59	3447	3184	3393	3184
query60	220	222	192	192
query61	94	94	99	94
query62	865	498	472	472
query63	214	179	176	176
query64	3120	1468	1477	1468
query65	3635	3561	3542	3542
query66	805	430	389	389
query67	16170	15279	16828	15279
query68	9115	632	641	632
query69	488	282	280	280
query70	1571	1421	1374	1374
query71	415	308	311	308
query72	6828	4673	4711	4673
query73	756	312	321	312
query74	6236	5803	5826	5803
query75	4548	3676	3691	3676
query76	4751	1162	1150	1150
query77	633	268	262	262
query78	12446	11901	19580	11901
query79	4349	638	628	628
query80	1991	399	376	376
query81	510	241	239	239
query82	271	104	100	100
query83	169	137	133	133
query84	265	71	70	70
query85	1430	313	303	303
query86	349	297	293	293
query87	3197	3009	2987	2987
query88	3673	2276	2290	2276
query89	342	305	298	298
query90	1757	218	210	210
query91	174	128	123	123
query92	60	51	50	50
query93	933	558	540	540
query94	902	203	211	203
query95	1973	1986	1968	1968
query96	632	317	322	317
query97	6561	6263	6379	6263
query98	210	210	217	210
query99	2925	929	866	866
Total cold run time: 305582 ms
Total hot run time: 212524 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b7f5d0d00ab0aebaeb1f17a85a22947e1cfe25d, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.26	0.05	0.05
query4	1.78	0.10	0.07
query5	0.54	0.53	0.53
query6	1.32	0.61	0.60
query7	0.02	0.01	0.01
query8	0.04	0.02	0.02
query9	0.54	0.48	0.47
query10	0.55	0.54	0.52
query11	0.13	0.09	0.09
query12	0.11	0.09	0.10
query13	0.63	0.62	0.61
query14	0.78	0.81	0.82
query15	0.81	0.77	0.78
query16	0.38	0.37	0.37
query17	1.03	1.02	1.01
query18	0.22	0.28	0.24
query19	1.94	1.84	1.88
query20	0.01	0.00	0.01
query21	15.50	0.55	0.54
query22	1.97	1.92	1.28
query23	16.77	1.01	0.98
query24	6.05	0.76	0.96
query25	0.37	0.10	0.06
query26	0.59	0.15	0.14
query27	0.05	0.04	0.03
query28	7.33	0.79	0.76
query29	12.75	2.32	2.25
query30	0.60	0.51	0.52
query31	2.83	0.40	0.38
query32	3.36	0.52	0.51
query33	3.20	3.15	3.13
query34	15.25	4.89	4.83
query35	4.88	4.90	4.86
query36	1.06	1.02	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.15
query41	0.07	0.01	0.01
query42	0.02	0.01	0.02
query43	0.03	0.02	0.02
Total cold run time: 104.14 s
Total hot run time: 30.44 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4b7f5d0d00ab0aebaeb1f17a85a22947e1cfe25d with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          57 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.2 seconds inserted 10000000 Rows, about 471K ops/s

@englefly
Copy link
Contributor Author

run p0

@englefly englefly merged commit b136c44 into apache:branch-2.0 Nov 1, 2024
22 of 24 checks passed
@englefly englefly deleted the pick41790-br2.0 branch November 1, 2024 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants