Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](point query) add query options for short circuit queries (#32530) #35548

Merged
merged 1 commit into from
May 30, 2024

Conversation

eldenmoon
Copy link
Member

Some options like be_exec_version needed for functions

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels May 28, 2024
@eldenmoon eldenmoon force-pushed the pick-pq branch 2 times, most recently from ba05bc7 to 69dd297 Compare May 28, 2024 10:42
@xiaokang
Copy link
Contributor

run buildall

@xiaokang
Copy link
Contributor

xiaokang commented May 29, 2024

@eldenmoon what's the problem this pr try to fix?

…e#32530)

Some options like `be_exec_version` needed for functions
@eldenmoon
Copy link
Member Author

eldenmoon commented May 29, 2024

@xiaokang some functions depend on be_exec_version in query options #32530

@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49856 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eeff7de828131aa07c1df3411cf22d4e8e0fcda5, data reload: false

------ Round 1 ----------------------------------
q1	17610	4537	4362	4362
q2	2067	190	153	153
q3	10415	1919	1950	1919
q4	10424	1210	1339	1210
q5	8428	3930	3956	3930
q6	237	125	125	125
q7	2080	1644	1632	1632
q8	9258	2746	2727	2727
q9	10696	10406	10272	10272
q10	8649	3536	3541	3536
q11	431	256	270	256
q12	485	300	300	300
q13	18351	3978	3973	3973
q14	352	322	340	322
q15	515	464	462	462
q16	687	575	577	575
q17	1133	971	973	971
q18	7390	7025	6786	6786
q19	1747	1629	1584	1584
q20	544	292	325	292
q21	4416	4191	4072	4072
q22	507	397	398	397
Total cold run time: 116422 ms
Total hot run time: 49856 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4390	4302	4312	4302
q2	324	222	224	222
q3	4203	4146	4123	4123
q4	2761	2756	2752	2752
q5	7216	7166	7089	7089
q6	238	117	124	117
q7	3274	2851	2873	2851
q8	4387	4509	4546	4509
q9	16972	16829	16867	16829
q10	4264	4255	4291	4255
q11	735	675	706	675
q12	1037	848	872	848
q13	7527	3738	3725	3725
q14	451	440	416	416
q15	512	474	469	469
q16	730	690	685	685
q17	3818	3857	3853	3853
q18	8890	8675	8853	8675
q19	1712	1723	1677	1677
q20	2395	2155	2084	2084
q21	8500	8505	8426	8426
q22	1030	965	960	960
Total cold run time: 85366 ms
Total hot run time: 79542 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203527 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eeff7de828131aa07c1df3411cf22d4e8e0fcda5, data reload: false

query1	940	394	375	375
query2	6607	2935	2628	2628
query3	6919	213	210	210
query4	20019	17951	17900	17900
query5	19735	6473	6518	6473
query6	289	215	230	215
query7	4154	312	308	308
query8	254	222	244	222
query9	3130	2706	2643	2643
query10	408	295	318	295
query11	11351	10588	10669	10588
query12	127	77	75	75
query13	5590	700	690	690
query14	17387	13107	13306	13107
query15	360	227	232	227
query16	6469	292	268	268
query17	1697	1461	868	868
query18	2331	431	415	415
query19	219	149	148	148
query20	80	81	78	78
query21	192	96	90	90
query22	5226	4933	4934	4933
query23	32716	32108	31949	31949
query24	7052	6545	6544	6544
query25	532	437	416	416
query26	527	164	162	162
query27	1899	294	298	294
query28	6091	2352	2310	2310
query29	2973	2745	2668	2668
query30	238	164	171	164
query31	908	737	750	737
query32	70	68	59	59
query33	404	265	252	252
query34	860	492	478	478
query35	1115	956	925	925
query36	1341	1176	1192	1176
query37	89	63	63	63
query38	3108	2877	2886	2877
query39	1396	1312	1318	1312
query40	209	97	90	90
query41	48	38	38	38
query42	87	82	81	81
query43	821	612	662	612
query44	1143	723	711	711
query45	246	226	235	226
query46	1217	973	1002	973
query47	1860	1613	1800	1613
query48	1014	727	724	724
query49	615	390	379	379
query50	857	595	611	595
query51	4754	4728	4648	4648
query52	97	85	89	85
query53	445	323	330	323
query54	2682	2443	2437	2437
query55	80	84	82	82
query56	243	221	213	213
query57	1196	1138	1112	1112
query58	212	199	216	199
query59	4491	3906	3723	3723
query60	218	209	198	198
query61	101	98	100	98
query62	771	488	476	476
query63	492	343	343	343
query64	4226	1667	1495	1495
query65	3650	3628	3568	3568
query66	703	376	378	376
query67	16818	16588	15515	15515
query68	9165	649	635	635
query69	560	351	338	338
query70	1690	1522	1467	1467
query71	413	312	325	312
query72	6485	3533	3486	3486
query73	730	329	322	322
query74	6325	5816	5797	5797
query75	5373	3770	3708	3708
query76	5732	1141	1204	1141
query77	979	256	265	256
query78	12652	19371	12386	12386
query79	10015	645	612	612
query80	927	414	405	405
query81	497	238	235	235
query82	260	93	99	93
query83	186	136	136	136
query84	269	75	72	72
query85	807	317	319	317
query86	326	332	295	295
query87	3308	3008	3021	3008
query88	5174	2323	2317	2317
query89	357	285	289	285
query90	2710	209	211	209
query91	178	151	141	141
query92	59	51	51	51
query93	980	531	554	531
query94	1089	217	205	205
query95	1118	1097	1058	1058
query96	634	327	319	319
query97	6520	6346	6297	6297
query98	195	178	171	171
query99	3027	847	946	847
Total cold run time: 312839 ms
Total hot run time: 203527 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eeff7de828131aa07c1df3411cf22d4e8e0fcda5, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.05
query4	1.79	0.06	0.06
query5	0.53	0.52	0.52
query6	1.24	0.60	0.63
query7	0.01	0.01	0.02
query8	0.04	0.02	0.03
query9	0.52	0.47	0.49
query10	0.54	0.53	0.54
query11	0.12	0.09	0.08
query12	0.11	0.10	0.09
query13	0.62	0.62	0.61
query14	0.78	0.78	0.78
query15	0.78	0.76	0.77
query16	0.38	0.36	0.36
query17	1.01	1.02	1.01
query18	0.24	0.25	0.26
query19	1.87	1.87	1.78
query20	0.01	0.01	0.01
query21	15.46	0.55	0.55
query22	2.14	2.34	1.38
query23	17.15	0.99	0.80
query24	4.95	1.13	1.48
query25	0.36	0.16	0.04
query26	0.54	0.17	0.15
query27	0.04	0.03	0.04
query28	7.96	0.74	0.72
query29	12.61	2.30	2.13
query30	0.66	0.52	0.53
query31	2.80	0.39	0.38
query32	3.38	0.50	0.50
query33	3.06	3.08	3.02
query34	15.24	4.83	4.80
query35	4.87	4.86	4.82
query36	1.10	1.02	1.01
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.15	0.14	0.14
query41	0.07	0.02	0.01
query42	0.03	0.01	0.02
query43	0.02	0.01	0.02
Total cold run time: 103.63 s
Total hot run time: 30.25 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit eeff7de828131aa07c1df3411cf22d4e8e0fcda5 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8099/21421)
Line Coverage: 29.46% (66119/224422)
Region Coverage: 28.95% (34085/117721)
Branch Coverage: 24.81% (17493/70522)
Coverage Report: http://coverage.selectdb-in.cc/coverage/eeff7de828131aa07c1df3411cf22d4e8e0fcda5_eeff7de828131aa07c1df3411cf22d4e8e0fcda5/report/index.html

@xiaokang xiaokang merged commit a55c7da into apache:branch-2.0 May 30, 2024
21 of 24 checks passed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
…e#32530) (apache#35548)

Some options like `be_exec_version` needed for functions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/planner Issues or PRs related to the query planner kind/test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants