Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](Nereids) fix cancel task jobs failed #44755

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

LiBinfeng-01
Copy link
Collaborator

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #44279

Problem Summary:

when cancel task, it should use "taskId" as unbound slot in and right child's left child, otherwise it would failed
Regression test to be added by @CalvinKirs

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@@ -79,14 +80,17 @@ private void validate() throws AnalysisException {
throw new AnalysisException("JobName value must is string");
}
this.jobName = ((StringLikeLiteral) expr.child(0).child(1)).getStringValue();
String taskIdInput = ((StringLikeLiteral) expr.child(1).child(0)).getStringValue();
if (!(expr.child(1).child(0) instanceof UnboundSlot)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if jobName must string, i think we should change syntax to ensure it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe u should use get text from identifier to keep it same as legacy planner

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39831 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 00c6e5be060294635b33ce2fe8980f8b8dbf508f, data reload: false

------ Round 1 ----------------------------------
q1	17613	7457	7323	7323
q2	2060	175	175	175
q3	10860	1116	1199	1116
q4	10570	747	781	747
q5	7607	2730	2675	2675
q6	235	148	149	148
q7	999	671	598	598
q8	9236	1804	1871	1804
q9	6650	6433	6463	6433
q10	6983	2328	2371	2328
q11	461	252	261	252
q12	427	225	224	224
q13	17794	3012	3082	3012
q14	238	210	225	210
q15	581	528	524	524
q16	676	583	600	583
q17	971	588	534	534
q18	7176	6841	6728	6728
q19	1343	999	1038	999
q20	472	180	182	180
q21	4048	3247	2930	2930
q22	372	320	308	308
Total cold run time: 107372 ms
Total hot run time: 39831 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7237	7264	7214	7214
q2	342	224	225	224
q3	2911	2802	2952	2802
q4	2080	1846	1919	1846
q5	5647	5665	5633	5633
q6	232	147	150	147
q7	2273	1848	1810	1810
q8	3418	3516	3509	3509
q9	8960	8990	8987	8987
q10	3594	3582	3537	3537
q11	614	502	497	497
q12	835	628	613	613
q13	12169	3264	3222	3222
q14	301	289	277	277
q15	582	525	537	525
q16	706	635	633	633
q17	1883	1643	1609	1609
q18	8172	7931	7594	7594
q19	1691	1637	1630	1630
q20	2139	1914	1902	1902
q21	5663	5451	5565	5451
q22	646	577	551	551
Total cold run time: 72095 ms
Total hot run time: 60213 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197890 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 00c6e5be060294635b33ce2fe8980f8b8dbf508f, data reload: false

query1	1241	947	948	947
query2	6242	2042	2061	2042
query3	10977	4512	4497	4497
query4	66844	29184	23537	23537
query5	4982	471	461	461
query6	396	187	171	171
query7	5522	308	286	286
query8	317	247	252	247
query9	8731	2712	2688	2688
query10	442	244	263	244
query11	17248	15279	16066	15279
query12	156	106	110	106
query13	1465	428	415	415
query14	11028	7486	6883	6883
query15	225	193	189	189
query16	6665	487	461	461
query17	1221	574	578	574
query18	1337	296	306	296
query19	207	145	145	145
query20	120	111	146	111
query21	205	107	101	101
query22	4711	4644	4671	4644
query23	35159	34796	34774	34774
query24	5591	2488	2519	2488
query25	478	388	382	382
query26	651	147	156	147
query27	2045	278	284	278
query28	4375	2435	2436	2435
query29	668	407	417	407
query30	205	155	156	155
query31	1025	865	844	844
query32	64	53	58	53
query33	417	293	299	293
query34	940	506	535	506
query35	908	797	768	768
query36	1087	974	984	974
query37	122	74	71	71
query38	4481	4484	4381	4381
query39	1524	1512	1495	1495
query40	198	107	102	102
query41	45	41	43	41
query42	108	93	100	93
query43	553	490	487	487
query44	1179	819	814	814
query45	224	175	176	175
query46	1175	721	726	721
query47	2033	1965	1918	1918
query48	391	341	319	319
query49	741	395	383	383
query50	833	381	385	381
query51	7457	7206	7107	7107
query52	107	88	90	88
query53	255	177	177	177
query54	487	396	382	382
query55	78	76	73	73
query56	254	239	231	231
query57	1248	1100	1122	1100
query58	227	228	217	217
query59	3050	3019	3045	3019
query60	273	230	234	230
query61	100	101	101	101
query62	806	681	672	672
query63	208	192	179	179
query64	1331	689	622	622
query65	3266	3230	3272	3230
query66	700	295	309	295
query67	15921	15698	15672	15672
query68	3784	563	552	552
query69	430	249	247	247
query70	1233	1132	1124	1124
query71	362	248	245	245
query72	6308	4083	4019	4019
query73	771	362	359	359
query74	9565	9108	9146	9108
query75	3426	2667	2642	2642
query76	1827	1179	1116	1116
query77	518	311	281	281
query78	10459	9445	9433	9433
query79	1670	578	592	578
query80	873	420	437	420
query81	497	237	225	225
query82	1313	132	124	124
query83	252	142	141	141
query84	289	69	67	67
query85	885	301	287	287
query86	335	295	313	295
query87	4666	4635	4633	4633
query88	3526	2198	2228	2198
query89	412	296	291	291
query90	2006	192	192	192
query91	140	112	104	104
query92	66	56	52	52
query93	1938	540	533	533
query94	741	276	269	269
query95	345	247	253	247
query96	631	275	297	275
query97	2912	2727	2725	2725
query98	215	195	201	195
query99	1849	1353	1309	1309
Total cold run time: 317514 ms
Total hot run time: 197890 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 00c6e5be060294635b33ce2fe8980f8b8dbf508f, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.08	0.07
query4	1.61	0.10	0.10
query5	0.41	0.42	0.42
query6	1.14	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.51	0.50
query10	0.55	0.56	0.55
query11	0.15	0.11	0.10
query12	0.14	0.12	0.11
query13	0.61	0.61	0.59
query14	2.70	2.70	2.77
query15	0.91	0.83	0.83
query16	0.39	0.38	0.38
query17	1.07	1.06	0.97
query18	0.23	0.21	0.21
query19	1.88	1.71	1.83
query20	0.02	0.01	0.01
query21	15.37	0.59	0.59
query22	2.34	2.61	1.61
query23	17.02	1.18	0.91
query24	2.84	1.80	2.09
query25	0.30	0.06	0.05
query26	0.64	0.14	0.14
query27	0.05	0.06	0.04
query28	9.77	1.10	1.08
query29	12.61	3.24	3.21
query30	0.25	0.07	0.07
query31	2.86	0.38	0.38
query32	3.28	0.47	0.48
query33	3.03	3.09	3.06
query34	17.29	4.44	4.44
query35	4.48	4.50	4.47
query36	0.65	0.49	0.49
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.03	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.05 s
Total hot run time: 33.14 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39869 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2816bc2db0afd9121be898f05e80ea8f8a47a207, data reload: false

------ Round 1 ----------------------------------
q1	17675	7491	7270	7270
q2	2049	172	165	165
q3	10636	1127	1245	1127
q4	10226	693	745	693
q5	7623	2691	2680	2680
q6	236	149	143	143
q7	994	626	611	611
q8	9238	1805	1934	1805
q9	6626	6466	6435	6435
q10	6955	2289	2312	2289
q11	454	266	254	254
q12	410	219	218	218
q13	17761	3032	2986	2986
q14	245	215	213	213
q15	564	526	541	526
q16	644	586	590	586
q17	989	604	518	518
q18	7278	6668	6768	6668
q19	1341	979	990	979
q20	471	185	182	182
q21	4387	3214	3274	3214
q22	385	307	313	307
Total cold run time: 107187 ms
Total hot run time: 39869 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7280	7233	7193	7193
q2	329	228	231	228
q3	2886	2751	3026	2751
q4	2121	1868	1771	1771
q5	5597	5695	5650	5650
q6	217	139	141	139
q7	2196	1748	1874	1748
q8	3392	3526	3467	3467
q9	8983	8999	9153	8999
q10	3583	3535	3517	3517
q11	587	501	489	489
q12	836	629	639	629
q13	12846	3203	3166	3166
q14	295	261	266	261
q15	586	528	516	516
q16	694	640	630	630
q17	1788	1611	1533	1533
q18	7955	7381	7414	7381
q19	1642	1558	1492	1492
q20	2053	1821	1831	1821
q21	5405	5285	5429	5285
q22	636	548	555	548
Total cold run time: 71907 ms
Total hot run time: 59214 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191193 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2816bc2db0afd9121be898f05e80ea8f8a47a207, data reload: false

query1	977	377	362	362
query2	6516	2073	2034	2034
query3	6706	209	215	209
query4	33985	23556	23696	23556
query5	4342	477	482	477
query6	285	194	193	193
query7	4620	291	302	291
query8	298	236	230	230
query9	9513	2694	2688	2688
query10	482	276	260	260
query11	18278	15242	15189	15189
query12	140	99	101	99
query13	1644	399	411	399
query14	9122	7122	7142	7122
query15	254	175	178	175
query16	8126	430	460	430
query17	1610	559	560	559
query18	2123	281	285	281
query19	338	154	138	138
query20	113	119	107	107
query21	204	99	105	99
query22	4692	4462	4282	4282
query23	35505	34306	34278	34278
query24	10562	2401	2368	2368
query25	636	383	376	376
query26	1468	148	153	148
query27	2838	276	266	266
query28	7985	2407	2423	2407
query29	915	406	398	398
query30	294	148	152	148
query31	1000	799	864	799
query32	96	61	57	57
query33	777	282	272	272
query34	985	502	515	502
query35	854	728	724	724
query36	1090	905	959	905
query37	141	87	73	73
query38	4221	4134	4211	4134
query39	1524	1421	1421	1421
query40	276	97	98	97
query41	46	43	44	43
query42	110	104	96	96
query43	532	490	481	481
query44	1281	795	791	791
query45	179	167	167	167
query46	1166	708	715	708
query47	1961	1889	1863	1863
query48	425	315	317	315
query49	1171	400	392	392
query50	817	389	380	380
query51	7272	7208	7194	7194
query52	99	98	87	87
query53	265	180	174	174
query54	1065	403	401	401
query55	82	76	75	75
query56	260	223	240	223
query57	1272	1095	1110	1095
query58	232	206	215	206
query59	3163	3019	2930	2930
query60	268	242	270	242
query61	120	108	110	108
query62	866	676	683	676
query63	216	188	190	188
query64	5431	761	733	733
query65	3311	3212	3260	3212
query66	1312	336	321	321
query67	16008	15813	15564	15564
query68	4903	548	535	535
query69	413	249	251	249
query70	1162	1113	1111	1111
query71	319	265	268	265
query72	6462	4122	4111	4111
query73	762	352	352	352
query74	10070	8960	8999	8960
query75	3397	2672	2674	2672
query76	2758	995	1033	995
query77	392	274	339	274
query78	10378	9524	9502	9502
query79	2488	592	602	592
query80	1198	474	423	423
query81	529	228	229	228
query82	928	126	120	120
query83	236	146	143	143
query84	235	68	67	67
query85	1254	298	300	298
query86	416	291	294	291
query87	4647	4619	4429	4429
query88	3986	2196	2206	2196
query89	416	302	304	302
query90	1904	186	186	186
query91	134	103	106	103
query92	74	51	53	51
query93	1673	540	542	540
query94	796	286	298	286
query95	346	246	241	241
query96	609	277	276	276
query97	2834	2672	2713	2672
query98	219	192	193	192
query99	1540	1329	1313	1313
Total cold run time: 301943 ms
Total hot run time: 191193 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.95 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2816bc2db0afd9121be898f05e80ea8f8a47a207, data reload: false

query1	0.03	0.04	0.03
query2	0.06	0.03	0.03
query3	0.24	0.08	0.07
query4	1.62	0.10	0.11
query5	0.42	0.42	0.43
query6	1.16	0.67	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.51	0.52
query10	0.56	0.56	0.56
query11	0.14	0.10	0.10
query12	0.13	0.12	0.11
query13	0.61	0.61	0.59
query14	2.84	2.72	2.76
query15	0.90	0.83	0.84
query16	0.39	0.38	0.38
query17	1.07	1.07	1.05
query18	0.23	0.20	0.20
query19	1.92	1.83	1.96
query20	0.02	0.01	0.02
query21	15.42	0.60	0.59
query22	2.50	2.21	2.38
query23	17.04	0.84	0.79
query24	2.58	0.94	1.52
query25	0.22	0.36	0.11
query26	0.32	0.14	0.14
query27	0.04	0.04	0.04
query28	10.59	1.10	1.08
query29	12.59	3.26	3.21
query30	0.25	0.06	0.05
query31	2.88	0.38	0.38
query32	3.28	0.48	0.47
query33	3.06	2.97	3.04
query34	16.79	4.48	4.44
query35	4.48	4.53	4.49
query36	0.68	0.50	0.51
query37	0.09	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.17 s
Total hot run time: 32.95 s

| DROP JOB (IF EXISTS)? wildWhere? #dropJob
| RESUME JOB wildWhere? #resumeJob
| CANCEL TASK wildWhere? #cancelJobTask
| PAUSE JOB WHERE (jobNameKey=STRING_LITERAL) EQ (jobNameValue=STRING_LITERAL) #pauseJob
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jobNameKey is not a string literal. should be identifier

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40235 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5feed090d94b480e6db76d73aa2ac26a3e5f2260, data reload: false

------ Round 1 ----------------------------------
q1	17579	7460	7314	7314
q2	2041	183	170	170
q3	10558	1093	1169	1093
q4	10449	723	689	689
q5	7635	2712	2744	2712
q6	248	151	149	149
q7	997	631	599	599
q8	9250	1857	1866	1857
q9	6731	6567	6564	6564
q10	7051	2331	2379	2331
q11	467	283	269	269
q12	432	226	223	223
q13	17781	3061	2989	2989
q14	260	212	212	212
q15	587	547	535	535
q16	650	567	579	567
q17	1003	586	571	571
q18	7344	6759	6787	6759
q19	1343	972	927	927
q20	462	178	182	178
q21	4290	3237	3225	3225
q22	387	325	302	302
Total cold run time: 107545 ms
Total hot run time: 40235 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7229	7284	7231	7231
q2	330	243	234	234
q3	2927	2886	2894	2886
q4	2064	1803	1817	1803
q5	5702	5675	5680	5675
q6	229	143	147	143
q7	2241	1883	1784	1784
q8	3447	3569	3516	3516
q9	8996	9082	9031	9031
q10	3619	3624	3559	3559
q11	606	513	514	513
q12	813	582	609	582
q13	11529	3274	3209	3209
q14	307	296	275	275
q15	579	530	529	529
q16	705	641	643	641
q17	1879	1671	1630	1630
q18	8258	7800	7592	7592
q19	1715	1571	1649	1571
q20	2082	1895	1859	1859
q21	5622	5444	5482	5444
q22	662	578	559	559
Total cold run time: 71541 ms
Total hot run time: 60266 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197336 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5feed090d94b480e6db76d73aa2ac26a3e5f2260, data reload: false

query1	1264	956	965	956
query2	6258	2051	2070	2051
query3	10938	4369	4603	4369
query4	67377	28543	23548	23548
query5	5001	470	444	444
query6	403	188	189	188
query7	5520	307	298	298
query8	314	225	224	224
query9	8727	2737	2718	2718
query10	450	248	248	248
query11	16933	15169	16017	15169
query12	163	108	105	105
query13	1444	459	443	443
query14	9801	7257	7503	7257
query15	229	199	190	190
query16	7132	488	500	488
query17	1071	580	613	580
query18	1832	349	322	322
query19	205	165	163	163
query20	122	112	112	112
query21	216	146	102	102
query22	4697	4682	4544	4544
query23	34635	34334	34495	34334
query24	5516	2539	2443	2443
query25	506	384	379	379
query26	676	156	148	148
query27	2164	283	289	283
query28	4506	2533	2488	2488
query29	681	429	414	414
query30	211	144	147	144
query31	1004	834	836	834
query32	68	60	55	55
query33	398	292	290	290
query34	924	516	539	516
query35	907	776	771	771
query36	1092	963	969	963
query37	128	76	73	73
query38	4543	4255	4444	4255
query39	1669	1467	1505	1467
query40	205	102	98	98
query41	48	44	43	43
query42	119	104	100	100
query43	555	509	501	501
query44	1179	852	844	844
query45	195	169	170	169
query46	1198	741	748	741
query47	2050	1931	1930	1930
query48	429	331	362	331
query49	747	397	400	397
query50	877	401	406	401
query51	7435	7240	7128	7128
query52	99	85	87	85
query53	258	183	178	178
query54	507	396	397	396
query55	79	79	75	75
query56	247	241	240	240
query57	1218	1139	1110	1110
query58	205	209	220	209
query59	3200	2839	2913	2839
query60	272	244	239	239
query61	110	106	104	104
query62	797	657	676	657
query63	212	189	189	189
query64	1414	654	644	644
query65	3264	3251	3284	3251
query66	663	299	303	299
query67	16083	15754	15857	15754
query68	3765	580	581	580
query69	447	250	240	240
query70	1170	1129	1123	1123
query71	336	250	257	250
query72	6413	4044	4068	4044
query73	776	357	357	357
query74	10301	8887	9041	8887
query75	3408	2646	2674	2646
query76	1872	1108	1076	1076
query77	487	278	272	272
query78	10542	9498	9450	9450
query79	2132	613	610	610
query80	1383	428	422	422
query81	503	247	237	237
query82	1125	124	118	118
query83	168	152	157	152
query84	274	72	70	70
query85	1002	298	361	298
query86	382	297	293	293
query87	4661	4532	4664	4532
query88	3655	2233	2180	2180
query89	422	297	292	292
query90	2008	187	192	187
query91	132	103	111	103
query92	65	50	51	50
query93	2629	542	540	540
query94	812	287	342	287
query95	338	252	251	251
query96	627	289	277	277
query97	2859	2707	2673	2673
query98	213	204	194	194
query99	1601	1317	1328	1317
Total cold run time: 319898 ms
Total hot run time: 197336 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.78 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5feed090d94b480e6db76d73aa2ac26a3e5f2260, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.63	0.11	0.10
query5	0.41	0.39	0.42
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.51	0.53
query10	0.56	0.55	0.56
query11	0.14	0.10	0.11
query12	0.14	0.11	0.11
query13	0.60	0.61	0.60
query14	2.82	2.88	2.82
query15	0.90	0.83	0.84
query16	0.38	0.38	0.37
query17	1.05	1.01	1.08
query18	0.23	0.21	0.21
query19	1.98	1.89	2.00
query20	0.01	0.01	0.01
query21	15.38	0.57	0.60
query22	2.70	2.24	1.66
query23	17.27	0.85	0.76
query24	3.21	1.29	1.59
query25	0.35	0.12	0.08
query26	0.52	0.13	0.14
query27	0.04	0.04	0.04
query28	9.81	1.10	1.09
query29	12.54	3.20	3.23
query30	0.25	0.07	0.07
query31	2.87	0.40	0.38
query32	3.27	0.47	0.46
query33	3.04	3.01	3.04
query34	17.10	4.45	4.42
query35	4.53	4.52	4.52
query36	0.66	0.50	0.49
query37	0.09	0.06	0.05
query38	0.05	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 107 s
Total hot run time: 32.78 s

@@ -59,36 +44,9 @@ public <R, C> R accept(PlanVisitor<R, C> visitor, C context) {

@Override
public void run(ConnectContext ctx, StmtExecutor executor) throws Exception {
validate();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we miss the permission check?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, mistoken delete

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40163 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1941723cb7d4adf1b62f0731847cd492e1db01b6, data reload: false

------ Round 1 ----------------------------------
q1	17713	7468	7309	7309
q2	2096	185	172	172
q3	10691	1130	1228	1130
q4	10422	769	693	693
q5	7597	2767	2756	2756
q6	238	149	157	149
q7	1022	672	622	622
q8	9260	1866	1968	1866
q9	6676	6516	6519	6516
q10	7045	2284	2283	2283
q11	472	266	258	258
q12	413	222	230	222
q13	17768	3084	3083	3083
q14	254	217	214	214
q15	565	540	521	521
q16	659	593	590	590
q17	989	534	586	534
q18	7517	6688	6605	6605
q19	1363	1010	1055	1010
q20	475	180	186	180
q21	4153	3142	3398	3142
q22	382	308	318	308
Total cold run time: 107770 ms
Total hot run time: 40163 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7361	7286	7282	7282
q2	323	228	233	228
q3	2934	2929	3114	2929
q4	2130	1815	1833	1815
q5	5582	5725	5645	5645
q6	223	142	143	142
q7	2222	1802	1847	1802
q8	3390	3501	3538	3501
q9	8990	9032	9110	9032
q10	3600	3579	3531	3531
q11	595	505	525	505
q12	816	625	608	608
q13	12280	3159	3230	3159
q14	304	275	255	255
q15	558	519	509	509
q16	674	626	629	626
q17	1828	1602	1556	1556
q18	7764	7503	7299	7299
q19	1656	1485	1569	1485
q20	2068	1809	1823	1809
q21	5483	5270	5325	5270
q22	628	560	559	559
Total cold run time: 71409 ms
Total hot run time: 59547 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191123 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1941723cb7d4adf1b62f0731847cd492e1db01b6, data reload: false

query1	1196	390	390	390
query2	6537	2082	1974	1974
query3	6711	213	214	213
query4	34014	23927	23453	23453
query5	4398	463	454	454
query6	302	196	188	188
query7	4631	294	299	294
query8	314	250	221	221
query9	9466	2687	2687	2687
query10	468	242	241	241
query11	18176	15187	15277	15187
query12	151	104	99	99
query13	1648	403	386	386
query14	10157	7267	7436	7267
query15	270	180	177	177
query16	7624	492	477	477
query17	1691	595	578	578
query18	1483	307	321	307
query19	368	150	152	150
query20	114	108	107	107
query21	209	99	99	99
query22	4605	4387	4214	4214
query23	34966	34061	34282	34061
query24	10356	2379	2470	2379
query25	631	371	376	371
query26	1220	152	151	151
query27	2612	289	280	280
query28	7797	2445	2436	2436
query29	791	396	411	396
query30	292	151	150	150
query31	1045	813	827	813
query32	90	55	55	55
query33	758	328	285	285
query34	934	503	524	503
query35	847	753	741	741
query36	1092	938	963	938
query37	148	76	73	73
query38	4408	4192	4189	4189
query39	1479	1470	1410	1410
query40	198	101	97	97
query41	46	42	43	42
query42	110	101	104	101
query43	538	507	476	476
query44	1270	802	790	790
query45	192	173	163	163
query46	1149	678	707	678
query47	1931	1876	1836	1836
query48	394	317	313	313
query49	1157	413	381	381
query50	812	383	377	377
query51	7226	6948	6985	6948
query52	97	90	87	87
query53	255	181	179	179
query54	1116	394	415	394
query55	82	79	79	79
query56	278	249	263	249
query57	1249	1130	1104	1104
query58	236	231	241	231
query59	3117	2949	3024	2949
query60	301	250	250	250
query61	111	109	109	109
query62	870	679	683	679
query63	222	187	181	181
query64	4593	688	640	640
query65	3330	3213	3235	3213
query66	1347	318	319	318
query67	16000	15698	15665	15665
query68	4067	568	560	560
query69	415	251	242	242
query70	1200	1152	1157	1152
query71	326	258	244	244
query72	6456	4342	4212	4212
query73	776	363	371	363
query74	10069	9096	8900	8900
query75	3392	2616	2650	2616
query76	2678	1089	1097	1089
query77	446	265	289	265
query78	10339	9476	9571	9476
query79	2551	605	616	605
query80	1149	429	518	429
query81	541	245	229	229
query82	680	116	113	113
query83	246	143	141	141
query84	236	67	75	67
query85	1497	308	299	299
query86	458	304	300	300
query87	4720	4511	4701	4511
query88	3959	2235	2187	2187
query89	411	308	297	297
query90	2102	188	182	182
query91	145	105	111	105
query92	67	52	53	52
query93	1445	545	545	545
query94	985	283	280	280
query95	343	252	250	250
query96	615	275	291	275
query97	2828	2723	2704	2704
query98	263	191	191	191
query99	1597	1319	1357	1319
Total cold run time: 299228 ms
Total hot run time: 191123 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.34 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1941723cb7d4adf1b62f0731847cd492e1db01b6, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.08	0.07
query4	1.61	0.10	0.11
query5	0.42	0.42	0.38
query6	1.17	0.65	0.66
query7	0.02	0.02	0.02
query8	0.03	0.04	0.03
query9	0.57	0.50	0.52
query10	0.56	0.55	0.54
query11	0.14	0.10	0.10
query12	0.16	0.12	0.11
query13	0.61	0.61	0.60
query14	2.74	2.74	2.78
query15	0.90	0.83	0.83
query16	0.38	0.40	0.40
query17	1.09	1.06	1.01
query18	0.23	0.21	0.21
query19	1.93	2.06	1.91
query20	0.02	0.01	0.01
query21	15.35	0.56	0.56
query22	2.39	1.72	1.79
query23	17.25	0.94	0.86
query24	3.22	1.60	0.65
query25	0.28	0.05	0.07
query26	0.47	0.15	0.14
query27	0.07	0.04	0.04
query28	10.54	1.10	1.09
query29	12.58	3.22	3.25
query30	0.26	0.07	0.07
query31	2.86	0.39	0.40
query32	3.24	0.46	0.46
query33	3.01	3.05	3.04
query34	17.10	4.49	4.50
query35	4.50	4.59	4.49
query36	0.67	0.48	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 107.21 s
Total hot run time: 32.34 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39688 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1f92321d8079a4d6f13a0b11b981af3fd47f8323, data reload: false

------ Round 1 ----------------------------------
q1	17572	7450	7248	7248
q2	2055	180	166	166
q3	10551	1142	1122	1122
q4	10230	745	688	688
q5	7614	2647	2634	2634
q6	236	147	145	145
q7	993	623	616	616
q8	9262	1853	1941	1853
q9	6726	6508	6441	6441
q10	6966	2278	2304	2278
q11	456	264	249	249
q12	402	216	224	216
q13	17772	3066	3045	3045
q14	245	202	214	202
q15	569	534	528	528
q16	633	583	603	583
q17	982	599	530	530
q18	7405	6661	6614	6614
q19	1335	999	927	927
q20	473	180	178	178
q21	4175	3317	3102	3102
q22	391	323	327	323
Total cold run time: 107043 ms
Total hot run time: 39688 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7243	7266	7207	7207
q2	335	232	232	232
q3	2902	2760	2945	2760
q4	2116	1855	1828	1828
q5	5533	5655	5619	5619
q6	221	140	138	138
q7	2223	1791	1808	1791
q8	3352	3570	3487	3487
q9	8992	8981	8961	8961
q10	3613	3570	3532	3532
q11	621	509	496	496
q12	830	626	592	592
q13	12805	3271	3206	3206
q14	296	268	273	268
q15	573	525	529	525
q16	670	615	623	615
q17	1774	1566	1565	1565
q18	7862	7458	7427	7427
q19	1647	1559	1568	1559
q20	2028	1840	1794	1794
q21	5418	5176	5219	5176
q22	633	588	588	588
Total cold run time: 71687 ms
Total hot run time: 59366 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189460 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f92321d8079a4d6f13a0b11b981af3fd47f8323, data reload: false

query1	960	384	370	370
query2	6521	2072	2016	2016
query3	6711	216	212	212
query4	34060	23637	23450	23450
query5	4398	458	440	440
query6	282	180	202	180
query7	4646	291	311	291
query8	299	243	214	214
query9	9751	2669	2691	2669
query10	467	243	242	242
query11	18235	15245	15177	15177
query12	147	99	105	99
query13	1640	396	377	377
query14	9625	6418	6372	6372
query15	254	172	182	172
query16	8042	469	431	431
query17	1617	571	546	546
query18	2113	297	300	297
query19	233	159	155	155
query20	120	111	118	111
query21	213	102	102	102
query22	4384	4397	4342	4342
query23	34811	33790	34342	33790
query24	11329	2427	2576	2427
query25	692	407	395	395
query26	1751	154	155	154
query27	2933	281	287	281
query28	8071	2438	2432	2432
query29	1028	421	442	421
query30	303	149	148	148
query31	1015	826	833	826
query32	97	54	56	54
query33	772	285	284	284
query34	1004	491	514	491
query35	888	806	753	753
query36	1122	955	923	923
query37	162	79	70	70
query38	4364	4177	4224	4177
query39	1468	1400	1428	1400
query40	286	99	96	96
query41	47	42	43	42
query42	111	96	98	96
query43	527	486	488	486
query44	1299	817	798	798
query45	187	162	173	162
query46	1153	708	704	704
query47	1966	1886	1869	1869
query48	394	303	327	303
query49	1262	378	380	378
query50	807	387	370	370
query51	7339	7134	6995	6995
query52	99	90	91	90
query53	254	185	180	180
query54	1195	402	395	395
query55	79	75	73	73
query56	264	228	248	228
query57	1239	1099	1149	1099
query58	235	214	210	210
query59	3287	2873	2991	2873
query60	260	255	240	240
query61	113	106	111	106
query62	909	669	662	662
query63	215	187	182	182
query64	5234	647	619	619
query65	3285	3190	3292	3190
query66	1429	294	310	294
query67	15915	15794	15751	15751
query68	4949	539	527	527
query69	429	251	265	251
query70	1240	1078	1131	1078
query71	335	251	243	243
query72	6419	4025	3877	3877
query73	765	351	363	351
query74	10444	9094	9024	9024
query75	3451	2660	2645	2645
query76	2915	1083	1065	1065
query77	426	261	263	261
query78	10380	9391	9447	9391
query79	1616	592	586	586
query80	1090	427	419	419
query81	518	239	235	235
query82	937	120	166	120
query83	240	142	145	142
query84	232	71	71	71
query85	1308	294	295	294
query86	363	303	304	303
query87	4740	4590	4529	4529
query88	3504	2196	2166	2166
query89	402	289	292	289
query90	2063	188	183	183
query91	138	101	103	101
query92	57	51	54	51
query93	1094	568	522	522
query94	1043	291	294	291
query95	341	240	243	240
query96	610	287	274	274
query97	2843	2653	2652	2652
query98	223	191	188	188
query99	1559	1309	1311	1309
Total cold run time: 302592 ms
Total hot run time: 189460 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f92321d8079a4d6f13a0b11b981af3fd47f8323, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.03
query3	0.23	0.08	0.07
query4	1.59	0.10	0.10
query5	0.43	0.39	0.41
query6	1.17	0.65	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.58	0.51	0.51
query10	0.56	0.55	0.56
query11	0.13	0.10	0.11
query12	0.13	0.11	0.11
query13	0.60	0.60	0.60
query14	2.80	2.82	2.70
query15	0.90	0.82	0.83
query16	0.40	0.39	0.38
query17	1.04	1.05	0.98
query18	0.23	0.21	0.22
query19	1.96	1.84	2.03
query20	0.01	0.02	0.01
query21	15.36	0.60	0.60
query22	2.30	2.53	2.37
query23	17.32	0.84	0.80
query24	3.45	1.26	1.22
query25	0.32	0.21	0.25
query26	0.44	0.14	0.13
query27	0.03	0.04	0.04
query28	10.07	1.10	1.06
query29	12.60	3.27	3.28
query30	0.24	0.06	0.06
query31	2.86	0.39	0.37
query32	3.28	0.46	0.46
query33	2.99	3.03	3.03
query34	17.01	4.43	4.46
query35	4.53	4.42	4.49
query36	0.66	0.48	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.17	0.13	0.14
query41	0.07	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.88 s
Total hot run time: 33.37 s

CalvinKirs
CalvinKirs previously approved these changes Dec 11, 2024
starocean999
starocean999 previously approved these changes Dec 11, 2024
@LiBinfeng-01 LiBinfeng-01 dismissed stale reviews from starocean999 and CalvinKirs via c32dabf December 11, 2024 06:37
@LiBinfeng-01
Copy link
Collaborator Author

run buildall

wildWhere = getWildWhere(ctx.wildWhere());
}
return new PauseJobCommand(wildWhere);
return new PauseJobCommand(stripQuotes(ctx.jobNameValue.getText()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so we do not check jobNameKey?

Comment on lines +791 to +792
String jobName = stripQuotes(ctx.jobNameValue.getText());
Long taskId = Long.valueOf(ctx.taskIdValue.getText());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will happen if sql like cancel task where task_id = '123' and job_name = 123?

@doris-robot
Copy link

TPC-H: Total hot run time: 39707 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c32dabf8f29b5036de748bf9afe6d7add2249863, data reload: false

------ Round 1 ----------------------------------
q1	17590	7447	7246	7246
q2	2050	173	162	162
q3	10608	1102	1171	1102
q4	10573	736	700	700
q5	7589	2678	2702	2678
q6	240	146	145	145
q7	995	631	596	596
q8	9255	1824	1897	1824
q9	6603	6499	6449	6449
q10	7033	2311	2262	2262
q11	475	253	258	253
q12	424	234	223	223
q13	17797	2986	3025	2986
q14	243	221	214	214
q15	567	536	505	505
q16	685	577	577	577
q17	969	570	560	560
q18	7145	6590	6537	6537
q19	1339	990	929	929
q20	472	185	173	173
q21	4264	3282	3323	3282
q22	377	304	310	304
Total cold run time: 107293 ms
Total hot run time: 39707 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7245	7201	7197	7197
q2	336	231	230	230
q3	2859	2812	2917	2812
q4	2109	1787	1828	1787
q5	5730	5659	5658	5658
q6	224	139	139	139
q7	2310	1802	1838	1802
q8	3376	3528	3495	3495
q9	8898	9019	8992	8992
q10	3592	3551	3530	3530
q11	606	510	502	502
q12	816	603	615	603
q13	12103	3259	3244	3244
q14	303	269	284	269
q15	579	516	519	516
q16	709	638	656	638
q17	1851	1641	1619	1619
q18	8171	7660	7617	7617
q19	2354	1552	1586	1552
q20	2086	1943	1907	1907
q21	5730	5532	5515	5515
q22	640	556	564	556
Total cold run time: 72627 ms
Total hot run time: 60180 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196157 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c32dabf8f29b5036de748bf9afe6d7add2249863, data reload: false

query1	1286	946	957	946
query2	6237	2018	2071	2018
query3	10987	4405	4456	4405
query4	33056	23448	23378	23378
query5	3556	450	434	434
query6	272	172	169	169
query7	3977	308	298	298
query8	298	241	239	239
query9	9363	2713	2713	2713
query10	465	241	245	241
query11	17783	15173	15296	15173
query12	137	98	102	98
query13	1561	414	410	410
query14	9036	7573	6803	6803
query15	273	227	200	200
query16	8200	510	494	494
query17	1676	634	591	591
query18	2164	312	320	312
query19	398	165	171	165
query20	126	118	115	115
query21	220	107	114	107
query22	4675	4677	4626	4626
query23	35053	34021	33977	33977
query24	10481	2471	2537	2471
query25	599	377	416	377
query26	1157	148	153	148
query27	2698	273	283	273
query28	7922	2508	2517	2508
query29	826	427	426	426
query30	238	152	157	152
query31	1055	829	897	829
query32	96	54	55	54
query33	799	303	315	303
query34	1098	509	512	509
query35	948	763	772	763
query36	1131	954	962	954
query37	139	71	72	71
query38	4486	4492	4353	4353
query39	1520	1474	1439	1439
query40	200	126	96	96
query41	44	43	45	43
query42	112	97	96	96
query43	528	508	493	493
query44	1286	817	827	817
query45	191	179	171	171
query46	1196	763	714	714
query47	2031	1950	1914	1914
query48	428	331	349	331
query49	962	392	391	391
query50	847	384	397	384
query51	7475	7358	7273	7273
query52	100	87	93	87
query53	258	185	191	185
query54	1309	407	416	407
query55	100	74	78	74
query56	261	224	237	224
query57	1266	1071	1113	1071
query58	228	218	205	205
query59	3396	2948	2938	2938
query60	264	245	250	245
query61	107	106	108	106
query62	875	683	671	671
query63	211	189	181	181
query64	4994	661	708	661
query65	3272	3171	3189	3171
query66	1394	302	297	297
query67	15992	15637	15513	15513
query68	4903	550	533	533
query69	429	251	245	245
query70	1215	1124	1164	1124
query71	333	256	246	246
query72	6341	4000	3997	3997
query73	772	353	357	353
query74	10090	8901	8999	8901
query75	3414	2619	2693	2619
query76	2641	1161	1122	1122
query77	375	271	260	260
query78	10401	9609	9439	9439
query79	2226	590	614	590
query80	1070	416	420	416
query81	546	237	239	237
query82	843	120	120	120
query83	225	140	142	140
query84	236	72	69	69
query85	1457	305	297	297
query86	456	296	309	296
query87	4794	4580	4586	4580
query88	3559	2259	2180	2180
query89	427	290	282	282
query90	1997	186	184	184
query91	133	104	100	100
query92	65	46	48	46
query93	2188	531	532	531
query94	953	279	283	279
query95	357	242	252	242
query96	630	280	286	280
query97	2859	2649	2623	2623
query98	223	188	192	188
query99	1549	1337	1354	1337
Total cold run time: 303026 ms
Total hot run time: 196157 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.59 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c32dabf8f29b5036de748bf9afe6d7add2249863, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.63	0.10	0.10
query5	0.43	0.42	0.39
query6	1.15	0.67	0.65
query7	0.03	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.50	0.51
query10	0.55	0.59	0.56
query11	0.14	0.11	0.10
query12	0.14	0.11	0.12
query13	0.61	0.61	0.61
query14	2.71	2.80	2.76
query15	0.92	0.82	0.82
query16	0.39	0.38	0.38
query17	1.04	1.05	0.99
query18	0.23	0.22	0.22
query19	1.88	1.83	2.02
query20	0.01	0.01	0.01
query21	15.35	0.60	0.58
query22	2.67	2.66	1.44
query23	16.81	1.29	0.86
query24	3.20	1.79	0.37
query25	0.17	0.21	0.16
query26	0.58	0.13	0.14
query27	0.05	0.04	0.04
query28	10.62	1.10	1.07
query29	12.50	3.23	3.17
query30	0.25	0.06	0.06
query31	2.86	0.38	0.36
query32	3.28	0.46	0.48
query33	2.97	3.20	3.01
query34	17.01	4.41	4.44
query35	4.52	4.47	4.48
query36	0.66	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.79 s
Total hot run time: 31.59 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40301 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c7a7652baa16131f39358e94096e53ce7cf1db26, data reload: false

------ Round 1 ----------------------------------
q1	17608	7503	7324	7324
q2	2058	182	173	173
q3	10548	1166	1148	1148
q4	10223	752	724	724
q5	7608	2739	2745	2739
q6	240	150	147	147
q7	992	629	603	603
q8	9250	1966	1870	1870
q9	6745	6582	6642	6582
q10	6991	2335	2334	2334
q11	466	268	255	255
q12	418	220	215	215
q13	17774	2995	3002	2995
q14	244	209	216	209
q15	582	536	524	524
q16	643	595	590	590
q17	973	536	565	536
q18	7406	6657	6687	6657
q19	1361	1095	1049	1049
q20	478	179	181	179
q21	4099	3141	3198	3141
q22	370	307	320	307
Total cold run time: 107077 ms
Total hot run time: 40301 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7297	7242	7251	7242
q2	331	227	232	227
q3	2890	2816	2795	2795
q4	2011	1799	1763	1763
q5	5390	5419	5431	5419
q6	223	144	142	142
q7	2157	1752	1712	1712
q8	3301	3419	3459	3419
q9	8689	8612	8668	8612
q10	3520	3435	3453	3435
q11	589	491	489	489
q12	789	608	642	608
q13	13319	3032	3038	3032
q14	292	272	269	269
q15	561	534	505	505
q16	669	642	640	640
q17	1846	1594	1564	1564
q18	7764	7565	7565	7565
q19	1716	1520	1596	1520
q20	2067	1789	1805	1789
q21	5501	5385	5195	5195
q22	597	552	547	547
Total cold run time: 71519 ms
Total hot run time: 58489 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190024 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c7a7652baa16131f39358e94096e53ce7cf1db26, data reload: false

query1	973	400	389	389
query2	7267	2156	2151	2151
query3	6349	222	215	215
query4	33180	23428	23354	23354
query5	4323	478	448	448
query6	286	191	192	191
query7	4303	316	313	313
query8	314	242	231	231
query9	9213	2717	2710	2710
query10	466	256	263	256
query11	17994	15252	15123	15123
query12	155	100	103	100
query13	1773	427	419	419
query14	9713	6633	6721	6633
query15	251	189	186	186
query16	7992	457	446	446
query17	1618	570	532	532
query18	2121	295	290	290
query19	336	153	145	145
query20	115	104	104	104
query21	206	101	106	101
query22	4599	4305	4402	4305
query23	36030	33604	34376	33604
query24	11484	2508	2428	2428
query25	539	370	383	370
query26	1114	149	157	149
query27	2866	280	284	280
query28	8066	2451	2443	2443
query29	663	434	393	393
query30	296	149	154	149
query31	1031	800	820	800
query32	106	57	58	57
query33	761	294	284	284
query34	972	519	515	515
query35	875	734	742	734
query36	1061	951	949	949
query37	202	72	77	72
query38	4403	4177	4201	4177
query39	1461	1424	1404	1404
query40	286	102	99	99
query41	48	46	45	45
query42	117	101	99	99
query43	564	503	497	497
query44	1297	834	844	834
query45	193	173	164	164
query46	1177	701	708	701
query47	1946	1842	1821	1821
query48	412	328	330	328
query49	1275	386	372	372
query50	808	382	379	379
query51	7284	7059	7046	7046
query52	100	92	90	90
query53	259	179	181	179
query54	955	391	415	391
query55	84	81	84	81
query56	269	230	236	230
query57	1255	1130	1117	1117
query58	223	221	234	221
query59	3298	3109	3048	3048
query60	297	240	253	240
query61	111	104	106	104
query62	890	663	664	663
query63	216	196	199	196
query64	5005	653	640	640
query65	3268	3285	3257	3257
query66	1290	315	322	315
query67	15931	15724	15558	15558
query68	3520	575	571	571
query69	401	265	262	262
query70	1216	1092	1140	1092
query71	338	300	260	260
query72	5707	4149	4121	4121
query73	763	358	370	358
query74	9879	8872	8957	8872
query75	3429	2648	2655	2648
query76	2530	972	1092	972
query77	418	286	286	286
query78	10221	9314	9424	9314
query79	1338	601	605	601
query80	943	481	460	460
query81	524	256	236	236
query82	1301	128	123	123
query83	236	156	152	152
query84	241	77	77	77
query85	1345	415	315	315
query86	354	301	293	293
query87	4837	4641	4547	4547
query88	3545	2238	2195	2195
query89	407	301	302	301
query90	2018	188	188	188
query91	141	104	104	104
query92	62	56	54	54
query93	1083	541	547	541
query94	892	293	294	293
query95	353	262	245	245
query96	603	287	280	280
query97	2826	2604	2700	2604
query98	220	203	197	197
query99	1555	1380	1297	1297
Total cold run time: 297581 ms
Total hot run time: 190024 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.94 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c7a7652baa16131f39358e94096e53ce7cf1db26, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.24	0.08	0.07
query4	1.63	0.10	0.10
query5	0.44	0.40	0.41
query6	1.16	0.64	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.57	0.52	0.50
query10	0.56	0.59	0.55
query11	0.14	0.11	0.11
query12	0.13	0.11	0.11
query13	0.60	0.60	0.60
query14	2.73	2.72	2.85
query15	0.89	0.81	0.84
query16	0.39	0.40	0.38
query17	1.09	1.02	0.98
query18	0.23	0.21	0.21
query19	1.96	1.86	1.99
query20	0.02	0.00	0.01
query21	15.36	0.61	0.60
query22	2.62	1.85	1.99
query23	17.10	0.93	0.79
query24	3.29	1.47	0.52
query25	0.32	0.05	0.09
query26	0.48	0.14	0.14
query27	0.05	0.05	0.04
query28	10.94	1.10	1.08
query29	12.55	3.21	3.21
query30	0.24	0.05	0.05
query31	2.87	0.39	0.37
query32	3.30	0.47	0.46
query33	2.98	3.00	3.02
query34	17.01	4.47	4.41
query35	4.53	4.50	4.48
query36	0.63	0.48	0.47
query37	0.09	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.17	0.12	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.02	0.03
Total cold run time: 107.71 s
Total hot run time: 31.94 s

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39880 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6022e5c5a00b647192f2f56bd283325dfde102c8, data reload: false

------ Round 1 ----------------------------------
q1	17566	7429	7211	7211
q2	2042	180	165	165
q3	10600	1060	1199	1060
q4	10573	763	773	763
q5	7563	2679	2632	2632
q6	235	145	143	143
q7	981	613	606	606
q8	9452	1859	1938	1859
q9	6582	6419	6454	6419
q10	6979	2301	2322	2301
q11	461	248	261	248
q12	434	226	224	224
q13	17767	3055	3020	3020
q14	239	215	204	204
q15	565	516	521	516
q16	641	580	598	580
q17	968	515	546	515
q18	7284	6848	6819	6819
q19	1350	947	917	917
q20	467	180	180	180
q21	3996	3490	3185	3185
q22	381	324	313	313
Total cold run time: 107126 ms
Total hot run time: 39880 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7193	7198	7200	7198
q2	325	229	233	229
q3	2876	2796	2925	2796
q4	2111	1772	1735	1735
q5	5648	5655	5638	5638
q6	223	137	139	137
q7	2271	1839	1860	1839
q8	3380	3557	3477	3477
q9	8878	9079	9005	9005
q10	3569	3528	3535	3528
q11	604	505	506	505
q12	852	609	598	598
q13	11634	3251	3261	3251
q14	300	284	281	281
q15	586	516	531	516
q16	680	639	645	639
q17	1851	1622	1610	1610
q18	8371	7756	7718	7718
q19	2293	1557	1620	1557
q20	2084	1864	1869	1864
q21	5682	5448	5525	5448
q22	634	565	580	565
Total cold run time: 72045 ms
Total hot run time: 60134 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 196128 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6022e5c5a00b647192f2f56bd283325dfde102c8, data reload: false

query1	1264	1026	929	929
query2	6225	2138	2037	2037
query3	10998	4234	4345	4234
query4	66707	29288	23529	23529
query5	4908	439	469	439
query6	425	186	175	175
query7	5656	309	299	299
query8	332	244	234	234
query9	9325	2721	2716	2716
query10	459	252	255	252
query11	17202	15033	15779	15033
query12	161	105	103	103
query13	1553	447	428	428
query14	10416	6633	7206	6633
query15	226	191	197	191
query16	7117	467	429	429
query17	1252	590	575	575
query18	1867	305	301	301
query19	192	151	165	151
query20	124	112	120	112
query21	210	102	102	102
query22	4705	4562	4561	4561
query23	34449	34103	34135	34103
query24	5540	2524	2506	2506
query25	485	389	394	389
query26	658	152	155	152
query27	1868	285	297	285
query28	4470	2507	2508	2507
query29	663	422	413	413
query30	209	157	158	157
query31	993	848	862	848
query32	61	54	52	52
query33	410	280	294	280
query34	977	524	533	524
query35	844	771	737	737
query36	1085	946	989	946
query37	122	75	71	71
query38	4388	4302	4496	4302
query39	1501	1467	1441	1441
query40	201	99	96	96
query41	43	44	42	42
query42	108	97	95	95
query43	551	500	505	500
query44	1212	877	840	840
query45	183	172	162	162
query46	1159	733	726	726
query47	1984	1905	1874	1874
query48	430	320	329	320
query49	734	389	403	389
query50	849	409	401	401
query51	7445	7254	7172	7172
query52	95	85	88	85
query53	257	186	177	177
query54	524	411	411	411
query55	80	76	73	73
query56	248	236	239	236
query57	1217	1132	1105	1105
query58	215	208	226	208
query59	3430	3268	3258	3258
query60	267	244	241	241
query61	104	110	104	104
query62	808	668	686	668
query63	216	187	197	187
query64	1354	661	634	634
query65	3263	3230	3173	3173
query66	705	292	299	292
query67	16061	15665	15619	15619
query68	3984	592	561	561
query69	417	255	251	251
query70	1182	1108	1128	1108
query71	365	249	257	249
query72	6420	4047	4043	4043
query73	775	358	356	356
query74	10109	8901	9050	8901
query75	3442	2678	2818	2678
query76	1830	1060	1151	1060
query77	474	281	277	277
query78	10580	9412	9606	9412
query79	1343	602	598	598
query80	851	435	441	435
query81	489	225	227	225
query82	1218	119	113	113
query83	254	138	134	134
query84	276	76	69	69
query85	862	308	306	306
query86	336	306	303	303
query87	4559	4444	4819	4444
query88	3452	2205	2194	2194
query89	420	293	291	291
query90	2001	190	185	185
query91	139	118	107	107
query92	72	47	54	47
query93	1598	550	559	550
query94	828	300	287	287
query95	352	249	251	249
query96	608	276	286	276
query97	2865	2671	2677	2671
query98	233	193	200	193
query99	1584	1320	1291	1291
Total cold run time: 317708 ms
Total hot run time: 196128 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6022e5c5a00b647192f2f56bd283325dfde102c8, data reload: false

query1	0.04	0.03	0.05
query2	0.08	0.04	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.10
query5	0.42	0.40	0.42
query6	1.14	0.65	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.52	0.51
query10	0.55	0.59	0.55
query11	0.14	0.11	0.10
query12	0.14	0.10	0.11
query13	0.60	0.61	0.60
query14	2.67	2.71	2.84
query15	0.89	0.82	0.82
query16	0.39	0.39	0.40
query17	1.06	1.08	1.06
query18	0.23	0.22	0.21
query19	1.88	1.73	1.94
query20	0.02	0.01	0.02
query21	15.39	0.60	0.61
query22	2.62	2.52	1.76
query23	17.13	1.06	0.74
query24	2.68	1.26	1.03
query25	0.17	0.16	0.07
query26	0.57	0.14	0.12
query27	0.05	0.05	0.04
query28	10.70	1.09	1.06
query29	12.58	3.22	3.25
query30	0.25	0.06	0.06
query31	2.88	0.38	0.38
query32	3.30	0.45	0.46
query33	3.01	3.09	3.07
query34	17.32	4.46	4.44
query35	4.56	4.47	4.51
query36	0.66	0.49	0.48
query37	0.10	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.1 s
Total hot run time: 32.4 s

@LiBinfeng-01
Copy link
Collaborator Author

run p0

@@ -770,41 +770,38 @@ public LogicalPlan visitCreateScheduledJob(DorisParser.CreateScheduledJobContext
return new CreateJobCommand(createJobInfo);
}

private void checkJobNameKey(String key, String keyFormat) {
if (key.isEmpty() || !key.equalsIgnoreCase(keyFormat)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why add key.isEmpty()?

@@ -770,41 +770,38 @@ public LogicalPlan visitCreateScheduledJob(DorisParser.CreateScheduledJobContext
return new CreateJobCommand(createJobInfo);
}

private void checkJobNameKey(String key, String keyFormat) {
if (key.isEmpty() || !key.equalsIgnoreCase(keyFormat)) {
throw new ParseException(keyFormat + " should be: '" + keyFormat + "'");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error message is not very clear, we should use ParseException with Origin, or different statement use different error message to let user could know the error reason easily

Comment on lines 800 to 801
checkJobNameKey(stripQuotes(ctx.jobNameKey.getText()), "jobName");
checkJobNameKey(stripQuotes(ctx.taskIdKey.getText()), "taskId");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think must write jobName before taskId is a good idea. since we use where here, we should support jobName = xxx and taskId = xx and taskId = xx and jobName = xxx

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants