Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](Nereids) insert into with cte cannot be explained #46384

Merged
merged 1 commit into from
Jan 7, 2025
Merged

[fix](Nereids) insert into with cte cannot be explained #46384

merged 1 commit into from
Jan 7, 2025

Conversation

morrySnow
Copy link
Contributor

What problem does this PR solve?

Related PR: #36150

Problem Summary:

not assemble plan with CTE when do explain

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 3, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow
Copy link
Contributor Author

run buildall

@morrySnow morrySnow requested a review from 924060929 January 3, 2025 10:22
@doris-robot
Copy link

TPC-H: Total hot run time: 32891 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7f01c1506bd14ce0d6602c7decc4c8b256a65cac, data reload: false

------ Round 1 ----------------------------------
q1	17589	6148	6199	6148
q2	2051	297	158	158
q3	10428	1251	768	768
q4	10204	862	433	433
q5	7503	2195	1996	1996
q6	208	184	150	150
q7	903	766	608	608
q8	9244	1379	1223	1223
q9	5286	4993	4961	4961
q10	6774	2322	1838	1838
q11	468	279	252	252
q12	363	366	223	223
q13	17794	3663	3078	3078
q14	230	232	221	221
q15	545	507	489	489
q16	629	628	590	590
q17	578	857	328	328
q18	6862	6572	6342	6342
q19	1775	969	573	573
q20	307	320	183	183
q21	2947	2205	2026	2026
q22	364	341	303	303
Total cold run time: 103052 ms
Total hot run time: 32891 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6321	6276	6307	6276
q2	238	323	233	233
q3	2214	2658	2263	2263
q4	1435	1840	1336	1336
q5	4373	4708	4773	4708
q6	188	189	142	142
q7	2045	1940	1750	1750
q8	2620	2844	2684	2684
q9	7356	7348	7222	7222
q10	3048	3311	2855	2855
q11	575	509	490	490
q12	629	719	559	559
q13	3455	3916	3263	3263
q14	298	321	294	294
q15	583	515	487	487
q16	629	693	630	630
q17	1231	1708	1256	1256
q18	7753	7599	7245	7245
q19	879	1067	1150	1067
q20	2008	2021	1897	1897
q21	5694	5199	4773	4773
q22	609	609	576	576
Total cold run time: 54181 ms
Total hot run time: 52006 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195721 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7f01c1506bd14ce0d6602c7decc4c8b256a65cac, data reload: false

query1	1288	966	900	900
query2	6419	2291	2340	2291
query3	10904	4718	4478	4478
query4	33269	23917	23163	23163
query5	3703	613	441	441
query6	263	195	187	187
query7	3981	498	299	299
query8	291	255	247	247
query9	9349	2642	2617	2617
query10	461	317	238	238
query11	17829	15310	15012	15012
query12	157	102	98	98
query13	1561	553	390	390
query14	9157	6671	7170	6671
query15	271	222	208	208
query16	7929	620	457	457
query17	1509	758	586	586
query18	2010	447	314	314
query19	217	185	157	157
query20	145	114	114	114
query21	207	129	112	112
query22	4637	4828	4654	4654
query23	34454	33834	33729	33729
query24	6430	2311	2318	2311
query25	444	455	413	413
query26	817	288	164	164
query27	2128	466	350	350
query28	5576	2460	2441	2441
query29	616	568	456	456
query30	218	185	154	154
query31	998	922	837	837
query32	70	62	59	59
query33	478	367	309	309
query34	781	827	520	520
query35	837	856	770	770
query36	1025	1042	983	983
query37	117	118	80	80
query38	4261	4169	4266	4169
query39	1500	1483	1500	1483
query40	210	116	101	101
query41	45	45	44	44
query42	124	107	102	102
query43	492	542	496	496
query44	1289	819	818	818
query45	180	171	173	171
query46	895	1066	655	655
query47	1975	1994	1928	1928
query48	384	406	323	323
query49	715	504	419	419
query50	643	659	390	390
query51	7116	7282	7255	7255
query52	103	99	91	91
query53	235	256	190	190
query54	485	492	398	398
query55	85	78	74	74
query56	255	249	248	248
query57	1234	1220	1194	1194
query58	235	221	219	219
query59	3120	3316	3107	3107
query60	282	261	249	249
query61	111	103	107	103
query62	906	807	769	769
query63	231	187	193	187
query64	3081	1010	668	668
query65	3350	3286	3268	3268
query66	955	407	302	302
query67	16202	15872	15483	15483
query68	7946	694	523	523
query69	490	297	244	244
query70	1199	1077	1140	1077
query71	430	276	245	245
query72	6489	3860	3819	3819
query73	643	759	354	354
query74	10033	9052	9045	9045
query75	4032	3115	2638	2638
query76	3631	1175	760	760
query77	755	364	273	273
query78	10131	10007	9489	9489
query79	3931	812	576	576
query80	724	510	417	417
query81	464	360	222	222
query82	576	149	123	123
query83	192	160	149	149
query84	279	86	74	74
query85	783	373	306	306
query86	350	319	308	308
query87	4696	4339	4420	4339
query88	4534	2167	2161	2161
query89	428	322	305	305
query90	1865	186	187	186
query91	135	133	106	106
query92	68	54	54	54
query93	2609	875	533	533
query94	649	398	279	279
query95	331	264	250	250
query96	483	608	277	277
query97	2899	3014	2843	2843
query98	212	205	200	200
query99	1736	1567	1432	1432
Total cold run time: 294968 ms
Total hot run time: 195721 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.99 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7f01c1506bd14ce0d6602c7decc4c8b256a65cac, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.04	0.04
query3	0.24	0.06	0.07
query4	1.60	0.10	0.10
query5	0.40	0.41	0.40
query6	1.15	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.50
query10	0.55	0.57	0.54
query11	0.14	0.10	0.11
query12	0.13	0.11	0.11
query13	0.60	0.62	0.61
query14	2.70	2.81	2.85
query15	0.90	0.82	0.82
query16	0.38	0.36	0.38
query17	1.07	1.04	1.02
query18	0.23	0.21	0.20
query19	1.90	1.81	2.01
query20	0.02	0.01	0.01
query21	15.38	0.93	0.60
query22	0.74	0.74	0.72
query23	15.24	1.42	0.53
query24	3.54	0.77	1.45
query25	0.20	0.13	0.06
query26	0.26	0.15	0.13
query27	0.05	0.04	0.04
query28	13.77	1.51	1.04
query29	12.56	3.97	3.28
query30	0.24	0.09	0.06
query31	2.83	0.58	0.38
query32	3.23	0.53	0.46
query33	3.05	3.13	3.00
query34	16.63	5.11	4.48
query35	4.42	4.47	4.47
query36	0.78	0.50	0.47
query37	0.10	0.06	0.05
query38	0.04	0.03	0.04
query39	0.03	0.02	0.02
query40	0.18	0.13	0.14
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.18 s
Total hot run time: 30.99 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 6, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 6, 2025

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 6, 2025

PR approved by anyone and no changes requested.

@morrySnow morrySnow force-pushed the fix_cte_with_insert_explain branch from 7f01c15 to c65f499 Compare January 6, 2025 15:01
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33013 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c65f49999c3394285da3b68db13789d93f66ef06, data reload: false

------ Round 1 ----------------------------------
q1	17612	6193	6076	6076
q2	2045	294	176	176
q3	10412	1311	739	739
q4	10222	877	445	445
q5	7515	2226	1997	1997
q6	205	179	145	145
q7	901	748	618	618
q8	9239	1397	1196	1196
q9	5550	4951	5040	4951
q10	6858	2300	1864	1864
q11	484	283	266	266
q12	374	369	231	231
q13	17780	3800	3148	3148
q14	236	238	218	218
q15	566	519	502	502
q16	643	633	574	574
q17	574	852	331	331
q18	6932	6611	6537	6537
q19	1107	975	552	552
q20	305	329	190	190
q21	2874	2237	1944	1944
q22	365	330	313	313
Total cold run time: 102799 ms
Total hot run time: 33013 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6207	6250	6160	6160
q2	236	324	231	231
q3	2255	2641	2272	2272
q4	1429	1870	1348	1348
q5	4376	4778	4659	4659
q6	179	174	138	138
q7	1985	1847	1713	1713
q8	2551	2675	2650	2650
q9	7133	6960	6933	6933
q10	2954	3219	2678	2678
q11	580	511	491	491
q12	630	719	604	604
q13	3306	3714	3098	3098
q14	265	286	257	257
q15	563	509	501	501
q16	654	687	634	634
q17	1171	1692	1254	1254
q18	7164	7258	6910	6910
q19	814	1137	1090	1090
q20	1925	1980	1839	1839
q21	5515	5144	4944	4944
q22	652	597	613	597
Total cold run time: 52544 ms
Total hot run time: 51001 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190679 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c65f49999c3394285da3b68db13789d93f66ef06, data reload: false

query1	926	386	363	363
query2	4620	2305	2357	2305
query3	6145	216	214	214
query4	32435	24073	23714	23714
query5	3102	657	458	458
query6	275	208	197	197
query7	3943	501	309	309
query8	287	249	216	216
query9	8227	2665	2642	2642
query10	400	305	250	250
query11	18375	15349	15200	15200
query12	151	109	106	106
query13	1319	532	390	390
query14	9665	6820	6929	6820
query15	255	207	184	184
query16	7235	611	431	431
query17	1330	729	578	578
query18	2031	409	318	318
query19	209	195	158	158
query20	130	117	118	117
query21	209	127	106	106
query22	4298	4610	4471	4471
query23	34591	33346	33410	33346
query24	7596	2223	2240	2223
query25	485	448	361	361
query26	1049	272	149	149
query27	2628	461	322	322
query28	5766	2450	2410	2410
query29	759	542	405	405
query30	248	180	190	180
query31	1020	865	817	817
query32	85	61	61	61
query33	514	356	281	281
query34	736	829	500	500
query35	791	855	750	750
query36	1022	1059	951	951
query37	130	98	75	75
query38	4064	4167	4155	4155
query39	1491	1412	1429	1412
query40	204	112	98	98
query41	44	45	45	45
query42	123	104	103	103
query43	520	525	491	491
query44	1283	803	824	803
query45	177	179	165	165
query46	873	1036	645	645
query47	1944	1958	1898	1898
query48	373	397	312	312
query49	823	469	392	392
query50	627	654	383	383
query51	7176	7120	7099	7099
query52	103	101	90	90
query53	231	258	180	180
query54	490	480	413	413
query55	83	77	81	77
query56	255	259	252	252
query57	1229	1208	1121	1121
query58	251	249	223	223
query59	3118	3215	3027	3027
query60	311	261	259	259
query61	113	106	107	106
query62	871	808	717	717
query63	234	198	198	198
query64	4338	989	659	659
query65	3322	3184	3250	3184
query66	1013	475	308	308
query67	16262	15824	15573	15573
query68	7740	691	502	502
query69	461	297	256	256
query70	1189	1120	1136	1120
query71	436	273	260	260
query72	6236	3868	3884	3868
query73	691	751	357	357
query74	10225	9380	8869	8869
query75	3638	3162	2659	2659
query76	3642	1198	775	775
query77	732	353	267	267
query78	10119	10429	9605	9605
query79	1891	789	616	616
query80	586	530	434	434
query81	483	270	232	232
query82	403	150	122	122
query83	170	164	150	150
query84	255	95	70	70
query85	755	407	303	303
query86	393	317	296	296
query87	4578	4546	4490	4490
query88	4430	2189	2149	2149
query89	404	341	302	302
query90	1886	187	200	187
query91	133	197	110	110
query92	65	62	55	55
query93	1218	860	532	532
query94	646	379	280	280
query95	324	265	259	259
query96	495	604	276	276
query97	2924	2970	2866	2866
query98	236	200	194	194
query99	1722	1550	1452	1452
Total cold run time: 284039 ms
Total hot run time: 190679 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.19 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c65f49999c3394285da3b68db13789d93f66ef06, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.11
query5	0.43	0.42	0.40
query6	1.13	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.51	0.50
query10	0.55	0.57	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.61	0.58
query14	2.73	2.75	2.84
query15	0.90	0.83	0.83
query16	0.39	0.38	0.39
query17	1.05	1.05	1.04
query18	0.22	0.21	0.20
query19	1.95	1.87	1.98
query20	0.01	0.01	0.01
query21	15.35	0.93	0.59
query22	0.77	0.80	0.61
query23	15.33	1.43	0.49
query24	3.34	1.50	0.95
query25	0.12	0.17	0.13
query26	0.27	0.14	0.14
query27	0.07	0.07	0.04
query28	14.01	1.52	1.05
query29	12.57	3.95	3.22
query30	0.24	0.08	0.06
query31	2.82	0.62	0.39
query32	3.23	0.53	0.46
query33	3.12	3.15	3.11
query34	16.84	5.11	4.49
query35	4.50	4.44	4.50
query36	0.65	0.51	0.48
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.15	0.13
query41	0.09	0.02	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 106.55 s
Total hot run time: 31.19 s

@morrySnow morrySnow merged commit aba84f9 into apache:master Jan 7, 2025
24 checks passed
@morrySnow morrySnow deleted the fix_cte_with_insert_explain branch January 7, 2025 02:02
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 7, 2025
@morrySnow
branch-2.1: [fix](Nereids) insert into with cte cannot be explained (a…
44e90ab 
…pache#46384)

pick from master apache#46384
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 7, 2025
@morrySnow
branch-2.1: [fix](Nereids) insert into with cte cannot be explained (a…
220436b 
…pache#46384)

pick from master apache#46384
@morrySnow morrySnow mentioned this pull request Jan 7, 2025
Merged
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 7, 2025
@morrySnow
branch-3.0: [fix](Nereids) insert into with cte cannot be explained (a…
3f98c64 
…pache#46384)

pick from master apache#46384
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 7, 2025
@morrySnow
branch-3.0: [fix](Nereids) insert into with cte cannot be explained (a…
25b6630 
…pache#46384)

pick from master apache#46384
@morrySnow morrySnow mentioned this pull request Jan 7, 2025
Merged
yiguolei pushed a commit that referenced this pull request Jan 7, 2025
@morrySnow
branch-2.1: [fix](Nereids) insert into with cte cannot be explained (#…
abbf75e 
…46384) (#46501)

pick from master #46384
morrySnow added a commit that referenced this pull request Jan 7, 2025
@morrySnow
branch-3.0: [fix](Nereids) insert into with cte cannot be explained (#…
22c6895 
…46384) (#46504)

pick from master #46384
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@starocean999 starocean999 starocean999 approved these changes

@924060929 924060929 924060929 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@morrySnow @Thearas @doris-robot @924060929 @starocean999 @yiguolei