Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix](nereids) fix create view and alter view using old parser checking #43263

Merged
merged 5 commits into from
Nov 22, 2024

Conversation

feiniaofeiafei
Copy link
Contributor

@feiniaofeiafei feiniaofeiafei commented Nov 5, 2024

What problem does this PR solve?

Problem:
Creating a view on a table with materialized views (MV) results in an error: failed to init view stmt. For example, executing the following SQL:
create view v1 as select * from t1 index t_mv_mv.

Root Cause:
The syntax index t_mv_mv is supported only by the new planner and not by the old planner. During the view creation process, regardless of whether the new or old optimizer is used, the process eventually calls the createView method in Env. In the createView method, the persisted SQL of the view is parsed using the old optimizer's parser. If the parsing fails, an error is thrown.
As a result, if the SQL for creating a view includes syntax supported only by the new optimizer, an error occurs.

Solution:
This PR addresses the issue by removing the parsing check from the createView method in Env. Instead, the check is moved to the respective classes responsible for creating views in the new and old optimizers.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Check List (For Committer)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No colde files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.
  • Release note

    None

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@feiniaofeiafei
Copy link
Contributor Author

run buildall

1 similar comment
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 41561 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc350929c403a7aaa76865da8b605a5020896ca4, data reload: false

------ Round 1 ----------------------------------
q1	17644	7575	7339	7339
q2	2065	179	154	154
q3	10630	1169	1174	1169
q4	10330	800	879	800
q5	7759	3184	3080	3080
q6	238	144	148	144
q7	1023	616	609	609
q8	9361	1993	2032	1993
q9	6675	6502	6542	6502
q10	7060	2403	2423	2403
q11	465	256	255	255
q12	406	215	209	209
q13	17783	3011	2996	2996
q14	231	217	205	205
q15	577	536	506	506
q16	637	585	588	585
q17	987	589	590	589
q18	7522	6872	6689	6689
q19	1326	952	1037	952
q20	479	178	176	176
q21	4019	3182	3261	3182
q22	1097	1034	1024	1024
Total cold run time: 108314 ms
Total hot run time: 41561 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7411	7254	7316	7254
q2	322	225	227	225
q3	2940	2827	2809	2809
q4	1950	1709	1670	1670
q5	5470	5510	5516	5510
q6	220	136	137	136
q7	2156	1716	1722	1716
q8	3270	3470	3458	3458
q9	8619	8654	8560	8560
q10	3531	3489	3451	3451
q11	592	490	506	490
q12	790	613	591	591
q13	9199	3018	2992	2992
q14	300	263	260	260
q15	569	518	510	510
q16	685	629	637	629
q17	1844	1611	1575	1575
q18	7859	7563	7601	7563
q19	1699	1557	1583	1557
q20	2090	1848	1827	1827
q21	5454	5087	5327	5087
q22	1131	1025	1015	1015
Total cold run time: 68101 ms
Total hot run time: 58885 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191835 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc350929c403a7aaa76865da8b605a5020896ca4, data reload: false

query1	971	374	360	360
query2	6509	2080	2114	2080
query3	6779	216	216	216
query4	34038	23691	23574	23574
query5	4299	451	441	441
query6	259	167	173	167
query7	4586	289	300	289
query8	281	226	218	218
query9	9438	2659	2641	2641
query10	484	266	249	249
query11	18255	15248	15262	15248
query12	164	101	102	101
query13	1668	438	410	410
query14	10585	7062	7504	7062
query15	274	180	173	173
query16	8128	481	497	481
query17	1760	592	568	568
query18	2139	296	295	295
query19	370	156	147	147
query20	116	112	107	107
query21	206	101	103	101
query22	4457	4310	4287	4287
query23	34855	34372	34088	34088
query24	11132	2726	2764	2726
query25	684	400	396	396
query26	1419	159	156	156
query27	2784	280	284	280
query28	7981	2448	2414	2414
query29	924	426	436	426
query30	314	160	157	157
query31	1016	798	828	798
query32	96	57	60	57
query33	763	281	267	267
query34	957	517	530	517
query35	923	723	721	721
query36	1116	974	958	958
query37	141	80	74	74
query38	4483	4309	4274	4274
query39	1472	1422	1432	1422
query40	277	99	101	99
query41	50	48	46	46
query42	111	98	99	98
query43	532	485	489	485
query44	1237	829	814	814
query45	188	168	166	166
query46	1159	703	698	698
query47	1958	1869	1844	1844
query48	418	328	326	326
query49	1161	396	399	396
query50	802	387	382	382
query51	7358	7217	7008	7008
query52	102	86	87	86
query53	255	181	184	181
query54	1312	410	424	410
query55	80	77	81	77
query56	264	245	253	245
query57	1289	1161	1151	1151
query58	245	211	211	211
query59	3243	2933	2966	2933
query60	267	246	240	240
query61	106	103	103	103
query62	898	687	662	662
query63	206	182	183	182
query64	5287	666	610	610
query65	3308	3214	3209	3209
query66	1433	319	316	316
query67	16245	15681	15736	15681
query68	5003	559	546	546
query69	429	250	252	250
query70	1203	1083	1120	1083
query71	419	246	244	244
query72	6385	4111	4016	4016
query73	761	355	370	355
query74	10275	9100	9068	9068
query75	3416	2670	2645	2645
query76	2905	1080	1085	1080
query77	412	271	288	271
query78	10327	9498	9435	9435
query79	1435	596	593	593
query80	977	439	412	412
query81	560	242	238	238
query82	1112	121	120	120
query83	214	141	133	133
query84	237	73	69	69
query85	1275	297	290	290
query86	367	296	287	287
query87	4788	4790	4828	4790
query88	3483	2166	2142	2142
query89	389	291	294	291
query90	2045	187	184	184
query91	128	102	112	102
query92	58	49	49	49
query93	1080	556	556	556
query94	978	283	291	283
query95	345	246	238	238
query96	599	273	277	273
query97	2849	2687	2696	2687
query98	218	194	190	190
query99	1587	1310	1352	1310
Total cold run time: 302876 ms
Total hot run time: 191835 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc350929c403a7aaa76865da8b605a5020896ca4, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.64	0.10	0.10
query5	0.42	0.40	0.41
query6	1.16	0.67	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.50	0.50
query10	0.56	0.54	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.59	0.59
query14	2.71	2.84	2.85
query15	0.91	0.82	0.82
query16	0.38	0.38	0.40
query17	1.04	1.06	1.03
query18	0.20	0.20	0.20
query19	1.84	1.82	1.99
query20	0.01	0.01	0.01
query21	15.36	0.62	0.60
query22	2.50	2.25	2.20
query23	17.03	1.03	0.82
query24	3.32	1.54	1.19
query25	0.17	0.21	0.14
query26	0.36	0.13	0.14
query27	0.05	0.06	0.05
query28	10.27	1.09	1.08
query29	12.55	3.28	3.23
query30	0.25	0.06	0.06
query31	2.86	0.39	0.37
query32	3.26	0.47	0.45
query33	2.98	3.00	3.11
query34	17.10	4.52	4.47
query35	4.49	4.53	4.44
query36	0.67	0.49	0.49
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.03	0.03	0.02
query40	0.15	0.12	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 106.43 s
Total hot run time: 33.28 s

@feiniaofeiafei feiniaofeiafei force-pushed the fix_create_view_old_parser branch from fc35092 to 52f90aa Compare November 15, 2024 13:20
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@feiniaofeiafei feiniaofeiafei force-pushed the fix_create_view_old_parser branch from 52f90aa to 77f898d Compare November 18, 2024 03:31
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@feiniaofeiafei feiniaofeiafei force-pushed the fix_create_view_old_parser branch from 77f898d to 6fab571 Compare November 18, 2024 11:48
@feiniaofeiafei
Copy link
Contributor Author

run compile

@feiniaofeiafei feiniaofeiafei force-pushed the fix_create_view_old_parser branch from 1a9dd02 to e97063f Compare November 18, 2024 12:52
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 20, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@morrySnow morrySnow merged commit f8191d8 into apache:master Nov 22, 2024
29 of 30 checks passed
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
…ng (#43263)

Problem:
Creating a view on a table with materialized views (MV) results in an
error: failed to init view stmt. For example, executing the following
SQL:
create view v1 as select * from t1 index t_mv_mv.

Root Cause:
The syntax index t_mv_mv is supported only by the new planner and not by
the old planner. During the view creation process, regardless of whether
the new or old optimizer is used, the process eventually calls the
createView method in Env. In the createView method, the persisted SQL of
the view is parsed using the old optimizer's parser. If the parsing
fails, an error is thrown.
As a result, if the SQL for creating a view includes syntax supported
only by the new optimizer, an error occurs.

Solution:
This PR addresses the issue by removing the parsing check from the
createView method in Env. Instead, the check is moved to the respective
classes responsible for creating views in the new and old optimizers.
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
…ng (#43263)

Problem:
Creating a view on a table with materialized views (MV) results in an
error: failed to init view stmt. For example, executing the following
SQL:
create view v1 as select * from t1 index t_mv_mv.

Root Cause:
The syntax index t_mv_mv is supported only by the new planner and not by
the old planner. During the view creation process, regardless of whether
the new or old optimizer is used, the process eventually calls the
createView method in Env. In the createView method, the persisted SQL of
the view is parsed using the old optimizer's parser. If the parsing
fails, an error is thrown.
As a result, if the SQL for creating a view includes syntax supported
only by the new optimizer, an error occurs.

Solution:
This PR addresses the issue by removing the parsing check from the
createView method in Env. Instead, the check is moved to the respective
classes responsible for creating views in the new and old optimizers.
dataroaring pushed a commit that referenced this pull request Nov 26, 2024
…arser checking #43263 (#44455)

Cherry-picked from #43263

Co-authored-by: feiniaofeiafei <[email protected]>
morrySnow pushed a commit that referenced this pull request Nov 27, 2024
…arser checking #43263 (#44456)

Cherry-picked from #43263

Co-authored-by: feiniaofeiafei <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants