Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[test](mtmv) Add inject statistic when mv rewrite regression test to make sure rewrite result stable #43785

Merged
merged 7 commits into from
Nov 29, 2024
Merged

[test](mtmv) Add inject statistic when mv rewrite regression test to make sure rewrite result stable #43785

merged 7 commits into from
Nov 29, 2024

Conversation

seawinde
Copy link
Contributor

@seawinde seawinde commented Nov 12, 2024 •
edited
Loading

What problem does this PR solve?

The result of successful rewriting by the cbo optimizer depends on the statistics.
The priority of the optimizer consumption statistics in descending order is

  1. the injected statistics
  2. the statistics reported by be
  3. and the statistics analyzed actively.

When the pipeline runs the case, the statistics reported by be may not be timely. Therefore, the outcome that leads to the cbo optimizer's successful selection of overwrites is not very certain, so the statistics are currently injected manually in the test cases

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@seawinde
Copy link
Contributor Author

run buildall

3 similar comments
@seawinde
Copy link
Contributor Author

run buildall

@seawinde
Copy link
Contributor Author

run buildall

@seawinde
Copy link
Contributor Author

run buildall

@seawinde seawinde force-pushed the fix_mv_test_unstable_inject_statistic branch from 8c5650e to e0149ac Compare November 14, 2024 06:23
@seawinde
Copy link
Contributor Author

run buildall

@seawinde seawinde force-pushed the fix_mv_test_unstable_inject_statistic branch from e0149ac to 91409b6 Compare November 14, 2024 23:50
@seawinde
Copy link
Contributor Author

run buildall

@seawinde
Copy link
Contributor Author

run p0 5

morrySnow
morrySnow previously approved these changes Nov 22, 2024
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 22, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@seawinde seawinde force-pushed the fix_mv_test_unstable_inject_statistic branch from 91409b6 to dde9346 Compare November 27, 2024 09:26
@seawinde
Copy link
Contributor Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Nov 27, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 39871 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit dde934637e7ae243c6e2bbb327b5a418c7f5f3d7, data reload: false

------ Round 1 ----------------------------------
q1	17570	7711	7251	7251
q2	2042	172	164	164
q3	10725	1075	1177	1075
q4	10750	719	773	719
q5	7610	2701	2666	2666
q6	247	149	148	148
q7	982	630	613	613
q8	9530	1840	1917	1840
q9	6956	6439	6437	6437
q10	7026	2319	2354	2319
q11	466	259	255	255
q12	419	209	207	207
q13	20884	3025	3026	3025
q14	254	207	218	207
q15	579	519	515	515
q16	680	582	579	579
q17	972	504	607	504
q18	7313	6688	6684	6684
q19	1346	1051	978	978
q20	461	178	178	178
q21	3950	3291	3161	3161
q22	387	346	351	346
Total cold run time: 111149 ms
Total hot run time: 39871 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7388	7357	7351	7351
q2	327	232	228	228
q3	3142	3127	3096	3096
q4	2167	1892	1869	1869
q5	5839	5783	5766	5766
q6	223	143	145	143
q7	2239	1809	1821	1809
q8	3355	3503	3484	3484
q9	8854	8837	8811	8811
q10	3594	3531	3497	3497
q11	598	500	510	500
q12	841	625	638	625
q13	12501	3292	3189	3189
q14	309	285	269	269
q15	575	537	519	519
q16	697	663	660	660
q17	1821	1616	1584	1584
q18	8318	7791	7636	7636
q19	1745	1349	1584	1349
q20	2132	1928	1882	1882
q21	5409	5312	5266	5266
q22	654	562	566	562
Total cold run time: 72728 ms
Total hot run time: 60095 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197014 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit dde934637e7ae243c6e2bbb327b5a418c7f5f3d7, data reload: false

query1	1247	949	930	930
query2	6229	2121	2095	2095
query3	10957	4009	3996	3996
query4	67888	29082	24008	24008
query5	4949	460	446	446
query6	427	188	178	178
query7	5588	289	282	282
query8	294	222	212	212
query9	9142	2681	2693	2681
query10	459	246	252	246
query11	17475	15250	15691	15250
query12	162	100	105	100
query13	1529	414	399	399
query14	10991	7079	7139	7079
query15	217	185	200	185
query16	7219	442	467	442
query17	1243	596	600	596
query18	1831	311	301	301
query19	225	155	158	155
query20	127	117	116	116
query21	208	113	107	107
query22	4619	4803	4483	4483
query23	34771	34627	34344	34344
query24	5457	2465	2441	2441
query25	487	377	427	377
query26	645	150	147	147
query27	1981	279	282	279
query28	4395	2468	2450	2450
query29	684	423	435	423
query30	215	150	143	143
query31	1007	820	881	820
query32	67	57	58	57
query33	446	298	293	293
query34	947	523	522	522
query35	892	756	749	749
query36	1115	963	986	963
query37	123	77	75	75
query38	4421	4379	4485	4379
query39	1558	1494	1472	1472
query40	199	98	95	95
query41	44	41	47	41
query42	110	100	99	99
query43	553	520	508	508
query44	1175	815	798	798
query45	187	163	170	163
query46	1140	707	726	707
query47	2104	1938	1962	1938
query48	428	324	347	324
query49	728	380	391	380
query50	843	390	402	390
query51	7480	7318	7100	7100
query52	99	85	90	85
query53	256	176	179	176
query54	497	393	401	393
query55	76	78	76	76
query56	249	236	248	236
query57	1303	1206	1171	1171
query58	213	206	214	206
query59	3244	3137	3078	3078
query60	258	239	248	239
query61	109	106	103	103
query62	771	657	645	645
query63	215	198	185	185
query64	1357	704	644	644
query65	3297	3190	3215	3190
query66	703	294	290	290
query67	16037	15647	15663	15647
query68	3996	581	556	556
query69	432	248	268	248
query70	1175	1152	1090	1090
query71	356	258	259	258
query72	6400	3997	4070	3997
query73	761	366	355	355
query74	10299	9079	9037	9037
query75	3397	2665	2644	2644
query76	2127	1063	964	964
query77	459	274	283	274
query78	10451	9490	9310	9310
query79	1908	598	585	585
query80	1326	484	427	427
query81	523	241	228	228
query82	1316	123	116	116
query83	225	155	158	155
query84	284	71	67	67
query85	997	304	293	293
query86	412	311	315	311
query87	4729	4650	4640	4640
query88	3630	2228	2295	2228
query89	419	296	292	292
query90	1953	190	184	184
query91	137	102	105	102
query92	60	50	56	50
query93	2535	529	522	522
query94	813	290	279	279
query95	358	249	246	246
query96	633	283	287	283
query97	2874	2708	2645	2645
query98	208	193	194	193
query99	1591	1307	1311	1307
Total cold run time: 322696 ms
Total hot run time: 197014 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit dde934637e7ae243c6e2bbb327b5a418c7f5f3d7, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.04
query3	0.24	0.06	0.07
query4	1.64	0.10	0.10
query5	0.43	0.41	0.40
query6	1.15	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.57	0.53	0.49
query10	0.55	0.58	0.56
query11	0.14	0.09	0.10
query12	0.14	0.12	0.11
query13	0.61	0.61	0.60
query14	2.83	2.85	2.86
query15	0.90	0.82	0.84
query16	0.38	0.38	0.39
query17	1.06	0.96	1.08
query18	0.22	0.22	0.20
query19	1.92	1.74	2.03
query20	0.01	0.01	0.02
query21	15.39	0.58	0.58
query22	3.15	1.65	1.86
query23	17.00	0.83	0.79
query24	3.40	1.33	1.79
query25	0.26	0.19	0.09
query26	0.63	0.13	0.14
query27	0.04	0.04	0.04
query28	9.46	1.12	1.09
query29	12.60	3.26	3.22
query30	0.25	0.07	0.06
query31	2.86	0.39	0.37
query32	3.27	0.47	0.46
query33	2.99	3.06	3.06
query34	17.15	4.44	4.44
query35	4.52	4.55	4.48
query36	0.68	0.48	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.02
query40	0.15	0.12	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 107.06 s
Total hot run time: 32.7 s

@seawinde
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40071 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ed3daf98afd168e98dcf264c244311c7db56c757, data reload: false

------ Round 1 ----------------------------------
q1	17614	7441	7306	7306
q2	2041	178	175	175
q3	10542	1129	1184	1129
q4	10218	784	757	757
q5	7609	2762	2704	2704
q6	240	149	151	149
q7	981	636	620	620
q8	9260	1855	1907	1855
q9	6570	6387	6322	6322
q10	6997	2300	2312	2300
q11	455	259	259	259
q12	407	223	223	223
q13	17758	3035	3108	3035
q14	249	206	220	206
q15	554	524	524	524
q16	651	585	583	583
q17	977	574	543	543
q18	7628	6711	6703	6703
q19	1336	1043	1055	1043
q20	480	195	189	189
q21	3980	3189	3127	3127
q22	384	319	322	319
Total cold run time: 106931 ms
Total hot run time: 40071 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7269	7357	7234	7234
q2	320	230	235	230
q3	2880	2730	3111	2730
q4	2085	1822	1735	1735
q5	5531	5695	5624	5624
q6	223	142	143	142
q7	2251	1784	1787	1784
q8	3382	3555	3489	3489
q9	8905	8913	8870	8870
q10	3609	3537	3540	3537
q11	602	501	488	488
q12	839	645	587	587
q13	13133	3267	3267	3267
q14	293	278	264	264
q15	570	532	523	523
q16	680	648	621	621
q17	1868	1623	1566	1566
q18	7919	7426	7274	7274
q19	1661	1423	1482	1423
q20	2029	1806	1822	1806
q21	5274	5263	5236	5236
q22	620	554	590	554
Total cold run time: 71943 ms
Total hot run time: 58984 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191174 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ed3daf98afd168e98dcf264c244311c7db56c757, data reload: false

query1	962	389	378	378
query2	6515	2176	2082	2082
query3	6711	209	210	209
query4	34272	23674	23525	23525
query5	4323	456	444	444
query6	295	184	171	171
query7	4636	302	304	302
query8	303	227	228	227
query9	9471	2712	2699	2699
query10	468	253	254	253
query11	18203	15314	15156	15156
query12	147	101	100	100
query13	1639	435	401	401
query14	9655	7516	6793	6793
query15	302	167	183	167
query16	8198	421	488	421
query17	1800	567	555	555
query18	2142	293	292	292
query19	362	147	140	140
query20	113	107	119	107
query21	204	102	102	102
query22	4617	4218	4185	4185
query23	35057	34320	34043	34043
query24	10389	2426	2424	2424
query25	635	386	388	386
query26	1400	153	154	153
query27	2883	281	276	276
query28	8024	2424	2466	2424
query29	865	403	419	403
query30	297	150	153	150
query31	1038	785	812	785
query32	97	56	58	56
query33	779	285	321	285
query34	947	496	515	496
query35	905	743	766	743
query36	1107	922	958	922
query37	149	77	75	75
query38	4451	4295	4248	4248
query39	1460	1402	1414	1402
query40	275	99	98	98
query41	47	47	44	44
query42	123	104	103	103
query43	546	490	495	490
query44	1234	800	793	793
query45	182	179	160	160
query46	1142	714	697	697
query47	1971	1847	1867	1847
query48	396	323	315	315
query49	1147	389	386	386
query50	813	388	388	388
query51	7356	7088	7201	7088
query52	101	97	85	85
query53	256	179	180	179
query54	1304	405	415	405
query55	80	78	84	78
query56	265	255	229	229
query57	1362	1170	1144	1144
query58	246	216	215	215
query59	3248	3195	3051	3051
query60	279	263	257	257
query61	136	126	132	126
query62	883	681	688	681
query63	212	190	184	184
query64	5414	757	785	757
query65	3310	3187	3251	3187
query66	1468	315	330	315
query67	16043	15552	15659	15552
query68	4509	564	562	562
query69	420	251	259	251
query70	1174	1122	1107	1107
query71	316	250	246	246
query72	6422	4084	4040	4040
query73	779	448	367	367
query74	10123	9060	9066	9060
query75	3437	2653	2685	2653
query76	2869	1107	1137	1107
query77	385	279	277	277
query78	10502	9501	9376	9376
query79	2506	605	624	605
query80	1146	429	438	429
query81	559	226	230	226
query82	635	120	118	118
query83	239	147	148	147
query84	241	74	74	74
query85	1737	302	295	295
query86	485	307	303	303
query87	4817	4770	4689	4689
query88	3924	2256	2209	2209
query89	401	303	300	300
query90	1983	189	190	189
query91	143	105	108	105
query92	66	60	54	54
query93	1762	545	535	535
query94	849	288	295	288
query95	363	249	248	248
query96	630	286	287	286
query97	2894	2676	2676	2676
query98	229	195	196	195
query99	1589	1341	1323	1323
Total cold run time: 303794 ms
Total hot run time: 191174 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ed3daf98afd168e98dcf264c244311c7db56c757, data reload: false

query1	0.03	0.03	0.02
query2	0.06	0.03	0.03
query3	0.23	0.08	0.08
query4	1.61	0.10	0.10
query5	0.42	0.40	0.41
query6	1.14	0.66	0.64
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.56	0.53	0.50
query10	0.56	0.56	0.56
query11	0.15	0.11	0.11
query12	0.14	0.10	0.12
query13	0.62	0.61	0.60
query14	2.70	2.83	2.83
query15	0.91	0.84	0.82
query16	0.41	0.38	0.38
query17	1.06	1.05	1.05
query18	0.21	0.20	0.21
query19	1.85	1.82	1.99
query20	0.02	0.01	0.02
query21	15.36	0.60	0.58
query22	2.31	2.19	1.85
query23	16.90	0.91	0.94
query24	3.16	0.78	1.14
query25	0.18	0.20	0.11
query26	0.39	0.14	0.14
query27	0.05	0.06	0.04
query28	10.65	1.11	1.08
query29	12.57	3.27	3.20
query30	0.27	0.06	0.06
query31	2.87	0.38	0.37
query32	3.27	0.46	0.46
query33	3.14	3.06	3.01
query34	17.14	4.48	4.48
query35	4.46	4.54	4.48
query36	0.67	0.51	0.48
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.64 s
Total hot run time: 32.63 s

zfr9527
zfr9527 approved these changes Nov 28, 2024
View reviewed changes
Copy link
Contributor

@zfr9527 zfr9527 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 29, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit 9daa3b7 into apache:master Nov 29, 2024
24 of 25 checks passed
seawinde added a commit to seawinde/doris that referenced this pull request Dec 1, 2024
@seawinde
[test](mtmv) Add inject statistic when mv rewrite regression test to …
caebc1d 
…make sure rewrite result stable (apache#43785)

The result of successful rewriting by the cbo optimizer depends on the
statistics.
The priority of the optimizer consumption statistics in descending order
is
1. the injected statistics
2. the statistics reported by be
3. and the statistics analyzed actively.

When the pipeline runs the case, the statistics reported by be may not
be timely. Therefore, the outcome that leads to the cbo optimizer's
successful selection of overwrites is not very certain, so the
statistics are currently injected manually in the test cases
@seawinde seawinde mentioned this pull request Dec 1, 2024
Merged
16 tasks
morrySnow pushed a commit that referenced this pull request Dec 6, 2024
@seawinde
[test](mtmv) Add inject statistic when mv rewrite regression test to …
5980ab0 
…make sure rewrite result stable (#43785) (#44826)

pick from master #43785
seawinde added a commit to seawinde/doris that referenced this pull request Dec 6, 2024
@seawinde
[test](mtmv) Add inject statistic when mv rewrite regression test to …
f59403b 
…make sure rewrite result stable (apache#43785)

The result of successful rewriting by the cbo optimizer depends on the
statistics.
The priority of the optimizer consumption statistics in descending order
is
1. the injected statistics
2. the statistics reported by be
3. and the statistics analyzed actively.

When the pipeline runs the case, the statistics reported by be may not
be timely. Therefore, the outcome that leads to the cbo optimizer's
successful selection of overwrites is not very certain, so the
statistics are currently injected manually in the test cases
@seawinde seawinde mentioned this pull request Dec 6, 2024
Merged
16 tasks
yiguolei pushed a commit that referenced this pull request Dec 9, 2024
@seawinde
Pick some pr to 21 #43010 #43030 #43785 #44779 #44786 #44857 (#45129)
1662e47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow morrySnow approved these changes

@zfr9527 zfr9527 zfr9527 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@seawinde @doris-robot @zfr9527 @morrySnow @yiguolei