Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](row store) forbit short circuit row store cache temporaryly #45842

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

eldenmoon
Copy link
Member

The cache eviction may not be in time

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.80% (10094/26015)
Line Coverage: 29.79% (85150/285862)
Region Coverage: 28.90% (43467/150407)
Branch Coverage: 25.43% (22156/87122)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f06df46f87f53ad6a7eef74ca8eb4a43d27f7672_f06df46f87f53ad6a7eef74ca8eb4a43d27f7672/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 39962 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f06df46f87f53ad6a7eef74ca8eb4a43d27f7672, data reload: false

------ Round 1 ----------------------------------
q1	17644	7873	7383	7383
q2	2047	174	168	168
q3	10821	1069	1216	1069
q4	10541	719	683	683
q5	7607	2737	2615	2615
q6	239	153	151	151
q7	987	618	619	618
q8	9251	1816	1932	1816
q9	6606	6403	6404	6403
q10	7036	2314	2341	2314
q11	463	261	264	261
q12	438	231	226	226
q13	17785	2938	2867	2867
q14	252	219	224	219
q15	552	512	519	512
q16	655	612	617	612
q17	996	534	541	534
q18	7742	6806	6872	6806
q19	1344	994	968	968
q20	492	192	199	192
q21	4139	3420	3225	3225
q22	387	338	320	320
Total cold run time: 108024 ms
Total hot run time: 39962 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7247	7191	7162	7162
q2	328	238	238	238
q3	3027	2948	2958	2948
q4	2155	1883	1913	1883
q5	5655	5686	5668	5668
q6	226	148	148	148
q7	2249	1804	1876	1804
q8	3390	3562	3566	3562
q9	8925	9118	9120	9118
q10	3593	3573	3547	3547
q11	625	504	510	504
q12	810	646	613	613
q13	15229	3105	3100	3100
q14	322	272	293	272
q15	546	523	519	519
q16	704	638	657	638
q17	1842	1596	1598	1596
q18	8426	7697	7560	7560
q19	1777	1487	1458	1458
q20	2081	1942	1895	1895
q21	5471	5479	5492	5479
q22	631	586	554	554
Total cold run time: 75259 ms
Total hot run time: 60266 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 198682 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f06df46f87f53ad6a7eef74ca8eb4a43d27f7672, data reload: false

query1	1295	942	937	937
query2	6232	2383	2418	2383
query3	11113	4832	4748	4748
query4	33622	23512	23520	23512
query5	4010	455	443	443
query6	278	192	184	184
query7	3986	315	317	315
query8	302	231	222	222
query9	9359	2773	2758	2758
query10	474	247	272	247
query11	18060	15168	15328	15168
query12	157	107	103	103
query13	1591	436	427	427
query14	8915	7923	7460	7460
query15	261	193	202	193
query16	8250	467	506	467
query17	1620	624	611	611
query18	2200	337	346	337
query19	370	174	178	174
query20	125	116	117	116
query21	214	111	112	111
query22	4796	4511	4655	4511
query23	35253	36262	34358	34358
query24	10795	2606	2642	2606
query25	519	395	408	395
query26	705	166	153	153
query27	2580	330	340	330
query28	7677	2506	2533	2506
query29	698	448	424	424
query30	230	160	154	154
query31	1067	865	831	831
query32	102	55	60	55
query33	781	303	285	285
query34	976	516	519	516
query35	877	778	765	765
query36	1134	959	953	953
query37	124	81	80	80
query38	4209	4101	4226	4101
query39	1493	1475	1460	1460
query40	231	104	101	101
query41	46	43	42	42
query42	113	101	102	101
query43	553	527	519	519
query44	1296	847	846	846
query45	191	171	170	170
query46	1170	716	732	716
query47	2018	1921	1931	1921
query48	418	324	334	324
query49	879	396	399	396
query50	855	415	413	413
query51	7272	7264	7166	7166
query52	109	92	91	91
query53	265	191	191	191
query54	1201	416	435	416
query55	86	86	81	81
query56	277	248	250	248
query57	1275	1183	1185	1183
query58	246	226	240	226
query59	3446	3259	3127	3127
query60	279	255	258	255
query61	109	114	112	112
query62	841	686	689	686
query63	218	192	194	192
query64	3965	731	680	680
query65	3286	3194	3268	3194
query66	716	303	295	295
query67	15877	15567	15594	15567
query68	5784	571	580	571
query69	451	260	251	251
query70	1215	1134	1162	1134
query71	486	257	256	256
query72	6767	4148	4107	4107
query73	811	369	371	369
query74	10069	8955	8905	8905
query75	3483	2691	2696	2691
query76	3756	1116	963	963
query77	630	283	279	279
query78	10305	9555	9401	9401
query79	1794	609	601	601
query80	947	423	434	423
query81	507	229	237	229
query82	686	113	120	113
query83	211	151	143	143
query84	284	67	71	67
query85	1290	307	303	303
query86	415	307	298	298
query87	4586	4568	4498	4498
query88	3784	2253	2220	2220
query89	446	297	296	296
query90	2078	193	192	192
query91	141	102	107	102
query92	65	51	53	51
query93	2068	569	560	560
query94	714	281	296	281
query95	347	257	253	253
query96	623	287	282	282
query97	2881	2686	2707	2686
query98	235	192	195	192
query99	1627	1307	1313	1307
Total cold run time: 303174 ms
Total hot run time: 198682 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.09 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f06df46f87f53ad6a7eef74ca8eb4a43d27f7672, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.08	0.07
query4	1.61	0.11	0.11
query5	0.42	0.42	0.41
query6	1.15	0.65	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.59	0.49	0.51
query10	0.55	0.59	0.55
query11	0.15	0.10	0.10
query12	0.15	0.10	0.11
query13	0.61	0.61	0.59
query14	2.75	2.86	2.76
query15	0.90	0.82	0.83
query16	0.39	0.37	0.38
query17	1.00	1.06	1.05
query18	0.22	0.21	0.21
query19	1.91	1.85	2.04
query20	0.01	0.01	0.01
query21	15.37	0.60	0.58
query22	2.66	2.33	1.57
query23	16.98	0.94	0.91
query24	2.94	1.49	1.58
query25	0.34	0.20	0.09
query26	0.40	0.15	0.13
query27	0.04	0.05	0.04
query28	9.74	1.13	1.07
query29	12.61	3.25	3.27
query30	0.24	0.06	0.06
query31	2.86	0.40	0.37
query32	3.26	0.47	0.45
query33	3.13	3.07	3.08
query34	17.10	4.49	4.54
query35	4.50	4.54	4.49
query36	0.67	0.48	0.48
query37	0.10	0.07	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.04
Total cold run time: 106.15 s
Total hot run time: 33.09 s

@eldenmoon
Copy link
Member Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 38.81% (10096/26017)
Line Coverage: 29.80% (85204/285938)
Region Coverage: 28.93% (43519/150453)
Branch Coverage: 25.46% (22185/87142)
Coverage Report: http://coverage.selectdb-in.cc/coverage/e6c03d083e7cfbb6819654f4797a379799a644d7_e6c03d083e7cfbb6819654f4797a379799a644d7/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 32771 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e6c03d083e7cfbb6819654f4797a379799a644d7, data reload: false

------ Round 1 ----------------------------------
q1	17609	6126	6088	6088
q2	2052	296	168	168
q3	10429	1276	724	724
q4	10216	884	430	430
q5	7520	2250	2057	2057
q6	203	185	148	148
q7	909	766	606	606
q8	9216	1357	1188	1188
q9	5305	4952	4986	4952
q10	6748	2328	1864	1864
q11	477	291	254	254
q12	346	360	218	218
q13	17765	3613	3016	3016
q14	234	237	209	209
q15	551	499	485	485
q16	635	624	590	590
q17	565	867	330	330
q18	7206	6451	6387	6387
q19	2329	970	558	558
q20	300	319	186	186
q21	2806	2168	2009	2009
q22	360	332	304	304
Total cold run time: 103781 ms
Total hot run time: 32771 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6282	6260	6227	6227
q2	237	327	233	233
q3	2248	2657	2339	2339
q4	1443	1868	1437	1437
q5	4400	4796	4947	4796
q6	183	177	144	144
q7	2142	1930	1824	1824
q8	2590	2780	2659	2659
q9	7398	7321	7269	7269
q10	3056	3352	2822	2822
q11	580	528	498	498
q12	699	735	584	584
q13	3412	3745	3118	3118
q14	286	297	304	297
q15	568	513	501	501
q16	653	710	650	650
q17	1212	1730	1239	1239
q18	7713	7385	7148	7148
q19	800	1113	1044	1044
q20	1964	1955	1856	1856
q21	5443	5242	4781	4781
q22	617	614	572	572
Total cold run time: 53926 ms
Total hot run time: 52038 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189673 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e6c03d083e7cfbb6819654f4797a379799a644d7, data reload: false

query1	990	377	378	377
query2	6528	2436	2341	2341
query3	6706	211	208	208
query4	33936	23453	23250	23250
query5	4392	624	456	456
query6	284	210	192	192
query7	4626	500	308	308
query8	307	261	254	254
query9	9638	2711	2746	2711
query10	479	318	255	255
query11	18156	15372	15155	15155
query12	166	110	109	109
query13	1674	571	418	418
query14	10058	6725	7018	6725
query15	242	191	191	191
query16	8262	628	449	449
query17	1597	743	564	564
query18	2114	390	304	304
query19	203	186	151	151
query20	125	110	110	110
query21	204	124	103	103
query22	4283	4401	4295	4295
query23	35339	33183	33650	33183
query24	7068	2243	2260	2243
query25	497	463	379	379
query26	1192	265	151	151
query27	2665	447	330	330
query28	5788	2449	2430	2430
query29	757	530	415	415
query30	231	179	150	150
query31	989	901	784	784
query32	90	64	64	64
query33	521	336	293	293
query34	742	832	507	507
query35	808	799	745	745
query36	975	1061	945	945
query37	122	97	77	77
query38	4092	4318	4138	4138
query39	1471	1439	1403	1403
query40	209	124	101	101
query41	48	45	46	45
query42	119	104	101	101
query43	510	522	499	499
query44	1305	803	793	793
query45	178	175	173	173
query46	849	1036	647	647
query47	1904	1926	1842	1842
query48	366	404	328	328
query49	759	478	383	383
query50	653	640	391	391
query51	7201	7152	7026	7026
query52	103	100	94	94
query53	221	247	184	184
query54	471	482	403	403
query55	77	77	78	77
query56	250	254	235	235
query57	1184	1200	1132	1132
query58	243	218	218	218
query59	2926	3136	3068	3068
query60	277	279	256	256
query61	113	147	110	110
query62	878	797	746	746
query63	224	183	193	183
query64	4546	990	638	638
query65	3281	3147	3215	3147
query66	885	418	307	307
query67	15903	15908	15544	15544
query68	9936	743	503	503
query69	455	290	257	257
query70	1231	1181	1133	1133
query71	434	279	253	253
query72	5938	3829	3811	3811
query73	660	737	360	360
query74	9939	8938	9400	8938
query75	4617	3146	2750	2750
query76	5536	1195	747	747
query77	1017	343	295	295
query78	10081	10342	9458	9458
query79	5648	907	622	622
query80	715	512	439	439
query81	487	263	228	228
query82	283	150	120	120
query83	205	168	163	163
query84	292	94	74	74
query85	740	352	318	318
query86	358	330	298	298
query87	4522	4528	4773	4528
query88	3658	2241	2219	2219
query89	441	343	295	295
query90	2131	191	189	189
query91	132	139	110	110
query92	68	59	51	51
query93	3294	884	523	523
query94	719	390	285	285
query95	327	268	265	265
query96	485	606	277	277
query97	2739	2836	2710	2710
query98	227	197	206	197
query99	1614	1590	1423	1423
Total cold run time: 302929 ms
Total hot run time: 189673 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e6c03d083e7cfbb6819654f4797a379799a644d7, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.61	0.10	0.10
query5	0.42	0.40	0.40
query6	1.16	0.66	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.58	0.50	0.51
query10	0.55	0.58	0.56
query11	0.15	0.10	0.10
query12	0.14	0.10	0.11
query13	0.61	0.60	0.59
query14	2.74	2.75	2.77
query15	0.89	0.84	0.83
query16	0.39	0.36	0.39
query17	1.05	1.01	0.98
query18	0.23	0.21	0.21
query19	1.88	1.76	2.01
query20	0.01	0.01	0.01
query21	15.39	0.96	0.60
query22	0.75	0.80	0.64
query23	15.33	1.42	0.55
query24	2.60	1.19	1.06
query25	0.22	0.26	0.06
query26	0.18	0.14	0.14
query27	0.05	0.05	0.04
query28	13.79	1.54	1.05
query29	12.58	4.06	3.33
query30	0.25	0.09	0.07
query31	2.81	0.58	0.40
query32	3.23	0.55	0.47
query33	3.07	3.10	3.12
query34	16.92	5.11	4.48
query35	4.54	4.51	4.44
query36	0.65	0.49	0.49
query37	0.10	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.18	0.13	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.7 s
Total hot run time: 31.31 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants