Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

branch-3.0: [fix](move-memtable) tolerate non-open streams in close wait #44680 #45153

Merged
merged 1 commit into from
Dec 11, 2024

Conversation

github-actions[bot]
Copy link
Contributor

@github-actions github-actions bot commented Dec 7, 2024

Cherry-picked from #44680

Related PR: #44344
`VTabletWriterV2::_select_streams()` is already checking if there is
enough downstream BE to meet the replication requirements.
`VTabletWriterV2::close()` should tolerate those non-open streams on
close wait.

Debug point `VTabletWriterV2._open_streams.skip_two_backends` is added
along with `VTabletWriterV2._open_streams.skip_one_backend` to check
this behavior.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Dec 7, 2024
@dataroaring dataroaring reopened this Dec 7, 2024
@doris-robot
Copy link

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40282 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c0ac10f79605e7c26c91c57ff8a1564a6e4d1bf0, data reload: false

------ Round 1 ----------------------------------
q1	17612	7405	7278	7278
q2	2072	167	164	164
q3	10670	1069	1174	1069
q4	10542	728	674	674
q5	7745	2850	2751	2751
q6	229	145	149	145
q7	956	603	614	603
q8	9349	1900	1992	1900
q9	6494	6403	6395	6395
q10	6965	2295	2306	2295
q11	450	260	253	253
q12	394	209	217	209
q13	17779	2999	2999	2999
q14	243	208	203	203
q15	557	501	523	501
q16	668	614	605	605
q17	953	583	543	543
q18	7297	6516	6543	6516
q19	3137	1059	1033	1033
q20	467	199	189	189
q21	3931	3150	2987	2987
q22	1057	970	976	970
Total cold run time: 109567 ms
Total hot run time: 40282 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7361	7199	7201	7199
q2	324	230	226	226
q3	2859	2859	2878	2859
q4	1986	1778	1766	1766
q5	5642	5683	5679	5679
q6	219	136	140	136
q7	2229	1766	1778	1766
q8	3318	3532	3458	3458
q9	8839	8851	8876	8851
q10	3559	3483	3518	3483
q11	598	495	492	492
q12	775	612	573	573
q13	16477	3101	3111	3101
q14	318	271	272	271
q15	581	522	525	522
q16	704	648	682	648
q17	1870	1595	1613	1595
q18	8160	7861	7585	7585
q19	3769	1576	1596	1576
q20	2083	1855	1842	1842
q21	5361	5319	5252	5252
q22	1111	1008	980	980
Total cold run time: 78143 ms
Total hot run time: 59860 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194180 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c0ac10f79605e7c26c91c57ff8a1564a6e4d1bf0, data reload: false

query1	1229	907	912	907
query2	6238	2040	2052	2040
query3	10836	4126	4041	4041
query4	66312	29017	23497	23497
query5	5304	441	429	429
query6	442	179	189	179
query7	5680	314	313	313
query8	313	236	227	227
query9	9141	2678	2659	2659
query10	504	259	250	250
query11	17770	15098	15945	15098
query12	157	99	99	99
query13	1524	451	410	410
query14	10915	6530	6865	6530
query15	215	184	175	175
query16	7294	529	475	475
query17	1043	578	574	574
query18	1906	339	314	314
query19	229	158	160	158
query20	123	115	118	115
query21	208	108	102	102
query22	4652	4388	4309	4309
query23	35097	33759	34050	33759
query24	6158	2877	2891	2877
query25	580	420	417	417
query26	677	172	172	172
query27	1834	324	312	312
query28	4266	2573	2487	2487
query29	692	464	429	429
query30	239	166	163	163
query31	1005	789	821	789
query32	67	52	60	52
query33	412	270	279	270
query34	902	500	520	500
query35	853	719	728	719
query36	1075	958	949	949
query37	113	71	66	66
query38	4055	4041	3971	3971
query39	1508	1468	1483	1468
query40	209	99	98	98
query41	46	45	45	45
query42	107	95	92	92
query43	514	482	495	482
query44	1164	812	819	812
query45	189	174	165	165
query46	1147	737	727	727
query47	1995	1900	1915	1900
query48	491	358	379	358
query49	728	381	377	377
query50	860	411	410	410
query51	7338	7205	6894	6894
query52	96	81	93	81
query53	262	176	183	176
query54	553	441	449	441
query55	75	73	73	73
query56	254	237	225	225
query57	1204	1107	1121	1107
query58	200	206	203	203
query59	3117	2790	2856	2790
query60	282	241	254	241
query61	110	107	105	105
query62	782	655	666	655
query63	211	189	187	187
query64	1627	661	650	650
query65	3269	3168	3254	3168
query66	648	298	295	295
query67	15838	15277	15458	15277
query68	4592	562	550	550
query69	411	254	249	249
query70	1154	1132	1124	1124
query71	357	258	246	246
query72	6382	4093	3868	3868
query73	765	353	344	344
query74	10027	8805	8959	8805
query75	3322	2683	2605	2605
query76	1988	1044	1013	1013
query77	515	262	256	256
query78	10723	9599	9447	9447
query79	8621	582	578	578
query80	2320	415	422	415
query81	557	238	235	235
query82	1390	121	113	113
query83	259	140	142	140
query84	291	89	77	77
query85	1848	302	298	298
query86	485	292	297	292
query87	4391	4225	4297	4225
query88	5720	2391	2385	2385
query89	564	286	287	286
query90	2137	182	189	182
query91	178	147	143	143
query92	70	47	48	47
query93	6678	531	522	522
query94	1004	290	285	285
query95	349	244	251	244
query96	644	278	279	278
query97	3340	3116	3205	3116
query98	227	197	200	197
query99	1609	1280	1287	1280
Total cold run time: 337802 ms
Total hot run time: 194180 ms

@dataroaring dataroaring merged commit d1eb413 into branch-3.0 Dec 11, 2024
18 of 21 checks passed
@github-actions github-actions bot deleted the auto-pick-44680-branch-3.0 branch December 11, 2024 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants