Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](Nereids) fix create routine load forward strategy #45253

Merged
merged 1 commit into from
Dec 12, 2024
Merged

[fix](Nereids) fix create routine load forward strategy #45253

merged 1 commit into from
Dec 12, 2024

Conversation

LiBinfeng-01
Copy link
Collaborator

@LiBinfeng-01 LiBinfeng-01 commented Dec 10, 2024 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

routine load job should be sync with follower

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@LiBinfeng-01
Copy link
Collaborator Author

run buildall

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 10, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 40005 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d51ce3d92410d2e28befdc2c0768ba1f59750697, data reload: false

------ Round 1 ----------------------------------
q1	17577	7454	7273	7273
q2	2044	169	169	169
q3	10650	1062	1154	1062
q4	10560	743	721	721
q5	7606	2743	2701	2701
q6	236	145	144	144
q7	1000	632	601	601
q8	9397	1889	1933	1889
q9	6709	6483	6487	6483
q10	7020	2276	2336	2276
q11	476	266	263	263
q12	428	220	220	220
q13	17800	3026	3036	3026
q14	249	209	214	209
q15	570	526	528	526
q16	667	610	581	581
q17	989	526	498	498
q18	7304	6738	6938	6738
q19	1342	1034	959	959
q20	479	181	181	181
q21	4060	3175	3199	3175
q22	383	327	310	310
Total cold run time: 107546 ms
Total hot run time: 40005 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7299	7229	7259	7229
q2	330	229	233	229
q3	2931	2875	2956	2875
q4	2049	1818	1820	1818
q5	5729	5700	5657	5657
q6	228	142	139	139
q7	2259	1806	1823	1806
q8	3415	3561	3509	3509
q9	8936	9053	9033	9033
q10	3628	3559	3556	3556
q11	598	530	514	514
q12	814	635	608	608
q13	13478	3263	3203	3203
q14	304	273	287	273
q15	569	534	524	524
q16	720	647	658	647
q17	1861	1658	1639	1639
q18	8322	7789	7669	7669
q19	1727	1624	1424	1424
q20	2104	1852	1885	1852
q21	5654	5346	5551	5346
q22	646	580	569	569
Total cold run time: 73601 ms
Total hot run time: 60119 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197124 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d51ce3d92410d2e28befdc2c0768ba1f59750697, data reload: false

query1	1239	987	950	950
query2	7016	2063	2118	2063
query3	10975	4575	4471	4471
query4	33315	23523	23607	23523
query5	3531	463	449	449
query6	271	208	174	174
query7	3998	293	302	293
query8	305	244	242	242
query9	9426	2706	2703	2703
query10	447	251	252	251
query11	17923	15220	15412	15220
query12	149	106	102	102
query13	1565	409	405	405
query14	9427	6590	7264	6590
query15	261	194	190	190
query16	8076	506	479	479
query17	1670	590	625	590
query18	2180	328	322	322
query19	374	165	173	165
query20	134	120	117	117
query21	210	109	153	109
query22	4933	4496	4538	4496
query23	35207	34484	34570	34484
query24	10535	2500	2533	2500
query25	619	410	401	401
query26	1194	169	157	157
query27	2456	287	278	278
query28	7742	2472	2451	2451
query29	859	419	418	418
query30	241	153	152	152
query31	1058	892	872	872
query32	95	60	56	56
query33	777	288	298	288
query34	1066	506	532	506
query35	909	774	755	755
query36	1138	979	1002	979
query37	155	77	78	77
query38	4578	4402	4375	4375
query39	1511	1489	1483	1483
query40	201	101	103	101
query41	49	42	46	42
query42	110	98	99	98
query43	530	491	493	491
query44	1273	837	856	837
query45	194	176	174	174
query46	1195	704	743	704
query47	2029	1942	1942	1942
query48	425	336	324	324
query49	969	382	391	382
query50	861	385	388	385
query51	7518	7297	7259	7259
query52	102	88	87	87
query53	259	186	196	186
query54	1161	435	417	417
query55	82	77	87	77
query56	278	232	240	232
query57	1319	1166	1099	1099
query58	222	208	221	208
query59	3183	2916	3079	2916
query60	281	246	241	241
query61	109	107	115	107
query62	879	671	659	659
query63	211	189	186	186
query64	4025	690	657	657
query65	3344	3213	3196	3196
query66	859	308	290	290
query67	16017	15808	15637	15637
query68	4838	539	550	539
query69	444	249	245	245
query70	1215	1163	1127	1127
query71	345	253	244	244
query72	6384	4041	4142	4041
query73	763	352	359	352
query74	10559	9060	9021	9021
query75	3444	2701	2651	2651
query76	2744	1089	1043	1043
query77	384	265	273	265
query78	10365	9515	9415	9415
query79	1414	595	579	579
query80	1103	436	426	426
query81	552	229	235	229
query82	959	123	115	115
query83	241	146	140	140
query84	238	70	71	70
query85	1361	363	315	315
query86	382	303	297	297
query87	4750	4595	4637	4595
query88	3232	2212	2178	2178
query89	424	293	296	293
query90	1941	198	182	182
query91	140	100	110	100
query92	60	50	51	50
query93	1571	539	527	527
query94	929	289	287	287
query95	349	316	243	243
query96	610	276	278	276
query97	2858	2718	2696	2696
query98	215	194	193	193
query99	1599	1320	1321	1320
Total cold run time: 301663 ms
Total hot run time: 197124 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.52 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d51ce3d92410d2e28befdc2c0768ba1f59750697, data reload: false

query1	0.03	0.03	0.05
query2	0.06	0.03	0.03
query3	0.23	0.07	0.07
query4	1.63	0.10	0.11
query5	0.43	0.42	0.39
query6	1.17	0.67	0.66
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.50	0.50
query10	0.55	0.56	0.56
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.62	0.59
query14	2.69	2.72	2.76
query15	0.91	0.84	0.84
query16	0.38	0.38	0.39
query17	1.06	1.05	1.09
query18	0.21	0.21	0.21
query19	1.99	1.89	1.97
query20	0.01	0.01	0.01
query21	15.36	0.58	0.58
query22	2.15	2.04	2.43
query23	17.06	0.92	0.89
query24	2.84	1.46	1.66
query25	0.24	0.20	0.10
query26	0.43	0.14	0.13
query27	0.04	0.04	0.04
query28	9.70	1.10	1.10
query29	12.55	3.26	3.28
query30	0.25	0.06	0.06
query31	2.87	0.38	0.37
query32	3.28	0.46	0.48
query33	2.98	3.02	3.03
query34	16.71	4.47	4.46
query35	4.55	4.48	4.47
query36	0.66	0.49	0.51
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.17	0.14	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.07 s
Total hot run time: 33.52 s

@LiBinfeng-01
Copy link
Collaborator Author

run p0

1 similar comment
@LiBinfeng-01
Copy link
Collaborator Author

run p0

@starocean999 starocean999 merged commit e70175f into apache:master Dec 12, 2024
34 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@morrySnow morrySnow Awaiting requested review from morrySnow

@starocean999 starocean999 Awaiting requested review from starocean999

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@LiBinfeng-01 @doris-robot @starocean999 @morrySnow