Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](cluster key) support fuzzy random_add_cluster_keys_for_mow config #44485

Merged
merged 1 commit into from
Nov 22, 2024

Conversation

mymeiyi
Copy link
Contributor

@mymeiyi mymeiyi commented Nov 22, 2024

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

support fuzzy set fe config for random_add_cluster_keys_for_mow by add:

use_fuzzy_conf=true
fuzzy_test_type=daily or rqg

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@mymeiyi
Copy link
Contributor Author

mymeiyi commented Nov 22, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39867 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d35ac5c0ea3265ac28e2ca6679e35ea3644993c0, data reload: false

------ Round 1 ----------------------------------
q1	17591	7448	7285	7285
q2	2065	169	157	157
q3	10947	1111	1211	1111
q4	10544	707	721	707
q5	7608	2776	2721	2721
q6	242	146	145	145
q7	994	627	591	591
q8	9230	1912	1872	1872
q9	6554	6372	6414	6372
q10	7016	2282	2319	2282
q11	463	263	251	251
q12	421	226	213	213
q13	17801	3038	3033	3033
q14	247	209	216	209
q15	584	526	536	526
q16	657	611	577	577
q17	981	549	567	549
q18	7219	6685	6893	6685
q19	1347	1039	951	951
q20	475	178	179	178
q21	3989	3263	3136	3136
q22	386	319	316	316
Total cold run time: 107361 ms
Total hot run time: 39867 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7284	7229	7231	7229
q2	335	226	231	226
q3	2926	2831	2993	2831
q4	2077	1753	1821	1753
q5	5797	5689	5694	5689
q6	232	142	144	142
q7	2227	1825	1777	1777
q8	3433	3541	3511	3511
q9	8863	8895	8844	8844
q10	3596	3553	3573	3553
q11	620	502	508	502
q12	831	603	629	603
q13	11881	3240	3200	3200
q14	314	269	274	269
q15	590	527	532	527
q16	673	654	628	628
q17	1860	1672	1592	1592
q18	8322	7804	7643	7643
q19	1736	1684	1447	1447
q20	2158	1868	1915	1868
q21	5602	5351	5460	5351
q22	643	549	582	549
Total cold run time: 72000 ms
Total hot run time: 59734 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 197041 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d35ac5c0ea3265ac28e2ca6679e35ea3644993c0, data reload: false

query1	1233	969	994	969
query2	6273	2078	2164	2078
query3	10837	3832	3824	3824
query4	67590	29425	23574	23574
query5	4929	444	451	444
query6	414	195	160	160
query7	5595	306	279	279
query8	299	216	225	216
query9	9112	2648	2635	2635
query10	485	262	247	247
query11	17412	15358	15822	15358
query12	159	106	103	103
query13	1572	444	431	431
query14	10948	7783	7113	7113
query15	221	193	181	181
query16	7167	459	461	459
query17	1437	601	599	599
query18	1788	344	326	326
query19	216	202	151	151
query20	120	114	114	114
query21	207	119	104	104
query22	4816	4837	4624	4624
query23	34753	34659	34283	34283
query24	5469	2457	2459	2457
query25	489	399	398	398
query26	654	178	146	146
query27	1970	282	287	282
query28	3881	2481	2467	2467
query29	682	421	434	421
query30	213	146	147	146
query31	994	813	853	813
query32	68	56	60	56
query33	463	294	295	294
query34	917	558	516	516
query35	834	719	733	719
query36	1106	960	995	960
query37	124	72	76	72
query38	4423	4454	4362	4362
query39	1528	1480	1461	1461
query40	206	100	98	98
query41	47	42	44	42
query42	108	101	101	101
query43	559	512	515	512
query44	1197	862	842	842
query45	179	167	164	164
query46	1164	697	709	697
query47	2059	1934	1965	1934
query48	434	329	307	307
query49	738	399	394	394
query50	862	397	394	394
query51	7379	7209	7189	7189
query52	97	86	89	86
query53	252	179	180	179
query54	504	405	400	400
query55	78	80	73	73
query56	247	249	237	237
query57	1291	1169	1149	1149
query58	220	220	232	220
query59	3265	3129	2967	2967
query60	262	258	282	258
query61	113	103	110	103
query62	778	656	663	656
query63	217	191	186	186
query64	1348	647	615	615
query65	3299	3217	3219	3217
query66	695	298	304	298
query67	16031	15786	15773	15773
query68	4005	586	560	560
query69	436	255	247	247
query70	1198	1151	1098	1098
query71	355	245	247	245
query72	6408	4098	3957	3957
query73	763	366	369	366
query74	10177	9039	9136	9039
query75	3397	2720	2694	2694
query76	1764	1093	1085	1085
query77	465	270	278	270
query78	10447	9408	9490	9408
query79	1671	604	603	603
query80	917	423	432	423
query81	518	240	316	240
query82	1287	114	120	114
query83	177	145	144	144
query84	280	69	67	67
query85	881	302	295	295
query86	339	301	292	292
query87	4800	4703	4535	4535
query88	3736	2243	2215	2215
query89	424	288	304	288
query90	2021	195	184	184
query91	136	103	102	102
query92	68	49	51	49
query93	1929	545	552	545
query94	816	291	289	289
query95	343	248	241	241
query96	663	277	276	276
query97	2917	2742	2727	2727
query98	215	193	194	193
query99	1671	1317	1338	1317
Total cold run time: 320251 ms
Total hot run time: 197041 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.96 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d35ac5c0ea3265ac28e2ca6679e35ea3644993c0, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.04
query3	0.24	0.08	0.07
query4	1.61	0.10	0.10
query5	0.42	0.42	0.41
query6	1.16	0.66	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.51
query10	0.55	0.54	0.56
query11	0.14	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.61	0.61
query14	2.83	2.88	2.73
query15	0.90	0.83	0.83
query16	0.39	0.39	0.38
query17	1.02	1.04	1.07
query18	0.22	0.21	0.21
query19	1.90	1.87	2.03
query20	0.01	0.01	0.02
query21	15.36	0.60	0.58
query22	2.69	2.57	1.83
query23	17.14	0.91	0.89
query24	3.38	1.12	1.20
query25	0.26	0.09	0.15
query26	0.48	0.14	0.14
query27	0.05	0.04	0.04
query28	10.26	1.10	1.07
query29	12.53	3.23	3.24
query30	0.25	0.06	0.06
query31	2.87	0.38	0.40
query32	3.26	0.47	0.48
query33	2.99	3.03	3.10
query34	16.73	4.50	4.44
query35	4.48	4.50	4.49
query36	0.67	0.50	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.12	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.81 s
Total hot run time: 32.96 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Nov 22, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 2753b70 into apache:master Nov 22, 2024
32 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants