Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pick](branch-3.0) pick #44647 #45839

Merged
merged 1 commit into from
Dec 25, 2024

Conversation

xinyiZzz
Copy link
Contributor

What problem does this PR solve?

opt Process profile add metadata profile (#44647)

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

### What problem does this PR solve?

Problem Summary:

http://127.0.0.1:8040/profile
```
ObjectHeapDump:
   - AllRowsetsMemBytes: 16.33 MB
   - AllRowsetsNum: 76.45K
   - AllSegmentsMemBytes: 4.16 KB
   - AllSegmentsNum: 1
   - BitmapIndexReaderMemBytes: 0
   - BitmapIndexReaderNum: 0
   - BloomFilterIndexReaderMemBytes: 0
   - BloomFilterIndexReaderNum: 0
   - ColumnReaderMemBytes: 0
   - ColumnReaderNum: 0
   - IndexPageReaderMemBytes: 0
   - IndexPageReaderNum: 0
   - IndexedColumnReaderMemBytes: 0
   - IndexedColumnReaderNum: 0
   - InvertedIndexReaderMemBytes: 0
   - InvertedIndexReaderNum: 0
   - OrdinalIndexReaderMemBytes: 0
   - OrdinalIndexReaderNum: 0
   - RowsetMetaMemBytes: 45.47 MB
   - RowsetMetaNum: 76.45K
   - TabletColumnMemBytes: 253.28 MB
   - TabletColumnNum: 922.18K
   - TabletIndexMemBytes: 16.78 MB
   - TabletIndexNum: 104.74K
   - TabletMetaMemBytes: 17.50 MB
   - TabletMetaNum: 45.89K
   - TabletSchemaMemBytes: 60.15 MB
   - TabletSchemaNum: 49.19K
   - ZoneMapIndexReaderMemBytes: 0
   - ZoneMapIndexReaderNum: 0
 ```
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@xinyiZzz
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40060 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b620d43ae895e949776ec613100721a17c9b15c, data reload: false

------ Round 1 ----------------------------------
q1	17577	7510	7317	7317
q2	2080	169	163	163
q3	10689	1082	1162	1082
q4	10540	738	681	681
q5	7751	2826	2777	2777
q6	236	147	145	145
q7	972	606	602	602
q8	9570	1907	1981	1907
q9	6929	6369	6337	6337
q10	7005	2264	2319	2264
q11	454	272	258	258
q12	412	215	212	212
q13	17807	2938	2937	2937
q14	244	221	215	215
q15	550	510	533	510
q16	658	617	594	594
q17	1002	494	524	494
q18	7247	6439	6671	6439
q19	1985	1054	957	957
q20	470	206	196	196
q21	3999	3192	3043	3043
q22	1040	986	930	930
Total cold run time: 109217 ms
Total hot run time: 40060 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7325	7161	7228	7161
q2	325	232	229	229
q3	2880	2907	2937	2907
q4	1987	1812	1812	1812
q5	5689	5688	5709	5688
q6	225	139	142	139
q7	2208	1779	1768	1768
q8	3327	3529	3513	3513
q9	8673	8758	8802	8758
q10	3526	3455	3497	3455
q11	586	509	499	499
q12	813	604	583	583
q13	16497	3149	3147	3147
q14	297	263	275	263
q15	571	520	524	520
q16	733	663	681	663
q17	1852	1640	1611	1611
q18	8252	7843	7587	7587
q19	7025	1564	1608	1564
q20	2053	1821	1865	1821
q21	8297	5303	5263	5263
q22	1102	1029	1026	1026
Total cold run time: 84243 ms
Total hot run time: 59977 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195661 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b620d43ae895e949776ec613100721a17c9b15c, data reload: false

query1	1293	897	903	897
query2	6246	2066	2015	2015
query3	10881	4142	4212	4142
query4	61842	29996	23513	23513
query5	5247	478	483	478
query6	430	180	184	180
query7	5715	319	317	317
query8	321	239	232	232
query9	9542	2719	2719	2719
query10	521	271	253	253
query11	18190	15127	17441	15127
query12	156	109	97	97
query13	1614	449	426	426
query14	10498	6974	7026	6974
query15	206	182	171	171
query16	7127	514	472	472
query17	1155	587	571	571
query18	1813	312	308	308
query19	266	156	160	156
query20	115	111	111	111
query21	65	44	50	44
query22	4583	4498	4484	4484
query23	34677	33916	34049	33916
query24	6175	2899	2916	2899
query25	542	416	415	415
query26	668	175	174	174
query27	1739	303	303	303
query28	4162	2568	2542	2542
query29	667	463	412	412
query30	251	161	164	161
query31	965	787	823	787
query32	63	56	54	54
query33	427	276	270	270
query34	928	502	499	499
query35	860	755	746	746
query36	1131	963	916	916
query37	118	71	71	71
query38	4056	4112	3890	3890
query39	1521	1490	1464	1464
query40	139	81	78	78
query41	47	47	47	47
query42	109	98	95	95
query43	525	476	496	476
query44	1171	817	816	816
query45	187	166	173	166
query46	1129	736	728	728
query47	1990	1877	1874	1874
query48	470	375	368	368
query49	707	373	363	363
query50	830	418	418	418
query51	7295	7266	7176	7176
query52	106	89	99	89
query53	255	188	179	179
query54	552	453	440	440
query55	81	82	74	74
query56	257	242	243	242
query57	1222	1136	1181	1136
query58	232	210	212	210
query59	3197	3007	2881	2881
query60	277	247	252	247
query61	108	109	106	106
query62	764	669	641	641
query63	201	178	176	176
query64	1445	655	625	625
query65	3248	3149	3246	3149
query66	691	296	291	291
query67	15546	15320	15248	15248
query68	4487	556	566	556
query69	456	254	254	254
query70	1191	1127	1081	1081
query71	409	261	263	261
query72	6646	3947	3958	3947
query73	759	341	339	339
query74	10218	8920	8891	8891
query75	3382	2611	2634	2611
query76	2095	1089	1055	1055
query77	582	263	271	263
query78	10801	9631	9474	9474
query79	8304	581	574	574
query80	2217	430	414	414
query81	567	236	234	234
query82	1420	117	116	116
query83	287	147	142	142
query84	290	77	75	75
query85	1217	299	289	289
query86	455	308	298	298
query87	4456	4256	4241	4241
query88	5695	2361	2409	2361
query89	556	290	289	289
query90	2078	177	179	177
query91	174	145	146	145
query92	60	49	46	46
query93	6810	537	533	533
query94	794	287	298	287
query95	349	243	246	243
query96	632	274	272	272
query97	3316	3166	3182	3166
query98	216	208	192	192
query99	1607	1300	1290	1290
Total cold run time: 332083 ms
Total hot run time: 195661 ms

@yiguolei yiguolei merged commit 207327f into apache:branch-3.0 Dec 25, 2024
17 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants