Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix](Variant) create table should not automatically add variant to keys #36609

Merged
merged 2 commits into from
Jun 21, 2024
Merged

[Fix](Variant) create table should not automatically add variant to keys #36609

merged 2 commits into from
Jun 21, 2024

Conversation

eldenmoon
Copy link
Member

Truncate when meet Variant type

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@eldenmoon eldenmoon changed the title [Fix](Variant) create table should not add variant to type keys [Fix](Variant) create table should not add variant to keys Jun 20, 2024
@eldenmoon
Copy link
Member Author

run buildall

morrySnow
morrySnow previously approved these changes Jun 20, 2024
View reviewed changes
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 39977 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0677e5a5124c9f1673d5bbcc012f0dc15f4840ca, data reload: false

------ Round 1 ----------------------------------
q1	17711	4405	4358	4358
q2	2689	214	195	195
q3	11055	1125	1119	1119
q4	11224	782	808	782
q5	7831	2704	2710	2704
q6	226	143	143	143
q7	969	620	612	612
q8	9581	2077	2096	2077
q9	8880	6495	6442	6442
q10	9030	3711	3739	3711
q11	455	244	245	244
q12	430	236	234	234
q13	18673	2974	3001	2974
q14	271	226	226	226
q15	530	473	487	473
q16	534	377	376	376
q17	963	687	671	671
q18	8055	7364	7397	7364
q19	5251	1493	1551	1493
q20	652	320	338	320
q21	4900	3108	3845	3108
q22	405	367	351	351
Total cold run time: 120315 ms
Total hot run time: 39977 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4388	4245	4215	4215
q2	417	284	267	267
q3	2936	2770	2679	2679
q4	1843	1659	1641	1641
q5	5249	5254	5256	5254
q6	214	126	129	126
q7	2102	1754	1759	1754
q8	3158	3337	3294	3294
q9	8317	8346	8380	8346
q10	3883	3700	3626	3626
q11	593	493	472	472
q12	789	593	605	593
q13	16363	2981	2952	2952
q14	304	263	259	259
q15	518	480	469	469
q16	459	407	421	407
q17	1737	1489	1466	1466
q18	7579	7518	7391	7391
q19	1670	1504	1617	1504
q20	2012	1777	1792	1777
q21	4804	4641	4755	4641
q22	628	541	599	541
Total cold run time: 69963 ms
Total hot run time: 53674 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173188 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0677e5a5124c9f1673d5bbcc012f0dc15f4840ca, data reload: false

query1	944	374	382	374
query2	6454	2286	2412	2286
query3	6649	204	209	204
query4	19392	17278	17307	17278
query5	4163	486	458	458
query6	261	162	167	162
query7	4582	291	284	284
query8	342	282	289	282
query9	8437	2449	2428	2428
query10	637	303	278	278
query11	10571	10002	10001	10001
query12	138	89	85	85
query13	1640	389	362	362
query14	9945	7567	7566	7566
query15	233	192	183	183
query16	7863	276	253	253
query17	1914	527	555	527
query18	1971	267	271	267
query19	193	155	150	150
query20	93	85	80	80
query21	208	127	130	127
query22	4479	4143	4050	4050
query23	33719	33110	33202	33110
query24	12017	2907	2867	2867
query25	664	363	356	356
query26	1749	154	154	154
query27	2975	321	312	312
query28	7396	2055	2048	2048
query29	1045	612	592	592
query30	268	153	154	153
query31	925	743	754	743
query32	92	55	54	54
query33	759	288	313	288
query34	924	472	451	451
query35	738	618	618	618
query36	1070	942	940	940
query37	173	69	77	69
query38	2875	2763	2710	2710
query39	851	811	807	807
query40	288	129	122	122
query41	55	54	54	54
query42	121	96	103	96
query43	586	569	541	541
query44	1234	726	740	726
query45	205	168	168	168
query46	1077	686	708	686
query47	1866	1790	1815	1790
query48	369	299	290	290
query49	1168	410	400	400
query50	761	387	370	370
query51	6767	6806	6746	6746
query52	114	90	98	90
query53	351	289	287	287
query54	1013	448	442	442
query55	76	107	73	73
query56	281	257	257	257
query57	1128	1065	1062	1062
query58	256	235	243	235
query59	3405	3221	3149	3149
query60	291	267	267	267
query61	92	93	95	93
query62	651	454	442	442
query63	317	286	282	282
query64	9904	2223	1746	1746
query65	3173	3126	3084	3084
query66	1377	329	348	329
query67	15680	15225	15014	15014
query68	4660	520	522	520
query69	453	301	316	301
query70	1178	1154	1133	1133
query71	385	277	271	271
query72	6979	5338	5193	5193
query73	754	319	313	313
query74	6023	5513	5489	5489
query75	3385	2664	2623	2623
query76	2420	880	960	880
query77	431	288	303	288
query78	10213	9826	9773	9773
query79	2237	503	514	503
query80	1104	450	467	450
query81	573	217	220	217
query82	690	102	100	100
query83	233	191	169	169
query84	234	86	85	85
query85	1985	283	278	278
query86	492	316	316	316
query87	3297	3089	3085	3085
query88	4050	2370	2384	2370
query89	471	382	386	382
query90	1820	201	196	196
query91	137	116	180	116
query92	61	48	47	47
query93	2252	513	488	488
query94	1259	197	188	188
query95	399	311	315	311
query96	588	266	266	266
query97	3204	3105	3023	3023
query98	212	193	189	189
query99	1148	827	842	827
Total cold run time: 275282 ms
Total hot run time: 173188 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.16 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0677e5a5124c9f1673d5bbcc012f0dc15f4840ca, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.67	0.07	0.08
query5	0.49	0.50	0.49
query6	1.12	0.73	0.72
query7	0.01	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.49	0.48
query10	0.54	0.54	0.54
query11	0.15	0.11	0.11
query12	0.15	0.13	0.13
query13	0.59	0.58	0.60
query14	0.76	0.81	0.76
query15	0.83	0.81	0.82
query16	0.36	0.37	0.38
query17	0.96	0.97	1.06
query18	0.22	0.24	0.25
query19	1.87	1.72	1.77
query20	0.02	0.01	0.01
query21	15.41	0.65	0.65
query22	4.44	7.48	1.65
query23	18.28	1.37	1.28
query24	2.13	0.23	0.21
query25	0.17	0.09	0.08
query26	0.25	0.17	0.18
query27	0.08	0.08	0.08
query28	13.23	1.01	1.00
query29	12.60	3.27	3.25
query30	0.26	0.06	0.06
query31	2.87	0.39	0.38
query32	3.28	0.47	0.46
query33	2.87	2.87	2.84
query34	17.19	4.41	4.41
query35	4.50	4.51	4.43
query36	0.65	0.47	0.46
query37	0.19	0.15	0.16
query38	0.16	0.14	0.15
query39	0.04	0.04	0.03
query40	0.16	0.14	0.14
query41	0.09	0.04	0.06
query42	0.06	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.62 s
Total hot run time: 30.16 s

xiaokang
xiaokang reviewed Jun 20, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls add testcase

@eldenmoon
Copy link
Member Author

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jun 20, 2024
@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40726 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8e8c15e788ae6643d134cb962178a1b799963837, data reload: false

------ Round 1 ----------------------------------
q1	17630	4498	4306	4306
q2	2039	200	204	200
q3	10434	1200	1052	1052
q4	10185	728	831	728
q5	7524	2832	2618	2618
q6	225	137	139	137
q7	978	606	603	603
q8	9219	2083	2071	2071
q9	9018	6531	6512	6512
q10	8892	3736	3749	3736
q11	450	238	246	238
q12	433	239	231	231
q13	17770	2995	3011	2995
q14	284	222	212	212
q15	532	476	468	468
q16	538	376	379	376
q17	982	706	685	685
q18	8075	7533	7464	7464
q19	3363	1537	1472	1472
q20	665	317	320	317
q21	4962	3998	3960	3960
q22	397	345	359	345
Total cold run time: 114595 ms
Total hot run time: 40726 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4352	4205	4302	4205
q2	381	277	266	266
q3	3023	2858	2915	2858
q4	2007	1746	1745	1745
q5	5674	5496	5564	5496
q6	221	130	136	130
q7	2236	1916	1860	1860
q8	3301	3461	3435	3435
q9	8751	8727	8873	8727
q10	4096	3842	3748	3748
q11	596	505	503	503
q12	806	632	643	632
q13	15982	3184	3195	3184
q14	301	282	284	282
q15	526	486	485	485
q16	514	437	442	437
q17	1815	1484	1510	1484
q18	8071	7962	7833	7833
q19	2778	1696	1624	1624
q20	2122	1948	1860	1860
q21	5243	4984	4730	4730
q22	637	554	576	554
Total cold run time: 73433 ms
Total hot run time: 56078 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.48% (9009/24695)
Line Coverage: 28.02% (73877/263628)
Region Coverage: 27.51% (38367/139486)
Branch Coverage: 24.21% (19559/80790)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8e8c15e788ae6643d134cb962178a1b799963837_8e8c15e788ae6643d134cb962178a1b799963837/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 173917 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8e8c15e788ae6643d134cb962178a1b799963837, data reload: false

query1	929	383	384	383
query2	6438	2499	2327	2327
query3	6639	205	206	205
query4	19919	17464	17320	17320
query5	3566	482	449	449
query6	249	161	164	161
query7	4576	289	288	288
query8	317	295	295	295
query9	8540	2456	2438	2438
query10	577	321	290	290
query11	10748	10294	10245	10245
query12	122	86	88	86
query13	1654	385	355	355
query14	10206	6712	7561	6712
query15	241	190	193	190
query16	7742	260	262	260
query17	1918	535	521	521
query18	1962	309	267	267
query19	200	160	152	152
query20	91	82	92	82
query21	207	135	125	125
query22	4574	4273	4243	4243
query23	33948	33659	33438	33438
query24	11936	2970	2890	2890
query25	701	402	393	393
query26	1717	170	161	161
query27	3016	341	330	330
query28	7788	2176	2179	2176
query29	1106	651	612	612
query30	240	151	156	151
query31	955	786	743	743
query32	96	52	55	52
query33	747	273	288	273
query34	1028	472	475	472
query35	763	641	618	618
query36	1139	953	955	953
query37	268	75	85	75
query38	2971	2837	2808	2808
query39	894	866	852	852
query40	257	126	119	119
query41	54	52	51	51
query42	115	96	94	94
query43	605	554	555	554
query44	1214	733	738	733
query45	195	171	161	161
query46	1080	711	704	704
query47	1883	1793	1828	1793
query48	375	302	298	298
query49	964	397	411	397
query50	762	389	387	387
query51	6857	6847	6800	6800
query52	102	90	93	90
query53	354	284	286	284
query54	923	440	440	440
query55	74	75	75	75
query56	300	254	264	254
query57	1165	1051	1067	1051
query58	250	266	237	237
query59	3275	3179	3179	3179
query60	294	273	275	273
query61	92	89	88	88
query62	624	436	449	436
query63	318	290	295	290
query64	9669	2313	1682	1682
query65	3221	3155	3099	3099
query66	1239	328	342	328
query67	15403	15190	14971	14971
query68	5319	530	526	526
query69	617	482	408	408
query70	1208	1104	1140	1104
query71	467	266	269	266
query72	7760	5249	5771	5249
query73	776	322	324	322
query74	5966	5557	5625	5557
query75	3766	2649	2667	2649
query76	3494	935	915	915
query77	604	294	308	294
query78	10379	9741	9793	9741
query79	2447	507	516	507
query80	1021	484	518	484
query81	586	226	220	220
query82	1430	99	115	99
query83	264	168	168	168
query84	243	87	85	85
query85	1462	286	269	269
query86	466	314	317	314
query87	3253	3067	3107	3067
query88	4165	2346	2348	2346
query89	470	387	385	385
query90	1747	190	191	190
query91	129	98	100	98
query92	57	50	50	50
query93	2314	500	498	498
query94	1114	188	187	187
query95	401	319	308	308
query96	598	270	262	262
query97	3210	3058	3052	3052
query98	209	204	220	204
query99	1287	861	834	834
Total cold run time: 279621 ms
Total hot run time: 173917 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.36 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8e8c15e788ae6643d134cb962178a1b799963837, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.09	0.09
query5	0.50	0.50	0.50
query6	1.12	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.05
query9	0.54	0.49	0.48
query10	0.55	0.55	0.53
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.60
query14	0.76	0.78	0.78
query15	0.83	0.80	0.80
query16	0.35	0.36	0.36
query17	1.01	1.07	0.96
query18	0.21	0.23	0.26
query19	1.83	1.73	1.74
query20	0.01	0.00	0.01
query21	15.41	0.66	0.66
query22	4.37	6.86	1.71
query23	18.24	1.49	1.27
query24	2.20	0.23	0.23
query25	0.15	0.09	0.08
query26	0.26	0.18	0.18
query27	0.08	0.08	0.08
query28	13.24	1.01	1.00
query29	12.64	3.30	3.26
query30	0.26	0.07	0.06
query31	2.85	0.40	0.38
query32	3.28	0.47	0.47
query33	2.91	2.90	2.88
query34	17.10	4.40	4.45
query35	4.51	4.46	4.50
query36	0.64	0.47	0.46
query37	0.18	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.04
query42	0.06	0.04	0.06
query43	0.05	0.04	0.04
Total cold run time: 109.6 s
Total hot run time: 30.36 s

xiaokang
xiaokang approved these changes Jun 21, 2024
View reviewed changes
Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang changed the title [Fix](Variant) create table should not add variant to keys [Fix](Variant) create table should not automatically add variant to keys Jun 21, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 21, 2024
amorynan
amorynan approved these changes Jun 21, 2024
View reviewed changes
Copy link
Contributor

@amorynan amorynan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eldenmoon eldenmoon merged commit 1b8214d into apache:master Jun 21, 2024
27 of 31 checks passed
iszhangpch pushed a commit to iszhangpch/doris-p that referenced this pull request Jun 21, 2024
@eldenmoon
[Fix](Variant) create table should not automatically add variant to k…
c0f3f41 
…eys (apache#36609)

Truncate when meet Variant type
dataroaring pushed a commit that referenced this pull request Jun 21, 2024
@eldenmoon @dataroaring
[Fix](Variant) create table should not automatically add variant to k…
a1becf2 
…eys (#36609)

Truncate when meet Variant type
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request Nov 28, 2024
@eldenmoon
[Fix](Variant) create table should not automatically add variant to k…
484a9aa 
…eys (apache#36609)

Truncate when meet Variant type
@eldenmoon eldenmoon mentioned this pull request Nov 28, 2024
Merged
eldenmoon added a commit that referenced this pull request Nov 29, 2024
@eldenmoon
[Fix](Variant) create table should not automatically add variant to k…
82c7a9d 
…ey (#44736)

#36609
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amorynan amorynan amorynan approved these changes

@xiaokang xiaokang xiaokang approved these changes

@morrySnow morrySnow morrySnow left review comments

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@eldenmoon @doris-robot @xiaokang @amorynan @morrySnow @dataroaring