Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix] (inverted index) fix the error result in the compound query #41297

Draft
wants to merge 2 commits into
base: branch-2.0
Choose a base branch
from
Draft

[fix] (inverted index) fix the error result in the compound query #41297

wants to merge 2 commits into from

Conversation

csun5285
Copy link
Contributor

@csun5285 csun5285 commented Sep 25, 2024 •
edited
Loading

Proposed changes

select count(*) from table where a = -2  AND ((CASE WHEN a = -2 THEN 1 ELSE NULL END = 1) OR b != 7

image

Under the expr tree of a compound node, if the parent node is not compound, the index column must be read.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

github-actions[bot]
github-actions bot reviewed Sep 25, 2024
View reviewed changes
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

be/src/olap/task/engine_storage_migration_task.h Show resolved Hide resolved
@csun5285 csun5285 changed the title [fix] (inverted index) fix the error in query [fix] (inverted index) fix the error in the compound query Sep 25, 2024
@csun5285 csun5285 changed the title [fix] (inverted index) fix the error in the compound query [fix] (inverted index) fix the error result in the compound query Sep 25, 2024
@csun5285
Copy link
Contributor Author

run buildall

@csun5285
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8146/21545)
Line Coverage: 29.55% (67057/226945)
Region Coverage: 29.02% (34567/119116)
Branch Coverage: 24.93% (17811/71434)
Coverage Report: http://coverage.selectdb-in.cc/coverage/d6be9a3bf992b0ba4da45be9627881c46eb8b1bf_d6be9a3bf992b0ba4da45be9627881c46eb8b1bf/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 48932 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d6be9a3bf992b0ba4da45be9627881c46eb8b1bf, data reload: false

------ Round 1 ----------------------------------
q1	17587	4380	4382	4380
q2	2050	154	148	148
q3	10416	1935	1968	1935
q4	10342	1232	1305	1232
q5	8499	3867	3911	3867
q6	229	156	123	123
q7	1994	1603	1602	1602
q8	9296	2736	2700	2700
q9	10306	10004	9705	9705
q10	8647	3501	3462	3462
q11	428	245	261	245
q12	464	304	308	304
q13	18318	4000	4047	4000
q14	351	338	327	327
q15	515	457	471	457
q16	536	454	456	454
q17	1130	962	911	911
q18	7325	6739	6806	6739
q19	1671	1560	1544	1544
q20	506	316	314	314
q21	4405	4205	4086	4086
q22	499	397	407	397
Total cold run time: 115514 ms
Total hot run time: 48932 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4341	4329	4290	4290
q2	315	222	216	216
q3	4158	4161	4117	4117
q4	2743	2737	2756	2737
q5	7200	7101	7168	7101
q6	241	127	120	120
q7	3279	2925	2920	2920
q8	4410	4481	4499	4481
q9	13661	13578	13622	13578
q10	4258	4317	4337	4317
q11	777	714	683	683
q12	1036	860	851	851
q13	8232	3818	3817	3817
q14	451	421	421	421
q15	498	453	468	453
q16	630	615	608	608
q17	3860	3864	3881	3864
q18	8854	8791	8788	8788
q19	1739	1690	1623	1623
q20	2409	2156	2121	2121
q21	8690	8864	8750	8750
q22	1089	1041	1001	1001
Total cold run time: 82871 ms
Total hot run time: 76857 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 212113 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d6be9a3bf992b0ba4da45be9627881c46eb8b1bf, data reload: false

query1	2518	448	389	389
query2	8349	2234	2182	2182
query3	6926	200	200	200
query4	23190	21434	21522	21434
query5	19743	6532	6535	6532
query6	413	219	225	219
query7	5211	303	304	303
query8	248	224	249	224
query9	3103	2645	2598	2598
query10	427	316	295	295
query11	16137	15224	14983	14983
query12	131	81	72	72
query13	1007	447	436	436
query14	19404	13116	13482	13116
query15	358	224	232	224
query16	6425	273	257	257
query17	1543	944	910	910
query18	895	320	330	320
query19	214	154	161	154
query20	97	96	102	96
query21	190	95	100	95
query22	5272	4952	5107	4952
query23	34257	33547	33366	33366
query24	6515	6216	6313	6216
query25	529	453	433	433
query26	795	165	160	160
query27	2263	301	299	299
query28	6112	2275	2221	2221
query29	2868	2828	2748	2748
query30	245	168	165	165
query31	928	771	742	742
query32	80	62	58	58
query33	446	256	265	256
query34	862	471	467	467
query35	1146	907	951	907
query36	1376	1384	1146	1146
query37	91	60	61	60
query38	3075	2917	2857	2857
query39	1383	1330	1351	1330
query40	211	95	100	95
query41	41	36	37	36
query42	89	84	84	84
query43	618	598	583	583
query44	1154	709	710	709
query45	252	236	229	229
query46	1227	978	936	936
query47	1749	1990	1881	1881
query48	514	410	402	402
query49	617	379	377	377
query50	860	589	565	565
query51	4784	4648	4691	4648
query52	89	96	80	80
query53	241	183	200	183
query54	2649	2498	2495	2495
query55	94	85	87	85
query56	219	216	214	214
query57	1296	1158	1130	1130
query58	217	216	212	212
query59	3474	3185	3384	3185
query60	221	212	207	207
query61	97	100	97	97
query62	801	448	437	437
query63	207	173	181	173
query64	3368	1524	1426	1426
query65	3622	3545	3579	3545
query66	767	392	413	392
query67	16411	15264	17254	15264
query68	6987	646	630	630
query69	491	289	275	275
query70	1595	1303	1446	1303
query71	412	304	307	304
query72	6663	4835	4767	4767
query73	771	321	319	319
query74	6250	5796	5713	5713
query75	4739	3698	3685	3685
query76	4025	1149	1159	1149
query77	544	252	249	249
query78	12537	12046	12096	12046
query79	8050	645	644	644
query80	892	396	389	389
query81	490	243	239	239
query82	1242	103	96	96
query83	165	141	137	137
query84	250	70	67	67
query85	852	324	317	317
query86	331	314	300	300
query87	3222	2977	3016	2977
query88	4596	2300	2293	2293
query89	431	299	282	282
query90	1907	210	209	209
query91	156	127	126	126
query92	60	48	50	48
query93	6008	590	550	550
query94	756	207	210	207
query95	2095	1912	1964	1912
query96	648	327	326	326
query97	6470	6456	6329	6329
query98	237	208	208	208
query99	2930	941	925	925
Total cold run time: 317463 ms
Total hot run time: 212113 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.47 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d6be9a3bf992b0ba4da45be9627881c46eb8b1bf, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.03	0.02
query3	0.26	0.05	0.04
query4	1.80	0.05	0.06
query5	0.54	0.53	0.52
query6	1.30	0.62	0.62
query7	0.02	0.01	0.01
query8	0.03	0.03	0.02
query9	0.51	0.48	0.48
query10	0.54	0.52	0.52
query11	0.11	0.08	0.09
query12	0.11	0.09	0.09
query13	0.61	0.60	0.60
query14	0.78	0.81	0.80
query15	0.77	0.75	0.76
query16	0.37	0.36	0.37
query17	1.03	1.04	0.97
query18	0.23	0.25	0.25
query19	1.91	1.85	1.87
query20	0.02	0.01	0.01
query21	15.44	0.56	0.55
query22	1.99	2.28	1.49
query23	16.99	0.95	0.91
query24	6.77	0.90	0.86
query25	0.41	0.10	0.08
query26	0.62	0.16	0.15
query27	0.04	0.04	0.03
query28	6.71	0.74	0.71
query29	12.70	2.34	2.35
query30	0.57	0.57	0.54
query31	2.78	0.38	0.38
query32	3.38	0.50	0.49
query33	3.08	3.05	3.06
query34	15.28	4.78	4.77
query35	4.86	4.82	4.81
query36	1.04	1.00	1.01
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.02	0.02
query40	0.16	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 104.07 s
Total hot run time: 30.47 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d6be9a3bf992b0ba4da45be9627881c46eb8b1bf with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@csun5285 csun5285 marked this pull request as draft October 10, 2024 10:31
@csun5285 csun5285 mentioned this pull request Oct 20, 2024
Merged
airborne12 pushed a commit that referenced this pull request Oct 22, 2024
@csun5285
[test] (inverted index) add some regression cases (#42131)
1915fcd 
## Proposed changes

I cherry-picked the regression test cases for issues fixed in branch-2.0
and branch-2.1 to ensure the correctness of the master branch.
#41200 #41297 #40425 #40630
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@csun5285 @doris-robot