Fix bug when reference test case of postgres_fdw.sql #43
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 5 commits
March 10, 2022 12:46
…lter for system column in extract_rowgroup_filters()
**Describe the bug**
Run test by makecheck in debug mode: crash if WHERE clause contain system column
**To Reproduce**
1. build to debug mode
Makefile: -g -O0
2. Makecheck
3. Add these SQL query into test file (reference to test case of postgres_fdw.sql)
CREATE FOREIGN TABLE ft1 (
c0 int,
c1 int NOT NULL,
c2 int NOT NULL,
c3 text,
-- c4 timestamptz,
c5 timestamp,
c6 text,
c7 text,
c8 text
) SERVER parquet_srv
OPTIONS (filename :'/ported_postgres/ft1.parquet', sorted 'c1');
ALTER FOREIGN TABLE ft1 DROP COLUMN c0;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT * FROM ft1 t1 WHERE t1.ctid = '(0,2)';
**Expected behavior**
QUERY PLAN
--------------------------------------
Foreign Scan on public.ft1 t1
Output: c1, c2, c3, c5, c6, c7, c8
Filter: (t1.ctid = '(0,2)'::tid)
Reader: Single File
Row groups: 1
(5 rows)
4. Crash in testcase: SELECT * FROM ft1 t1 WHERE t1.ctid = '(0,2)';
TRAP: FailedAssertion("strlen(input) < NAMEDATALEN - 1", File: "src/common.cpp", Line: 143, PID: 25595)
parquet_fdw.so(_Z11tolowercasePKcPc+0x48)[0x7fce0e1fed2f]
parquet_fdw.so(_Z22extract_rowgroups_listPKcS0_PN3Aws2S38S3ClientEP13TupleDescDataRSt4listI14RowGroupFilterSaIS8_EEPmSC_+0x536)[0x7fce0e21df07]
parquet_fdw.so(parquetGetForeignRelSize+0x243)[0x7fce0e22023b]
**Additional context**
Crash happen when makecheck in debug mode, and not happen on release mode.
Because debug mode, the value of pg_comumn (extract_rowgroups_list) will not be optimized like release mode when receiving an invalid value (filter.attnum = -1 (system column))
…SubPlan is always parallel restricted.
**Describe the bug**
Crash when lateral join in parallel mode
**To Reproduce**
1. Run testcase (reference test case of postgres_fdw.sql)
CREATE SCHEMA "S 1";
IMPORT FOREIGN SCHEMA "/ported_postgres" FROM SERVER parquet_s3_srv INTO "S 1" OPTIONS (sorted 'c1');
CREATE FOREIGN TABLE ft1 (
c0 int,
c1 int NOT NULL,
c2 int NOT NULL,
c3 text,
-- c4 timestamptz,
c5 timestamp,
c6 text,
c7 text,
c8 text
) SERVER parquet_srv
OPTIONS (filename 'ported_postgres/ft1.parquet', sorted 'c1');
ALTER FOREIGN TABLE ft1 DROP COLUMN c0;
CREATE FOREIGN TABLE ft2 (
c1 int NOT NULL,
c2 int NOT NULL,
cx int,
c3 text,
-- c4 timestamptz,
c5 timestamp,
c6 text,
c7 text,
c8 text
) SERVER parquet_srv
OPTIONS (filename 'ported_postgres/ft1.parquet', sorted 'c1');
ALTER FOREIGN TABLE ft2 DROP COLUMN cx;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT ref_0.c2, subq_1.*
FROM
"S 1"."T1" AS ref_0,
LATERAL (
SELECT ref_0.c1 c1, subq_0.*
FROM (SELECT ref_0.c2, ref_1.c3
FROM ft1 AS ref_1) AS subq_0
RIGHT JOIN ft2 AS ref_3 ON (subq_0.c3 = ref_3.c3)
) AS subq_1
WHERE ref_0.c1 < 10 AND subq_1.c3 = '00001'
ORDER BY ref_0.c1;
2. Crash
psql: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
3. Log
TRAP: FailedAssertion("bms_is_subset(baserel->lateral_relids, required_outer)", File: "relnode.c", Line: 1304, PID: 17124)
[local] EXPLAIN(ExceptionalCondition+0xb9)[0xb0a40a]
[local] EXPLAIN(get_baserel_parampathinfo+0x4b)[0x882524]
[local] EXPLAIN(create_gather_path+0xac)[0x873966]
[local] EXPLAIN(generate_gather_paths+0x9e)[0x8061aa]
[local] EXPLAIN(generate_useful_gather_paths+0x7e)[0x806489]
[local] EXPLAIN[0x802cd8]
[local] EXPLAIN[0x802876]
[local] EXPLAIN(make_one_rel+0x1ba)[0x8025d8]
[local] EXPLAIN(query_planner+0x373)[0x83ed1f]
**Expected behavior**
QUERY PLAN
--------------------------------------------------------------------------------------------
Nested Loop
Output: ref_0.c2, ref_0.c1, (ref_0.c2), ref_1.c3, ref_0.c1
-> Nested Loop
Output: ref_0.c2, ref_0.c1, ref_1.c3, (ref_0.c2)
-> Foreign Scan on "S 1"."T1" ref_0
Output: ref_0.c1, ref_0.c2, ref_0.c3, ref_0.c5, ref_0.c6, ref_0.c7, ref_0.c8
Filter: (ref_0.c1 < 10)
Reader: Single File
Row groups: 1
-> Foreign Scan on public.ft1 ref_1
Output: ref_1.c3, ref_0.c2
Filter: (ref_1.c3 = '00001'::text)
Reader: Single File
Row groups: 1
-> Materialize
Output: ref_3.c3
-> Foreign Scan on public.ft2 ref_3
Output: ref_3.c3
Filter: (ref_3.c3 = '00001'::text)
Reader: Single File
Row groups: 1
(21 rows)
**Additional context**
LATERAL JOIN is a correlated SubPlan and cannot execute in parallel mode.
Refer parallel restricted: https://www.postgresql.org/docs/14/parallel-safety.html
…value.
**Describe the bug**
rescan wrong value for row_groups
**To Reproduce**
1. Execute SQL
select * from example1, example2 WHERE example1.one = 1;
2. Actual result
one | two | three | four | five | six | seven | one | two | three | four | five | six | seven
-----+-----+-------+------+------+-----+-------+-----+-----+-------+------+------+-----+-------
(0 rows)
**Expected behavior**
one | two | three | four | five | six | seven | one | two | three | four | five | six | seven
-----+---------+-------+---------------------+------------+-----+-------+-----+---------+-------+---------------------+------------+-----+-------
1 | {1,2,3} | foo | 2018-01-01 00:00:00 | 2018-01-01 | t | 0.5 | 1 | {19,20} | eins | 2018-01-01 00:00:00 | 2018-01-01 | t |
1 | {1,2,3} | foo | 2018-01-01 00:00:00 | 2018-01-01 | t | 0.5 | 3 | {21,22} | zwei | 2018-01-03 00:00:00 | 2018-01-03 | f |
1 | {1,2,3} | foo | 2018-01-01 00:00:00 | 2018-01-01 | t | 0.5 | 5 | {23,24} | drei | 2018-01-05 00:00:00 | 2018-01-05 | t |
1 | {1,2,3} | foo | 2018-01-01 00:00:00 | 2018-01-01 | t | 0.5 | 7 | {25,26} | vier | 2018-01-07 00:00:00 | 2018-01-07 | f |
1 | {1,2,3} | foo | 2018-01-01 00:00:00 | 2018-01-01 | t | 0.5 | 9 | {27,28} | fünf | 2018-01-09 00:00:00 | 2018-01-09 | t |
(5 rows)
**Additional context**
Current row_group initialized with -1 (contructor function)
When RescanForeignScan called. row_group is recaned with 0 -> wrong row group index
Then the datas are get by IterateForeignScan will be missing
…scan_path.
**Describe the bug**
Missing outer rel either (baserel->lateral_reldis). So crash when lateral outer join
**To Reproduce**
1. Run test case (reference test case of postgres_fdw.sql)
CREATE SCHEMA "S 1";
IMPORT FOREIGN SCHEMA "/ported_postgres" FROM SERVER parquet_s3_srv INTO "S 1" OPTIONS (sorted 'c1');
CREATE FOREIGN TABLE ft1 (
c0 int,
c1 int NOT NULL,
c2 int NOT NULL,
c3 text,
-- c4 timestamptz,
c5 timestamp,
c6 text,
c7 text,
c8 text
) SERVER parquet_srv
OPTIONS (filename 'ported_postgres/ft1.parquet', sorted 'c1');
ALTER FOREIGN TABLE ft1 DROP COLUMN c0;
CREATE FOREIGN TABLE ft2 (
c1 int NOT NULL,
c2 int NOT NULL,
cx int,
c3 text,
-- c4 timestamptz,
c5 timestamp,
c6 text,
c7 text,
c8 text
) SERVER parquet_srv
OPTIONS (filename 'ported_postgres/ft1.parquet', sorted 'c1');
ALTER FOREIGN TABLE ft2 DROP COLUMN cx;
EXPLAIN (VERBOSE, COSTS OFF)
SELECT ref_0.c2, subq_1.*
FROM
"S 1"."T1" AS ref_0,
LATERAL (
SELECT ref_0.c1 c1, subq_0.*
FROM (SELECT ref_0.c2, ref_1.c3
FROM ft1 AS ref_1) AS subq_0
RIGHT JOIN ft2 AS ref_3 ON (subq_0.c3 = ref_3.c3)
) AS subq_1
WHERE ref_0.c1 < 10 AND subq_1.c3 = '00001'
ORDER BY ref_0.c1;
2. Crash
psql: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
**Expected behavior**
QUERY PLAN
--------------------------------------------------------------------------------------------
Nested Loop
Output: ref_0.c2, ref_0.c1, (ref_0.c2), ref_1.c3, ref_0.c1
-> Nested Loop
Output: ref_0.c2, ref_0.c1, ref_1.c3, (ref_0.c2)
-> Foreign Scan on "S 1"."T1" ref_0
Output: ref_0.c1, ref_0.c2, ref_0.c3, ref_0.c5, ref_0.c6, ref_0.c7, ref_0.c8
Filter: (ref_0.c1 < 10)
Reader: Single File
Row groups: 1
-> Foreign Scan on public.ft1 ref_1
Output: ref_1.c3, ref_0.c2
Filter: (ref_1.c3 = '00001'::text)
Reader: Single File
Row groups: 1
-> Materialize
Output: ref_3.c3
-> Foreign Scan on public.ft2 ref_3
Output: ref_3.c3
Filter: (ref_3.c3 = '00001'::text)
Reader: Single File
Row groups: 1
(21 rows)
… fdw_private lists. Follow postgres style: https://github.com/postgres/postgres/blob/master/contrib/postgres_fdw/postgres_fdw.c
|
Ping @za-arthur, is this repository active? |
|
It is active, but unfortunately we don't have time to work on every PR. |
|
Thanks, @za-arthur ! Maybe README unification without code change will be easier? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix crash if WHERE clause contain system column: ignore extract to filter for system column.
Please check describe in commmit:
MinhLA1410@c3e6aa2
Fix crash when lateral join in parallel mode: reference a correlated SubPlan is always parallel restricted.
Please check describe in commmit:
MinhLA1410@41c4fbe
Fix rescan wrong value for row_groups: row_group need rescan with -1 value.
Please check describe in commmit:
MinhLA1410@7c89a77
Support lateral join: add baserel->lateral_relids when create_foreignscan_path.
Please check describe in commmit:
MinhLA1410@535c6d7
Use FdwScanPrivateIndex: Indexes of FDW-private information stored in fdw_private lists.
Please check describe in commmit:
MinhLA1410@f040cc4