Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage/sources/postgres: avoid running incompatible queries against PostgreSQL <15 #29576

Open
benesch opened this issue Sep 16, 2024 · 0 comments
Labels
A-CLUSTER Topics related to the CLUSTER layer

Comments

@benesch
Copy link
Member

benesch commented Sep 16, 2024

When fetching schema info from PostgreSQL, the PostgreSQL source currently issues a query that's known to be compatible only with v15+. If the query fails, the source retries with a version of the query that's compatible with older versions of PostgreSQL:

// PG 15 adds UNIQUE NULLS NOT DISTINCT, which would let us use `UNIQUE` constraints over
// nullable columns as keys; i.e. aligns a PG index's NULL handling with an arrangement's
// keys. For more info, see https://www.postgresql.org/about/featurematrix/detail/392/
let pg_15_plus_keys = "
SELECT
pg_constraint.oid,
pg_constraint.conkey,
pg_constraint.conname,
pg_constraint.contype = 'p' AS is_primary,
pg_index.indnullsnotdistinct AS nulls_not_distinct
FROM
pg_constraint
JOIN
pg_index
ON pg_index.indexrelid = pg_constraint.conindid
WHERE
pg_constraint.conrelid = $1
AND
pg_constraint.contype =ANY (ARRAY['p', 'u']);";
// As above but for versions of PG without indnullsnotdistinct.
let pg_14_minus_keys = "
SELECT
pg_constraint.oid,
pg_constraint.conkey,
pg_constraint.conname,
pg_constraint.contype = 'p' AS is_primary,
false AS nulls_not_distinct
FROM pg_constraint
WHERE
pg_constraint.conrelid = $1
AND
pg_constraint.contype =ANY (ARRAY['p', 'u']);";
let keys = match client.query(pg_15_plus_keys, &[&oid]).await {
Ok(keys) => keys,
Err(e)
// PG versions prior to 15 do not contain this column.
if e.to_string()
== "db error: ERROR: column pg_index.indnullsnotdistinct does not exist" =>
{
client.query(pg_14_minus_keys, &[&oid]).await.map_err(PostgresError::from)?
}
e => e.map_err(PostgresError::from)?,
};

While this works fine for customers running versions of PostgreSQL before v15, it has an annoying side effect: those customers can observe the failed queries in their PostgreSQL log files. This has led at least two customers to raise concerns:

We should consider switching to sniffing out the version and deciding up front which query to run, to avoid running a query that's guaranteed to fail against older versions of PostgreSQL.

@benesch benesch added the A-CLUSTER Topics related to the CLUSTER layer label Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CLUSTER Topics related to the CLUSTER layer
Projects
None yet
Development

No branches or pull requests

1 participant