Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to create unlogged IMMVs #91

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Jamal-B
Copy link

@Jamal-B Jamal-B commented Jul 22, 2024

Description

In some cases (e.g. for very large IMMVs containing volatile data), and provided you are well aware of the risks or drawbacks involved, it may be useful to create an IMMV without writing to PostgreSQL Write-Ahead Logs.

PostgreSQL unlogged tables drawbacks are multiple (not replicated, not backuped, ...) but the main one to keep in mind is that they are automatically truncated after a crash or unclean shutdown. If these conditions meet your requirements, this, as with simple unlogged tables, allows to improve write performance, reduce vacuum impact, and produce fewer WALs (thus reducing network usage, backup size and restoration time).

I hesitated to propose this PR because allowing unlogged IMMVs goes a bit against the fact that PostgreSQL does not allow the creation of unlogged materialized views. But since this feature (based on the fact that IMMVs are, under the hood, classical PostgreSQL tables feeded by triggers) is very useful in our case, I thought that it could also be the same for some other users of this superb extension that is pg_ivm.

Please note that I am not a C developer, and I would obviously be more than happy to consider your comments related to the code... or, of course, the feature itself.

Additionally, I organized the commits based on the project's git history, particularly the Prepare 1.10 commit, without being sure that the PR, if accepted, would merit a version bump. I'll let you judge what's best to do.

Tests

  1. IMMVs are created according to the defined persistence
my_database=# \df create_immv
/*
                                   List of functions
   Schema   |    Name     | Result data type |        Argument data types        | Type 
------------+-------------+------------------+-----------------------------------+------
 pg_catalog | create_immv | bigint           | text, text, boolean DEFAULT false | func
*/

select create_immv('test_immv_1', 'select 1');
select create_immv('test_immv_2', 'select 1', false);
select create_immv('test_immv_3', 'select 1', true);

select relname, relpersistence
from   pg_class
where  relname ~ 'test_immv_';

/*
   relname   | relpersistence 
-------------+----------------
 test_immv_1 | p
 test_immv_2 | p
 test_immv_3 | u
*/
  1. IMMVs persistence is preserved after refresh_immv() using with_data = true
select refresh_immv('test_immv_1', true);
select refresh_immv('test_immv_2', true);
select refresh_immv('test_immv_3', true);

select relname, relpersistence
from   pg_class
where  relname ~ 'test_immv_';

/*
   relname   | relpersistence 
-------------+----------------
 test_immv_1 | p
 test_immv_2 | p
 test_immv_3 | u
*/
  1. IMMVs persistence is preserved after refresh_immv() using with_data = false
select refresh_immv('test_immv_1', false);
select refresh_immv('test_immv_2', false);
select refresh_immv('test_immv_3', false);

select relname, relpersistence
from   pg_class
where  relname ~ 'test_immv_';

/*
   relname   | relpersistence 
-------------+----------------
 test_immv_1 | p
 test_immv_2 | p
 test_immv_3 | u
*/

@yugo-n yugo-n self-requested a review July 23, 2024 07:45
@Jamal-B Jamal-B force-pushed the allow-to-create-unlogged-immv branch from 0cd7c28 to 434ff8f Compare August 26, 2024 15:21
@Jamal-B
Copy link
Author

Jamal-B commented Aug 26, 2024

I just pushed a rebased (and tested) version of this PR which follows the merge of version 1.9 a few weeks ago.

@Jamal-B Jamal-B force-pushed the allow-to-create-unlogged-immv branch from 434ff8f to e053a9e Compare August 26, 2024 15:54
@yugo-n
Copy link
Collaborator

yugo-n commented Oct 14, 2024

Thank you for your pull request.

Sorry for late reply, but after some consideration, I don't think supporting unlogged IMMVs is a good idea for now for the same reason why unlogged materialized views are disallowed in PostgreSQL.

Unlogged materialized views has been disallowed since the beginning of the matview support in PosgreSQL, i.e. 9.3, by the commit below.
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=3223b25ff737c2bf4a642c0deb7be2b30bfecc6e

As explained in the commit message and comments in the code, When a materialized view is made empty after a crash, user cannot distinguish if this has been unpopulated due to the crash or this is actually empty because the system catalog changes (disabling pg_class.relispopulated) are not handled by crash recovery. pg_ivm has the same problem, or It is worse for pg_ivm because IMMV's contents are supposed to be always up-to-date by the immediate maintenance. For this reason, I don't have plan to support unlogged IMMVs for now.

@yugo-n yugo-n closed this Oct 21, 2024
@yugo-n yugo-n reopened this Oct 21, 2024
Copy link
Collaborator

@yugo-n yugo-n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the comments in the Conversation.
#91 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants