Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup job for refreshing materialized view(s) #12

Open
maxachis opened this issue Jul 6, 2024 · 1 comment
Open

Setup job for refreshing materialized view(s) #12

maxachis opened this issue Jul 6, 2024 · 1 comment

Comments

@maxachis
Copy link
Collaborator

maxachis commented Jul 6, 2024

Now that we're setting up a materialized view as per Police-Data-Accessibility-Project/data-sources-app#345, we need to make sure it's refreshed regularly. PostgreSQL does not automatically refresh materialized views, so we would need to set up a job for managing this -- fortunately, we already have a function in dev_scripts.sql which can refresh our currently-only materialized view, so it'd be just a matter of calling that in the requisite environments.

There are a few considerations to consider:

How often to refresh?

The typeahead_suggestions materialized view takes data from the agencies, counties, and state_names tables, which will likely get data updates in descending order of frequency (with states probably never getting updated unless greater Idaho forms).

Do we put the code in here and rename the repository?

It'd be overkill to create a repository for one itty-bitty script, so naturally it'd make sense to put it in this repository. If we do that, though, the scope of this repository starts to expand to include a multitude of regular interfacing scripts. Maybe we should have one repository for migrations and another for regular database automation jobs. Or maybe we keep both in here. I'm not sure which is the best option.

@josh-chamberlain
Copy link
Contributor

  • we could rename this repository to database-migrations or something.
  • this needs to refresh as often as the names of geographic places changes, which is not too bad. every day or week?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants