Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clickhouse cluster config generation optimization #6909

Open
andrewjstone opened this issue Oct 21, 2024 · 0 comments
Open

Clickhouse cluster config generation optimization #6909

andrewjstone opened this issue Oct 21, 2024 · 0 comments
Assignees

Comments

@andrewjstone
Copy link
Contributor

When a the reconfiguration executor runs it pushes the latest configuration settings for clickhouse-server and clickhouse-keeper to their corresponding admin servers. These servers generate the XML configuration files which the servers and keepers will automatically reload.

The latest configuration settings are pushed on every execution which runs periodically (every 30s?). We don't want to rewrite the config file and reload it every time if nothing has changed. This burns flash lifetime and leaves the door open to failures to both save the file and reload config.

Instead we should cache the configuration settings, including, most importantly, the generation number of the configuration in a file in the persistent dataset of each clickhouse server and keeper. Then on every configuration we can check to see if the configuration matches the persisted generation. Only if the pushed configuration from the executor has a newer generation do we overwrite the cached settings and rewrite the XML configuration.

While we could solely persist the generation number, we choose to persist all the pushed settings in the cache file. This is solely for debugging purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants