Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reconfigurator] ClickHouse XML generation types #6412

Merged
merged 15 commits into from
Aug 28, 2024

Conversation

karencfv
Copy link
Contributor

@karencfv karencfv commented Aug 22, 2024

Overview

This commit adds a library to generate ClickHouse replica server and keeper configuration files. A lot of the code in the clickhouse-admin/types directory has been copied over from clickward, but there are several additions and modifications:

  • New new() and default() methods that set default Oxide values.
  • File generation is per node, as opposed to all files generated in a single directory like clickward.

Usage

To generate a replica server configuration file:

let keepers = vec![
    KeeperNodeConfig::new("ff::01".to_string()),
    KeeperNodeConfig::new("ff::02".to_string()),
    KeeperNodeConfig::new("ff::03".to_string()),
];

let servers = vec![
    ServerNodeConfig::new("ff::08".to_string()),
    ServerNodeConfig::new("ff::09".to_string()),
];

let config = ClickhouseServerConfig::new(
    Utf8PathBuf::from(config_dir.path()),
    ServerId(1),
    Utf8PathBuf::from_str("./").unwrap(),
    Ipv6Addr::from_str("ff::08").unwrap(),
    keepers,
    servers,
);

config.generate_xml_file().unwrap();

To generate a keeper configuration file:

let keepers = vec![
    RaftServerConfig::new(KeeperId(1), "ff::01".to_string()),
    RaftServerConfig::new(KeeperId(2), "ff::02".to_string()),
    RaftServerConfig::new(KeeperId(3), "ff::03".to_string()),
];

let config = ClickhouseKeeperConfig::new(
    Utf8PathBuf::from(config_dir.path()),
    KeeperId(1),
    keepers,
    Utf8PathBuf::from_str("./").unwrap(),
    Ipv6Addr::from_str("ff::08").unwrap(),
);

config.generate_xml_file().unwrap();

Purpose

As part of the work to roll out replicated ClickHouse, we'll need to dynamically generate the node configuration files via the clickhouse-admin service. This commit is part of the work necessary to do so.

Related: #5999 , #3824

@karencfv karencfv changed the title [reconfigurator] ClickHouse XML generation types [reconfigurator] WIP ClickHouse XML generation types Aug 23, 2024
Comment on lines +36 to +40
pub const CLICKHOUSE_HTTP_PORT: u16 = 8123;
pub const CLICKHOUSE_INTERSERVER_PORT: u16 = 9009;
pub const CLICKHOUSE_TCP_PORT: u16 = 9000;
pub const CLICKHOUSE_KEEPER_TCP_PORT: u16 = 9181;
pub const CLICKHOUSE_KEEPER_RAFT_PORT: u16 = 9234;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These ports are the default ClickHouse ports https://clickhouse.com/docs/en/guides/sre/network-ports

Comment on lines +59 to +61
pub config_dir: Utf8PathBuf,
pub id: ServerId,
pub datastore_path: Utf8PathBuf,
Copy link
Contributor Author

@karencfv karencfv Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The datastore_path and config_dir must be different directories. The clickhouse_{server|keeper} zones will have the clickhouse files like logs in the datastore directory, and the configuration files will need to live in in /opt/oxide/clickhouse_{server|keeper}/config.d/config_{server|keeper}.xml

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Thanks.

@karencfv karencfv marked this pull request as ready for review August 27, 2024 04:00
@karencfv karencfv changed the title [reconfigurator] WIP ClickHouse XML generation types [reconfigurator] ClickHouse XML generation types Aug 27, 2024
Copy link
Contributor

@andrewjstone andrewjstone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Karen, this looks great! Just some minor comments.

Comment on lines +59 to +61
pub config_dir: Utf8PathBuf,
pub id: ServerId,
pub datastore_path: Utf8PathBuf,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Thanks.

clickhouse-admin/types/src/lib.rs Outdated Show resolved Hide resolved
clickhouse-admin/types/src/lib.rs Show resolved Hide resolved
clickhouse-admin/types/src/config.rs Outdated Show resolved Hide resolved
clickhouse-admin/types/src/lib.rs Show resolved Hide resolved
clickhouse-admin/types/src/config.rs Show resolved Hide resolved
clickhouse-admin/types/src/lib.rs Show resolved Hide resolved
Copy link
Contributor Author

@karencfv karencfv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @andrewjstone! I'll update the PR with your comments

clickhouse-admin/types/src/config.rs Show resolved Hide resolved
clickhouse-admin/types/src/lib.rs Show resolved Hide resolved
@karencfv karencfv enabled auto-merge (squash) August 28, 2024 01:27
@karencfv karencfv merged commit 57b0900 into oxidecomputer:main Aug 28, 2024
18 checks passed
@karencfv karencfv deleted the clickward branch August 28, 2024 04:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants