diff --git a/README.md b/README.md index 029d5077..ca8fa0ed 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ # PREvant In a Nutshell -PREvant a is Docker container that serves as an abstraction layer between continuous integration pipelines and a container orchestration platform. This abstraction serves as a reviewing platform to ensure that developers have built the features that domain expert requested. +PREvant a is Docker container that serves as an abstraction layer between continuous integration pipelines and a container orchestration platform. This abstraction serves as a reviewing platform to ensure that developers have built the features that domain expert requested. PREvant's name originates from this requirement: _Preview servant (PREvant, `prɪˈvɛnt`, it's pronounced like prevent)_ __serves__ developers to deploy previews of their application as simple as possible when their application consists of multiple microservices distributed across multiple source code repositories. These previews should __PREvant__ to do mistakes in feature development because domain experts can review changes as soon as possible. @@ -14,6 +14,40 @@ Through PREvant's web interface domain experts, managers, developers, and sales ![Access the application](assets/screenshot.png "Access the application") +## Basic Terminology + +An *application*, that PREvant manages, is a composition of microservices based +on an “architectural pattern that arranges an application as a collection of +loosely coupled, fine-grained services, communicating through lightweight +protocols.” ([Wikipedia][wiki-microservices]) Each application has a unique +name which is the key to perform actions like creating, duplicating, modifying, +or deleting these applications via REST API or Web UI. + +In each application, PREvant manages the microservices as *services* which need +to be available in the [OCI Image Format][oci-image-spec] (a.k.a. Docker +images). At least one service needs to be available for an application. PREvant +manages the following kind of services: + +- *Instance*: a service labeled as instance is a service that has been + configured explicitly when creating or updating an application. +- *Replica*: a service labeled as replica is a service that has been replicated + from another application. By default if you create an application under any + name PREvant will replicate all instances from the application *master*. + Alternatively, any other application can be specified as a source of + replication. + +Additionally, PREvant provides a way of creating service everytime it creates +an application. These services are called *companions* and there are two types +of them. + +- An application wide companion (app companion) is an unique service for the + whole application. For example, a [Kafka][kafka] instance can be started + automatically everytime you create an application so that all services within + the application can synchronize via events. +- A companion can also be attached to a service a user wants to deploy (service + companion). For example, a [PostgreSQL][postgres] container can be started + for each service to provide a dedicated database for it. + # Usage Have a look at the examples directory. There you can find examples that deploy PREvant in different container environments: @@ -69,3 +103,8 @@ This paper is based on [the abstract](https://www.conf-micro.services/2019/paper The talk is available on [YouTube](http://www.youtube.com/watch?v=O9GxapQR5bk). Click on the image to start the playback: [![Video “PREvant: Composing Microservices into Reviewable and Testable Applications” at Microservices 2019](http://img.youtube.com/vi/O9GxapQR5bk/0.jpg)](http://www.youtube.com/watch?v=O9GxapQR5bk) + +[wiki-microservices]: https://en.wikipedia.org/wiki/Microservices +[oci-image-spec]: https://specs.opencontainers.org/image-spec/ +[kafka]: https://kafka.apache.org +[postgres]: https://www.postgresql.org diff --git a/api/Cargo.lock b/api/Cargo.lock index e41a023d..2f6751c6 100644 --- a/api/Cargo.lock +++ b/api/Cargo.lock @@ -186,12 +186,6 @@ version = "0.13.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" -[[package]] -name = "base64" -version = "0.20.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0ea22880d78093b0cbe17c89f64a7d457941e65759157ec6cb31a31d652b05e5" - [[package]] name = "base64" version = "0.21.7" @@ -547,9 +541,9 @@ dependencies = [ [[package]] name = "darling" -version = "0.14.4" +version = "0.20.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7b750cb3417fd1b327431a470f388520309479ab0bf5e323505daf0290cd3850" +checksum = "0209d94da627ab5605dcccf08bb18afa5009cfbef48d8a8b7d7bdbc79be25c5e" dependencies = [ "darling_core", "darling_macro", @@ -557,27 +551,27 @@ dependencies = [ [[package]] name = "darling_core" -version = "0.14.4" +version = "0.20.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "109c1ca6e6b7f82cc233a97004ea8ed7ca123a9af07a8230878fcfda9b158bf0" +checksum = "177e3443818124b357d8e76f53be906d60937f0d3a90773a664fa63fa253e621" dependencies = [ "fnv", "ident_case", "proc-macro2", "quote", "strsim", - "syn 1.0.109", + "syn 2.0.48", ] [[package]] name = "darling_macro" -version = "0.14.4" +version = "0.20.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a4aab4dbc9f7611d8b55048a3a16d2d010c2c8334e46304b40ac1cc14bf3b48e" +checksum = "836a9bbc7ad63342d6d6e7b815ccab164bc77a2d95d84bc3117a8c0d5c98e2d5" dependencies = [ "darling_core", "quote", - "syn 1.0.109", + "syn 2.0.48", ] [[package]] @@ -646,27 +640,6 @@ dependencies = [ "subtle", ] -[[package]] -name = "dirs-next" -version = "2.0.0" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b98cf8ebf19c3d1b223e151f99a4f9f0690dca41414773390fc824184ac833e1" -dependencies = [ - "cfg-if", - "dirs-sys-next", -] - -[[package]] -name = "dirs-sys-next" -version = "0.1.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4ebda144c4fe02d1f7ea1a7d9641b6fc6b580adcfa024ae48797ecdeb6825b4d" -dependencies = [ - "libc", - "redox_users", - "winapi", -] - [[package]] name = "displaydoc" version = "0.2.4" @@ -790,7 +763,7 @@ dependencies = [ "pear", "serde", "tempfile", - "toml 0.8.8", + "toml", "uncased", "version_check", ] @@ -1082,6 +1055,15 @@ dependencies = [ "digest", ] +[[package]] +name = "home" +version = "0.5.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e3d1354bf6b7235cb4a0576c2619fd4ed18183f689b12b006a0ee7329eeff9a5" +dependencies = [ + "windows-sys 0.52.0", +] + [[package]] name = "http" version = "0.2.11" @@ -1454,14 +1436,16 @@ dependencies = [ ] [[package]] -name = "jsonpath_lib" -version = "0.3.0" +name = "jsonpath-rust" +version = "0.3.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "eaa63191d68230cccb81c5aa23abd53ed64d83337cacbb25a7b8c7979523774f" +checksum = "06cc127b7c3d270be504572364f9569761a180b981919dd0d87693a7f5fb7829" dependencies = [ - "log", - "serde", + "pest", + "pest_derive", + "regex", "serde_json", + "thiserror", ] [[package]] @@ -1481,9 +1465,9 @@ dependencies = [ [[package]] name = "k8s-openapi" -version = "0.18.0" +version = "0.20.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cd990069640f9db34b3b0f7a1afc62a05ffaa3be9b66aa3c313f58346df7f788" +checksum = "edc3606fd16aca7989db2f84bb25684d0270c6d6fa1dbcd0025af7b4130523a6" dependencies = [ "base64 0.21.7", "bytes 1.5.0", @@ -1495,9 +1479,9 @@ dependencies = [ [[package]] name = "kube" -version = "0.84.0" +version = "0.87.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "14bd236a6f6ddeac3fefa2863eb4e363cb3a2c49d66619e181b5b8f8f0787575" +checksum = "3499c8d60c763246c7a213f51caac1e9033f46026904cb89bc8951ae8601f26e" dependencies = [ "k8s-openapi", "kube-client", @@ -1507,22 +1491,22 @@ dependencies = [ [[package]] name = "kube-client" -version = "0.84.0" +version = "0.87.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "04a28620131ca89b2509e52f5e1b71bfa3e61a50321836b2ae373bc18e0309e6" +checksum = "033450dfa0762130565890dadf2f8835faedf749376ca13345bcd8ecd6b5f29f" dependencies = [ - "base64 0.20.0", + "base64 0.21.7", "bytes 1.5.0", "chrono", - "dirs-next", "either", "futures 0.3.30", + "home", "http", "http-body", "hyper", "hyper-rustls", "hyper-timeout", - "jsonpath_lib", + "jsonpath-rust", "k8s-openapi", "kube-core", "pem", @@ -1543,9 +1527,9 @@ dependencies = [ [[package]] name = "kube-core" -version = "0.84.0" +version = "0.87.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8227a989f1eeee3bcbf045165d6aca462af3744ecd4dfdcfba81051fb7de428e" +checksum = "b5bba93d054786eba7994d03ce522f368ef7d48c88a1826faa28478d85fb63ae" dependencies = [ "chrono", "form_urlencoded", @@ -1560,15 +1544,15 @@ dependencies = [ [[package]] name = "kube-derive" -version = "0.84.0" +version = "0.87.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "19d227fcf3e12f53ea1a38d4766a8c29f8b27795579e4146464effb88d52dd99" +checksum = "91e98dd5e5767c7b894c1f0e41fd628b145f808e981feb8b08ed66455d47f1a4" dependencies = [ "darling", "proc-macro2", "quote", "serde_json", - "syn 1.0.109", + "syn 2.0.48", ] [[package]] @@ -1583,17 +1567,6 @@ version = "0.2.152" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "13e3bf6590cbc649f4d1a3eefc9d5d6eb746f5200ffb04e5e142700b8faa56e7" -[[package]] -name = "libredox" -version = "0.0.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "85c833ca1e66078851dba29046874e38f08b2c883700aa29a03ddd3b23814ee8" -dependencies = [ - "bitflags 2.4.1", - "libc", - "redox_syscall", -] - [[package]] name = "linux-raw-sys" version = "0.4.12" @@ -1965,11 +1938,12 @@ dependencies = [ [[package]] name = "pem" -version = "1.1.1" +version = "3.0.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a8835c273a76a90455d7344889b0964598e3316e2a79ede8e36f16bdcf2228b8" +checksum = "1b8fcc794035347fb64beda2d3b462595dd2753e3f268d89c5aae77e8cf2c310" dependencies = [ - "base64 0.13.1", + "base64 0.21.7", + "serde", ] [[package]] @@ -2186,6 +2160,7 @@ dependencies = [ "pest", "pest_derive", "regex", + "regex-syntax 0.8.2", "reqwest", "rocket", "schemars", @@ -2200,7 +2175,7 @@ dependencies = [ "shiplift", "tempfile", "tokio", - "toml 0.7.8", + "toml", "url", "uuid", "yansi 0.5.1", @@ -2286,17 +2261,6 @@ dependencies = [ "bitflags 1.3.2", ] -[[package]] -name = "redox_users" -version = "0.4.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a18479200779601e498ada4e8c1e1f50e3ee19deb0259c25825a98b5603b2cb4" -dependencies = [ - "getrandom", - "libredox", - "thiserror", -] - [[package]] name = "ref-cast" version = "1.0.22" @@ -2738,7 +2702,6 @@ version = "1.0.111" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "176e46fa42316f18edd598015a5166857fc835ec732f5215eac6b7bdbf0a84f4" dependencies = [ - "indexmap", "itoa", "ryu", "serde", @@ -3208,18 +3171,6 @@ dependencies = [ "tracing", ] -[[package]] -name = "toml" -version = "0.7.8" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dd79e69d3b627db300ff956027cc6c3798cef26d22526befdfcd12feeb6d2257" -dependencies = [ - "serde", - "serde_spanned", - "toml_datetime", - "toml_edit 0.19.15", -] - [[package]] name = "toml" version = "0.8.8" @@ -3248,8 +3199,6 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1b5bb770da30e5cbfde35a2d7b9b8a2c4b8ef89548a7a6aeab5c9a576e3e7421" dependencies = [ "indexmap", - "serde", - "serde_spanned", "toml_datetime", "winnow", ] diff --git a/api/Cargo.toml b/api/Cargo.toml index 19a6b67e..a1513a36 100644 --- a/api/Cargo.toml +++ b/api/Cargo.toml @@ -26,15 +26,16 @@ futures = { version = "0.3", features = ["compat"] } handlebars = "4.5" http-api-problem = "0.57" jira_query = "1.3" -k8s-openapi = { version = "0.18", default-features = false, features = ["v1_24"] } -kube = { version = "0.84", default-features = false, features = ["client", "derive", "rustls-tls"] } +k8s-openapi = { version = "0.20", default-features = false, features = ["v1_24"] } +kube = { version = "0.87", default-features = false, features = ["client", "derive", "rustls-tls"] } lazy_static = "1.4" log = "0.4" multimap = "0.9" oci-distribution = "0.10" pest = "2.6" pest_derive = "2.6" -regex = "1.9" +regex = "1.10" +regex-syntax = "0.8" reqwest = { version = "0.11", features = ["json"] } rocket = { version = "0.5", features = ["json"] } schemars = "0.8" @@ -46,9 +47,9 @@ serde_json = "1.0" serde_regex = "1.1" serde_yaml = "0.9" tokio = { version = "1.29", features = ["macros", "rt", "rt-multi-thread", "sync", "time"] } -toml = "0.7" +toml = "0.8" url = { version = "2.4", features = ["serde"] } -uuid = { version = "1.3", features = ["serde", "v4"] } +uuid = { version = "1.5", features = ["serde", "v4"] } yansi = "0.5" diff --git a/api/README.md b/api/README.md index 6b4ca2d3..c70184ed 100644 --- a/api/README.md +++ b/api/README.md @@ -94,123 +94,7 @@ data = "LS0tLS1CRUdJTiBFTkNSWVBURUQgUF…JVkFURSBLRVktLS0tLQo=" ## Companions -It is possible to start containers that will be started when the client requests to create a new service. For example, if the application requires an [OpenID](https://en.wikipedia.org/wiki/OpenID_Connect) provider, it is possible to create a configuration that starts the provider for each application. Another use case might be a Kafka services that is required by the application. - -Furthermore, it is also possible to create containers for each service. For example, for each service a database container could be started. - -For these use cases following sections provide example configurations. - -### Application Wide - -If you want to include an OpenID provider for every application, you could use following configuration. - -```toml -[companions.openid] -type = 'application' -image = 'private.example.com/library/openid:latest' -env = [ 'KEY=VALUE' ] -``` - -The provided values of `serviceName` and `env` can include the [handlebars syntax](https://handlebarsjs.com/) in order to access dynamic values. - -Additionally, you could mount files that are generated from handlebars templates (example contains a properties generation): - -```toml -[companions.openid.volumes] -"/path/to/volume.properties" = """ -remote.services={{#each services~}} - {{~#if (eq type 'instance')~}} - {{name}}:{{port}}, - {{~/if~}} -{{~/each~}} -""" -``` - -Furthermore, you can provide labels through handlebars templating: - -```toml -[companions.openid.labels] -"com.github.prevant" = "bar-{{application.name}}" -``` - -#### Template Variables - -The list of available handlebars variables: - -- `application`: The companion's application information - - `name`: The application name -- `services`: An array of the services of the application. Each element has following structure: - - `name`: The service name which is equivalent to the network alias - - `port`: The exposed port of the service - - `type`: The type of service. For example, `instance`, `replica`, `app-companion`, or `service-companion`. - -#### Handlebar Helpers - -PREvant provides some handlebars helpers which can be used to generate more complex configuration files. See handlerbar's [block helper documentation](https://handlebarsjs.com/block_helpers.html) for more details. - -- `{{#isCompanion }}` A conditional handlerbars block helper that checks if the given service type matches any companion type. -- `isNotCompanion ` A conditional handlerbars block helper that checks if the given service type does not match any companion type. - -### Service Based - -The service-based companions works the in the same way as the application-based services. Make sure, that the `serviceName` is unique by using the handlebars templating. - -```toml -[companions.service-name] -serviceName = '{{service.name}}-db' -image = 'postgres:11' -env = [ 'KEY=VALUE' ] - -[companions.service-name.postgres.volumes] -"/path/to/volume.properties" == "…" -[companions.openid.labels] -"com.github.prevant" = "bar-{{application.name}}" -``` - - -#### Template Variables - -The list of available handlebars variables: - -- `application`: The companion's application information - - `name`: The application name -- `service`: The companion's service containing following fields: - - `name`: The service name which is equivalent to the network alias - - `port`: The exposed port of the service - - `type`: The type of service. For example, `instance`, `replica`, `app-companion`, or `service-companion`. - -### Deployment Strategy - -Companions offer different deployment strategies so that a companion could be restarted or not under certain conditions. Therefore, PREvant offers following configuration flags: - -```toml -[companions.openid] -type = 'application' -image = 'private.example.com/library/openid:latest' -deploymentStrategy = 'redeploy-on-image-update' -``` - -`deploymentStrategy` offers following values and if a companion exists for an app following strategy will be applied: - -- `redeploy-always` (_default_): Re-deploys the companion every time there is a new deployment request. -- `redeploy-on-image-update`: Re-deploys the companion if there is a more rescent image available. -- `redeploy-never`: Even if there is a new deployment request the companion won't be redeployed and stays running. - -### Storage Strategy - -Companions may have varying storage requirements and storage strategies cater to these by offering the below configuration flags: - -```toml -[companions.postgres] -type = 'application' -image = 'postgres:latest' -storageStrategy = 'mount-declared-image-volumes' -``` - -`storageStrategy` offers following values to determine how storage is managed for a companion: - -- `none` (_default_): Companion is deployed without persistent storage. -- `mount-declared-image-volumes`: Mounts the volume paths declared within the image, providing persistent storage for the companion. +See [here](../docs/companions.md) how to configure companions. ## Hooks diff --git a/api/src/apps/mod.rs b/api/src/apps/mod.rs index 2884ce04..f9454578 100644 --- a/api/src/apps/mod.rs +++ b/api/src/apps/mod.rs @@ -291,6 +291,12 @@ impl AppsService { let deployment_unit = if let Ok(Some(base_traefik_ingress_route)) = self.infrastructure.base_traefik_ingress_route().await { + trace!( + "The base URL for {app_name} is: {:?}", + base_traefik_ingress_route + .to_url() + .map(|url| url.to_string()) + ); deployment_unit_builder .apply_base_traefik_ingress_route(base_traefik_ingress_route) .build() @@ -346,7 +352,7 @@ impl AppsService { pub async fn get_logs( &self, app_name: &AppName, - service_name: &String, + service_name: &str, since: &Option>, limit: usize, ) -> Result, AppsServiceError> { @@ -364,7 +370,7 @@ impl AppsService { pub async fn change_status( &self, app_name: &AppName, - service_name: &String, + service_name: &str, status: ServiceStatus, ) -> Result, AppsServiceError> { Ok(self @@ -517,7 +523,7 @@ mod tests { let apps = AppsService::new(config, infrastructure)?; apps.create_or_update( - &AppName::from_str("master").unwrap(), + &AppName::master(), &AppStatusChangeId::new(), None, &vec![sc!("service-a"), sc!("service-b")], @@ -527,7 +533,7 @@ mod tests { apps.create_or_update( &AppName::from_str("branch").unwrap(), &AppStatusChangeId::new(), - Some(AppName::from_str("master").unwrap()), + Some(AppName::master()), &vec![sc!("service-b")], ) .await?; @@ -569,7 +575,7 @@ mod tests { apps.create_or_update( &AppName::from_str("branch").unwrap(), &AppStatusChangeId::new(), - Some(AppName::from_str("master").unwrap()), + Some(AppName::master()), &vec![sc!("service-a")], ) .await?; @@ -602,7 +608,7 @@ mod tests { let apps = AppsService::new(config, infrastructure)?; apps.create_or_update( - &AppName::from_str("master").unwrap(), + &AppName::master(), &AppStatusChangeId::new(), None, &vec![sc!("mariadb")], @@ -666,7 +672,7 @@ mod tests { let infrastructure = Box::new(Dummy::new()); let apps = AppsService::new(config, infrastructure)?; - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); apps.create_or_update( &app_name, @@ -863,7 +869,7 @@ Log msg 3 of service-a of app master let infrastructure = Box::new(Dummy::new()); let apps = AppsService::new(config, infrastructure)?; - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); apps.create_or_update( &app_name, @@ -948,7 +954,7 @@ Log msg 3 of service-a of app master let infrastructure = Box::new(Dummy::with_delay(std::time::Duration::from_millis(500))); let apps = Arc::new(AppsService::new(config, infrastructure)?); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); apps.create_or_update( &app_name, &AppStatusChangeId::new(), @@ -965,7 +971,7 @@ Log msg 3 of service-a of app master .unwrap(); rt.block_on(apps_clone.delete_app(&app_name, &AppStatusChangeId::new())) }); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let handle2 = std::thread::spawn(move || { let rt = runtime::Builder::new_current_thread() .enable_time() @@ -1061,7 +1067,7 @@ Log msg 3 of service-a of app master "#; let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = &AppName::from_str("master").unwrap(); + let app_name = &AppName::master(); let infrastructure = Box::new(Dummy::new()); let apps = AppsService::new(config, infrastructure)?; @@ -1093,7 +1099,7 @@ Log msg 3 of service-a of app master ))); let apps = AppsService::new(Config::default(), infrastructure)?; - let app_name = &AppName::from_str("master").unwrap(); + let app_name = &AppName::master(); apps.create_or_update( &app_name, &AppStatusChangeId::new(), @@ -1135,7 +1141,7 @@ Log msg 3 of service-a of app master let infrastructure = Box::new(Dummy::new()); let apps = AppsService::new(Config::default(), infrastructure)?; - let app_name = &AppName::from_str("master").unwrap(); + let app_name = &AppName::master(); apps.create_or_update( &app_name, &AppStatusChangeId::new(), diff --git a/api/src/apps/routes.rs b/api/src/apps/routes.rs index b306fec7..acfd93ab 100644 --- a/api/src/apps/routes.rs +++ b/api/src/apps/routes.rs @@ -189,13 +189,13 @@ async fn change_status( "//logs/?&", format = "text/plain" )] -async fn logs( +async fn logs<'r>( app_name: Result, - service_name: String, + service_name: &'r str, since: Option, limit: Option, apps: &State>, -) -> HttpResult { +) -> HttpResult> { let app_name = app_name?; let since = match since { @@ -214,7 +214,7 @@ async fn logs( let limit = limit.unwrap_or(20_000); let log_chunk = apps - .get_logs(&app_name, &service_name, &since, limit) + .get_logs(&app_name, service_name, &since, limit) .await?; Ok(LogsResponse { @@ -265,10 +265,10 @@ fn map_join_error(err: tokio::task::JoinError) -> HttpApiError { .into() } -pub struct LogsResponse { +pub struct LogsResponse<'a> { log_chunk: Option, app_name: AppName, - service_name: String, + service_name: &'a str, limit: usize, } @@ -284,7 +284,7 @@ impl CreateAppOptions { } } -impl<'r> Responder<'r, 'static> for LogsResponse { +impl<'r> Responder<'r, 'static> for LogsResponse<'r> { fn respond_to(self, _request: &'r Request) -> Result, Status> { use std::io::Cursor; let log_chunk = match self.log_chunk { @@ -810,7 +810,7 @@ mod tests { "type": "https://httpstatuses.com/400", "status": 400, "title": "Bad Request", - "detail": "Invalid image: private-registry.example.com/_/postgres at line 1 column 70" + "detail": "Invalid image: private-registry.example.com/_/postgres at line 1 column 51" }) ); } diff --git a/api/src/config/app_selector.rs b/api/src/config/app_selector.rs index 1f6ddb8f..6922eaf3 100644 --- a/api/src/config/app_selector.rs +++ b/api/src/config/app_selector.rs @@ -23,16 +23,17 @@ * THE SOFTWARE. * =========================LICENSE_END================================== */ +use crate::models::AppName; use regex::Regex; #[derive(Clone)] pub(super) struct AppSelector(Regex); impl AppSelector { - pub fn matches(&self, app_name: &str) -> bool { + pub fn matches(&self, app_name: &AppName) -> bool { match self.0.captures(app_name) { None => false, - Some(captures) => captures.get(0).map_or("", |m| m.as_str()) == app_name, + Some(captures) => captures.get(0).map_or("", |m| m.as_str()) == app_name.as_str(), } } } diff --git a/api/src/config/companion.rs b/api/src/config/companion.rs index aa150b5f..b2eabaa9 100644 --- a/api/src/config/companion.rs +++ b/api/src/config/companion.rs @@ -25,11 +25,21 @@ */ use crate::config::AppSelector; use crate::models::service::ContainerType; -use crate::models::{Environment, Image, Router, ServiceConfig}; +use crate::models::{AppName, Environment, Image, Router, ServiceConfig}; +use handlebars::{Handlebars, RenderError}; use secstr::SecUtf8; use serde_value::Value; use std::collections::BTreeMap; use std::path::PathBuf; +use url::Url; + +#[derive(Clone, Default, Deserialize)] +pub(super) struct Companions { + #[serde(default)] + bootstrapping: Bootstrapping, + #[serde(flatten)] + companions: BTreeMap, +} #[derive(Clone, Deserialize)] #[serde(rename_all = "camelCase")] @@ -78,12 +88,52 @@ pub enum DeploymentStrategy { RedeployNever, } +#[derive(Clone, Default, Deserialize)] +struct Bootstrapping { + containers: Vec, +} + +#[derive(Clone, Deserialize)] +pub struct BootstrappingContainer { + image: Image, + #[serde(default)] + args: Vec, +} + +impl Companions { + pub(super) fn companion_configs

( + &self, + app_name: &AppName, + predicate: P, + ) -> Vec<(ServiceConfig, DeploymentStrategy, StorageStrategy)> + where + P: Fn(&Companion) -> bool, + { + self.companions + .iter() + .filter(|(_, companion)| companion.matches_app_name(app_name)) + .filter(|(_, companion)| predicate(companion)) + .map(|(_, companion)| { + ( + ServiceConfig::from(companion.clone()), + companion.deployment_strategy().clone(), + companion.storage_strategy().clone(), + ) + }) + .collect() + } + + pub(super) fn companion_bootstrapping_containers(&self) -> &Vec { + &self.bootstrapping.containers + } +} + impl Companion { pub fn companion_type(&self) -> &CompanionType { &self.companion_type } - pub fn matches_app_name(&self, app_name: &str) -> bool { + pub fn matches_app_name(&self, app_name: &AppName) -> bool { self.app_selector.matches(app_name) } @@ -149,6 +199,47 @@ impl Default for StorageStrategy { } } +impl BootstrappingContainer { + pub fn image(&self) -> &Image { + &self.image + } + + pub fn templated_args( + &self, + app_name: &AppName, + base_url: &Option, + ) -> Result, RenderError> { + let handlebars = Handlebars::new(); + + #[derive(Serialize)] + #[serde(rename_all = "camelCase")] + struct AppData<'a> { + name: &'a str, + #[serde(skip_serializing_if = "Option::is_none")] + base_url: &'a Option, + } + // TODO: apply same pattern as for companions. {{application.name}}, {{service.…}}… + #[derive(Serialize)] + struct Data<'a> { + application: AppData<'a>, + } + + let data = Data { + application: AppData { + name: app_name, + base_url, + }, + }; + + let mut args = Vec::with_capacity(self.args.len()); + for arg in &self.args { + args.push(handlebars.render_template(arg, &data)?); + } + + Ok(args) + } +} + #[cfg(test)] mod tests { use super::*; @@ -160,6 +251,12 @@ mod tests { }; } + macro_rules! companions_from_str { + ( $config_str:expr ) => { + toml::de::from_str::($config_str).unwrap() + }; + } + #[test] fn should_parse_companion_with_required_fields() { let companion = companion_from_str!( @@ -181,4 +278,68 @@ mod tests { DeploymentStrategy::RedeployAlways ); } + + #[test] + fn should_parse_companion_bootstrap_containers() { + let companions = companions_from_str!( + r#" + [[bootstrapping.containers]] + image = "busybox" + "# + ); + + let container = &companions.bootstrapping.containers[0]; + + assert_eq!(container.image, Image::from_str("busybox").unwrap()); + assert_eq!( + container.templated_args(&AppName::master(), &None).unwrap(), + Vec::::new() + ); + } + + #[test] + fn should_parse_companion_bootstrap_containers_and_template_args() { + let companions = companions_from_str!( + r#" + [[bootstrapping.containers]] + image = "busybox" + args = [ "echo", "Hello {{application.name}}" ] + "# + ); + + let container = &companions.bootstrapping.containers[0]; + + assert_eq!(container.image, Image::from_str("busybox").unwrap()); + assert_eq!( + container.templated_args(&AppName::master(), &None).unwrap(), + vec![String::from("echo"), String::from("Hello master")] + ); + } + + #[test] + fn should_parse_companion_bootstrap_containers_and_template_url_args() { + let companions = companions_from_str!( + r#" + [[bootstrapping.containers]] + image = "busybox" + args = [ "echo", "Hello {{application.baseUrl}}" ] + "# + ); + + let container = &companions.bootstrapping.containers[0]; + + assert_eq!(container.image, Image::from_str("busybox").unwrap()); + assert_eq!( + container + .templated_args( + &AppName::master(), + &Some(Url::parse("http://example.com").unwrap()) + ) + .unwrap(), + vec![ + String::from("echo"), + String::from("Hello http://example.com/") + ] + ); + } } diff --git a/api/src/config/mod.rs b/api/src/config/mod.rs index beb054a5..98cfadfd 100644 --- a/api/src/config/mod.rs +++ b/api/src/config/mod.rs @@ -24,11 +24,13 @@ * =========================LICENSE_END================================== */ +pub use self::companion::BootstrappingContainer; pub use self::companion::DeploymentStrategy; pub use self::companion::StorageStrategy; -use self::companion::{Companion, CompanionType}; +use self::companion::{Companion, CompanionType, Companions}; pub use self::container::ContainerConfig; pub use self::runtime::Runtime; +use crate::models::AppName; use crate::models::ServiceConfig; pub(self) use app_selector::AppSelector; use clap::Parser; @@ -146,7 +148,8 @@ pub struct Config { runtime: Runtime, containers: Option, jira: Option, - companions: Option>, + #[serde(default)] + companions: Companions, services: Option>, hooks: Option>, #[serde(default)] @@ -189,7 +192,7 @@ impl Config { pub fn service_companion_configs( &self, - app_name: &str, + app_name: &AppName, ) -> Vec<(ServiceConfig, DeploymentStrategy, StorageStrategy)> { self.companion_configs(app_name, |companion| { companion.companion_type() == &CompanionType::Service @@ -198,39 +201,29 @@ impl Config { pub fn application_companion_configs( &self, - app_name: &str, + app_name: &AppName, ) -> Vec<(ServiceConfig, DeploymentStrategy, StorageStrategy)> { self.companion_configs(app_name, |companion| { companion.companion_type() == &CompanionType::Application }) } + pub fn companion_bootstrapping_containers(&self) -> &Vec { + self.companions.companion_bootstrapping_containers() + } + fn companion_configs

( &self, - app_name: &str, + app_name: &AppName, predicate: P, ) -> Vec<(ServiceConfig, DeploymentStrategy, StorageStrategy)> where P: Fn(&Companion) -> bool, { - match &self.companions { - None => vec![], - Some(companions_map) => companions_map - .iter() - .filter(|(_, companion)| companion.matches_app_name(app_name)) - .filter(|(_, companion)| predicate(companion)) - .map(|(_, companion)| { - ( - companion.clone().into(), - companion.deployment_strategy().clone(), - companion.storage_strategy().clone(), - ) - }) - .collect(), - } + self.companions.companion_configs(app_name, predicate) } - pub fn add_secrets_to(&self, service_config: &mut ServiceConfig, app_name: &str) { + pub fn add_secrets_to(&self, service_config: &mut ServiceConfig, app_name: &AppName) { if let Some(services) = &self.services { if let Some(service) = services.get(service_config.service_name()) { service.add_secrets_to(service_config, app_name); @@ -262,7 +255,7 @@ impl JiraConfig { } impl Service { - pub fn add_secrets_to(&self, service_config: &mut ServiceConfig, app_name: &str) { + pub fn add_secrets_to(&self, service_config: &mut ServiceConfig, app_name: &AppName) { if let Some(secrets) = &self.secrets { for s in secrets.iter().filter(|s| s.matches_app_name(app_name)) { let (path, sec) = s.clone().into(); @@ -340,7 +333,7 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("master"); + let companion_configs = config.application_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(config, _, _)| { @@ -375,7 +368,7 @@ mod tests { "# ); - let companion_configs = config.service_companion_configs("master"); + let companion_configs = config.service_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(config, _, _)| { @@ -401,7 +394,7 @@ mod tests { "# ); - let companion_configs = config.service_companion_configs("master"); + let companion_configs = config.service_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(_, strategy, _)| { @@ -424,7 +417,7 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("master"); + let companion_configs = config.application_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(config, _, _)| { @@ -446,7 +439,7 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("master"); + let companion_configs = config.application_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(config, _, _)| { @@ -470,7 +463,7 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("master"); + let companion_configs = config.application_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(config, _, _)| { @@ -500,7 +493,8 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("random-name"); + let companion_configs = + config.application_companion_configs(&AppName::from_str("random-name").unwrap()); assert_eq!(companion_configs.len(), 0); } @@ -517,7 +511,7 @@ mod tests { ); let mut service_config = service_config!("mariadb"); - config.add_secrets_to(&mut service_config, &String::from("master")); + config.add_secrets_to(&mut service_config, &AppName::master()); let secret_file_content = service_config .files() .expect("File content is missing") @@ -539,7 +533,7 @@ mod tests { ); let mut service_config = service_config!("mariadb"); - config.add_secrets_to(&mut service_config, &String::from("master")); + config.add_secrets_to(&mut service_config, &AppName::master()); let secret_file_content = service_config .files() @@ -562,7 +556,10 @@ mod tests { ); let mut service_config = service_config!("mariadb"); - config.add_secrets_to(&mut service_config, &String::from("master-1.x")); + config.add_secrets_to( + &mut service_config, + &AppName::from_str("master-1.x").unwrap(), + ); let secret_file_content = service_config .files() @@ -585,7 +582,10 @@ mod tests { ); let mut service_config = service_config!("mariadb"); - config.add_secrets_to(&mut service_config, &String::from("random-app-name")); + config.add_secrets_to( + &mut service_config, + &AppName::from_str("random-app-name").unwrap(), + ); assert!(service_config.files().is_none()); } @@ -603,7 +603,10 @@ mod tests { ); let mut service_config = service_config!("mariadb"); - config.add_secrets_to(&mut service_config, &String::from("master-1.x")); + config.add_secrets_to( + &mut service_config, + &AppName::from_str("master-1.x").unwrap(), + ); assert_eq!(service_config.files(), None); } @@ -668,7 +671,7 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("master"); + let companion_configs = config.application_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs.iter().for_each(|(config, _, _)| { @@ -689,7 +692,7 @@ mod tests { "# ); - let companion_configs = config.application_companion_configs("master"); + let companion_configs = config.application_companion_configs(&AppName::master()); assert_eq!(companion_configs.len(), 1); companion_configs diff --git a/api/src/config/secret.rs b/api/src/config/secret.rs index a573a60d..ac46155d 100644 --- a/api/src/config/secret.rs +++ b/api/src/config/secret.rs @@ -23,7 +23,7 @@ * THE SOFTWARE. * =========================LICENSE_END================================== */ -use crate::config::AppSelector; +use crate::{config::AppSelector, models::AppName}; use base64::{engine::general_purpose, Engine}; use secstr::SecUtf8; use serde::{de, Deserialize, Deserializer}; @@ -53,7 +53,7 @@ impl Secret { Ok(SecUtf8::from(sec_value)) } - pub fn matches_app_name(&self, app_name: &str) -> bool { + pub fn matches_app_name(&self, app_name: &AppName) -> bool { self.app_selector.matches(app_name) } } diff --git a/api/src/deployment/deployment_unit.rs b/api/src/deployment/deployment_unit.rs index 8c21e183..05e6307d 100644 --- a/api/src/deployment/deployment_unit.rs +++ b/api/src/deployment/deployment_unit.rs @@ -97,6 +97,7 @@ pub struct WithAppliedHooks { pub struct WithAppliedIngressRoute { app_name: AppName, services: Vec, + route: TraefikIngressRoute, } pub struct DeploymentUnitBuilder { @@ -106,6 +107,7 @@ pub struct DeploymentUnitBuilder { pub struct DeploymentUnit { app_name: AppName, services: Vec, + route: TraefikIngressRoute, } #[derive(Clone, Debug)] @@ -174,6 +176,10 @@ impl DeploymentUnit { pub fn app_name(&self) -> &AppName { &self.app_name } + + pub fn app_base_route(&self) -> &TraefikIngressRoute { + &self.route + } } impl DeploymentUnitBuilder { @@ -347,7 +353,7 @@ impl DeploymentUnitBuilder { self.stage.service_companions.iter() { let templated_companion = service_companion - .apply_templating_for_service_companion(&self.stage.app_name, &service)?; + .apply_templating_for_service_companion(&self.stage.app_name, service)?; service_companions.push(ServiceCompanion { templated_companion, @@ -548,18 +554,26 @@ impl DeploymentUnitBuilder { service.ingress_route.merge_with(service_route); } + let mut route = route; + route.merge_with(TraefikIngressRoute::with_app_only_defaults( + &self.stage.app_name, + )); + DeploymentUnitBuilder { stage: WithAppliedIngressRoute { app_name: self.stage.app_name, services: self.stage.services, + route, }, } } pub fn build(self) -> DeploymentUnit { + let route = TraefikIngressRoute::with_app_only_defaults(&self.stage.app_name); DeploymentUnit { app_name: self.stage.app_name, services: self.stage.services, + route, } } } @@ -569,6 +583,7 @@ impl DeploymentUnitBuilder { DeploymentUnit { app_name: self.stage.app_name, services: self.stage.services, + route: self.stage.route, } } } @@ -580,13 +595,12 @@ mod tests { use crate::models::{Environment, EnvironmentVariable}; use crate::{config_from_str, sc}; use secstr::SecUtf8; - use std::str::FromStr; #[tokio::test] async fn should_return_unique_images() -> Result<(), AppsServiceError> { let config = Config::default(); let unit = DeploymentUnitBuilder::init( - AppName::from_str("master").unwrap(), + AppName::master(), vec![ sc!("http1", "nginx:1.13"), sc!("wordpress1", "wordpress:alpine"), @@ -617,7 +631,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!("http1", "nginx:1.13")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) @@ -650,7 +664,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!( "openid", labels = (), @@ -705,7 +719,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!( "openid", labels = (), @@ -754,7 +768,7 @@ mod tests { image = 'postgres:11' "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![ sc!("wordpress", "wordpress:alpine"), sc!("nextcloud", "nextcloud:alpine"), @@ -802,7 +816,7 @@ mod tests { ]))); let config = Config::default(); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let unit = DeploymentUnitBuilder::init(app_name, vec![service_config]) .extend_with_config(&config) @@ -836,7 +850,7 @@ mod tests { SecUtf8::from("{{application.name}}"), )]))); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let config = Config::default(); let unit = DeploymentUnitBuilder::init(app_name, vec![service_configs]) @@ -875,7 +889,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!("wordpress", "wordpress:latest")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) @@ -929,7 +943,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!("wordpress", "wordpress:alpine")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) @@ -983,7 +997,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!("openid", "private.example.com/library/openid:backup")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) @@ -1032,7 +1046,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![ sc!("wordpress", "wordpress:alpine"), sc!("wordpress-db", "postgres:11-alpine"), @@ -1081,7 +1095,7 @@ mod tests { #[tokio::test] async fn should_determine_deployment_strategy_for_requested_service( ) -> Result<(), AppsServiceError> { - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![ sc!("wordpress", "wordpress:alpine"), sc!("wordpress-db", "postgres:11-alpine"), @@ -1131,7 +1145,7 @@ mod tests { "# ); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!("wordpress", "wordpress:alpine")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) @@ -1161,7 +1175,7 @@ mod tests { async fn apply_base_traefik_router_rule() -> Result<(), AppsServiceError> { let config = config_from_str!(""); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![sc!("wordpress")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) diff --git a/api/src/deployment/hooks.rs b/api/src/deployment/hooks.rs index 17e6bb10..9cddd854 100644 --- a/api/src/deployment/hooks.rs +++ b/api/src/deployment/hooks.rs @@ -112,10 +112,7 @@ impl<'a> Hooks<'a> { } } - fn register_configs_as_global_property( - mut context: &mut Context, - services: &[DeployableService], - ) { + fn register_configs_as_global_property(context: &mut Context, services: &[DeployableService]) { let js_configs = services .iter() .map(JsServiceConfig::from) @@ -123,7 +120,7 @@ impl<'a> Hooks<'a> { let js_configs = serde_json::to_value(js_configs).expect("Should be serializable"); let js_configs = - JsValue::from_json(&js_configs, &mut context).expect("Unable to read JSON value"); + JsValue::from_json(&js_configs, context).expect("Unable to read JSON value"); context .register_global_property("serviceConfigs", js_configs, Attribute::READONLY) @@ -236,7 +233,6 @@ mod tests { use crate::deployment::deployment_unit::DeploymentUnitBuilder; use std::collections::HashMap; use std::io::Write; - use std::str::FromStr; use std::vec; use tempfile::NamedTempFile; @@ -269,7 +265,7 @@ mod tests { let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_configs = vec![crate::sc!("service-a")]; let unit = DeploymentUnitBuilder::init(app_name, service_configs) @@ -314,7 +310,7 @@ mod tests { let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let mut service_config = crate::sc!("service-a"); let mut files = BTreeMap::new(); @@ -359,7 +355,7 @@ mod tests { "#; let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let service_config = crate::sc!("service-a"); let unit = DeploymentUnitBuilder::init(app_name, vec![service_config]) @@ -401,7 +397,7 @@ mod tests { "#; let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let mut service_config = crate::sc!("service-a"); service_config.set_env(Some(Environment::new(vec![EnvironmentVariable::new( @@ -452,7 +448,7 @@ mod tests { let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let mut service_config = crate::sc!("service-a"); service_config.set_env(Some(Environment::new(vec![EnvironmentVariable::new( String::from("VARIABLE_X"), @@ -497,7 +493,7 @@ mod tests { "#; let service_config = crate::sc!("service-a"); let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let unit = DeploymentUnitBuilder::init(app_name, vec![service_config]) .extend_with_config(&config) @@ -533,7 +529,7 @@ mod tests { let service_config = crate::sc!("service-a"); let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); let unit = DeploymentUnitBuilder::init(app_name, vec![service_config]) .extend_with_config(&config) @@ -561,7 +557,7 @@ mod tests { let mut deployed_services_error = String::new(); let service_config = crate::sc!("service-a"); let (_temp_js_file, config) = config_with_deployment_hook(script); - let app_name = AppName::from_str("master").unwrap(); + let app_name = AppName::master(); match DeploymentUnitBuilder::init(app_name, vec![service_config]) .extend_with_config(&config) .extend_with_templating_only_service_configs(Vec::new()) diff --git a/api/src/infrastructure/dummy_infrastructure.rs b/api/src/infrastructure/dummy_infrastructure.rs index 9c089e48..8b6abc55 100644 --- a/api/src/infrastructure/dummy_infrastructure.rs +++ b/api/src/infrastructure/dummy_infrastructure.rs @@ -34,6 +34,7 @@ use async_trait::async_trait; use chrono::{DateTime, FixedOffset, Utc}; use multimap::MultiMap; use std::collections::HashSet; +use std::str::FromStr; use std::sync::Mutex; use std::time::Duration; @@ -113,7 +114,7 @@ impl Infrastructure for DummyInfrastructure { .build() .unwrap(); - s.insert(app.clone(), service); + s.insert(AppName::from_str(app).unwrap(), service); } } @@ -155,7 +156,8 @@ impl Infrastructure for DummyInfrastructure { self.delay_if_configured().await; let mut services = self.services.lock().unwrap(); - match services.remove(app_name) { + + match services.remove(&app_name) { Some(services) => Ok(services .into_iter() .map(|sc| { diff --git a/api/src/infrastructure/kubernetes/deployment_unit.rs b/api/src/infrastructure/kubernetes/deployment_unit.rs new file mode 100644 index 00000000..b3d87d03 --- /dev/null +++ b/api/src/infrastructure/kubernetes/deployment_unit.rs @@ -0,0 +1,1195 @@ +use super::{ + infrastructure::KubernetesInfrastructureError, + payloads::{ + convert_k8s_ingress_to_traefik_ingress, IngressRoute as TraefikIngressRoute, + Middleware as TraefikMiddleware, + }, +}; +use crate::{ + config::BootstrappingContainer, + deployment::DeploymentUnit, + infrastructure::{APP_NAME_LABEL, CONTAINER_TYPE_LABEL, SERVICE_NAME_LABEL}, + models::{AppName, ContainerType, Image}, +}; +use failure::Error; +use futures::{AsyncBufReadExt, AsyncReadExt, StreamExt, TryStreamExt}; +use handlebars::RenderError; +use k8s_openapi::{ + api::{ + apps::v1::{Deployment, StatefulSet}, + batch::v1::Job, + core::v1::{ + ConfigMap, Container, LocalObjectReference, PersistentVolumeClaim, Pod, PodSpec, + Secret, Service, ServiceAccount, + }, + networking::v1::Ingress, + rbac::v1::{Role, RoleBinding}, + }, + apimachinery::pkg::apis::meta::v1::LabelSelector, + DeepMerge, Metadata, Resource, +}; +use kube::{ + api::{LogParams, Patch, PatchParams, PostParams, WatchParams}, + core::{DynamicObject, ObjectMeta, WatchEvent}, + Api, Client, ResourceExt, +}; +use serde::Deserialize; +use std::{ + collections::{BTreeMap, HashSet}, + str::FromStr, +}; +use url::Url; + +#[derive(Default)] +pub(super) struct K8sDeploymentUnit { + roles: Vec, + role_bindings: Vec, + stateful_sets: Vec, + config_maps: Vec, + secrets: Vec, + pvcs: Vec, + services: Vec, + pods: Vec, + deployments: Vec, + jobs: Vec, + service_accounts: Vec, + traefik_ingresses: Vec, + traefik_middlewares: Vec, +} + +impl K8sDeploymentUnit { + async fn start_bootstrapping_pods( + app_name: &AppName, + client: Client, + bootstrapping_containers: &[BootstrappingContainer], + image_pull_secret: Option, + base_url: Option, + ) -> Result<(String, Vec), Error> { + let image_pull_secrets = match image_pull_secret { + Some(image_pull_secret) => { + let image_pull_secrets = vec![LocalObjectReference { + name: Some(image_pull_secret.metadata.name.clone().unwrap_or_default()), + }]; + create_or_patch(client.clone(), app_name, image_pull_secret).await?; + Some(image_pull_secrets) + } + None => None, + }; + + let containers = bootstrapping_containers + .iter() + .enumerate() + .map(|(i, bc)| { + Ok(Container { + name: format!("bootstrap-{i}"), + image: Some(bc.image().to_string()), + image_pull_policy: Some(String::from("Always")), + args: Some(bc.templated_args(app_name, &base_url)?), + ..Default::default() + }) + }) + .collect::, RenderError>>()?; + + let pod_name = format!( + "{}-bootstrap-{}", + app_name.to_rfc1123_namespace_id(), + uuid::Uuid::new_v4() + ); + + let pod = Pod { + metadata: ObjectMeta { + name: Some(pod_name.clone()), + labels: Some(BTreeMap::from([( + APP_NAME_LABEL.to_string(), + app_name.to_string(), + )])), + ..Default::default() + }, + spec: Some(PodSpec { + containers, + image_pull_secrets, + restart_policy: Some(String::from("Never")), + ..Default::default() + }), + ..Default::default() + }; + create_or_patch(client.clone(), app_name, pod).await?; + + let api: Api = Api::namespaced(client, &app_name.to_rfc1123_namespace_id()); + + // Wait for a bookmark event to be sure that the log is ready to be consumed + let wp = WatchParams::default() + .fields(&format!("metadata.name={pod_name}")) + .timeout(10); + let mut stream = api.watch(&wp, "0").await?.boxed(); + while let Some(status) = stream.try_next().await? { + trace!("Saw watch event for bootstrapping pod {pod_name} in {app_name}: {status:?}"); + + if let WatchEvent::Bookmark(_bookmark) = status { + debug!("Boot strapping pod {pod_name} for {app_name} ready."); + break; + } + } + + loop { + let pod = api.get_status(&pod_name).await?; + + if let Some(phase) = pod.status.and_then(|status| status.phase) { + match phase.as_str() { + "Running" | "Succeeded" => { + break; + } + "Failed" | "Unknown" => { + return Err(KubernetesInfrastructureError::UnexpectedError { + internal_message: format!( + "Bootstrap pod {pod_name} for {app_name} failed" + ), + } + .into()); + } + phase => { + trace!("Boot strapping pod {pod_name} for {app_name} still not in running phase. Currently in {phase}."); + } + } + } + + tokio::time::sleep(tokio::time::Duration::from_secs(5)).await; + } + + let mut log_streams = Vec::with_capacity(bootstrapping_containers.len()); + + for i in 0..bootstrapping_containers.len() { + log_streams.push( + api.log_stream( + &pod_name, + &LogParams { + container: Some(format!("bootstrap-{i}")), + follow: true, + ..Default::default() + }, + ) + .await?, + ); + } + + Ok((pod_name, log_streams)) + } + + pub(super) async fn bootstrap( + deployment_unit: &DeploymentUnit, + client: Client, + bootstrapping_container: &[BootstrappingContainer], + image_pull_secret: Option, + ) -> Result { + if bootstrapping_container.is_empty() { + return Ok(Default::default()); + } + + let app_name = deployment_unit.app_name(); + + let (bootstrapping_pod_name, mut log_streams) = Self::start_bootstrapping_pods( + app_name, + client.clone(), + bootstrapping_container, + image_pull_secret, + deployment_unit.app_base_route().to_url(), + ) + .await?; + + let result = Self::parse_from_log_streams(deployment_unit, &mut log_streams).await; + + let pod_api: Api = Api::namespaced(client, &app_name.to_rfc1123_namespace_id()); + pod_api + .delete(&bootstrapping_pod_name, &Default::default()) + .await?; + + result + } + + async fn parse_from_log_streams( + deployment_unit: &DeploymentUnit, + log_streams: L, + ) -> Result + where + L: IntoIterator, + ::Item: AsyncBufReadExt, + ::Item: Unpin, + { + let app_name = deployment_unit.app_name(); + let mut roles = Vec::new(); + let mut role_bindings = Vec::new(); + let mut stateful_sets = Vec::new(); + let mut config_maps = Vec::new(); + let mut secrets = Vec::new(); + let mut pvcs = Vec::new(); + let mut services = Vec::new(); + let mut pods = Vec::new(); + let mut deployments = Vec::new(); + let mut jobs = Vec::new(); + let mut service_accounts = Vec::new(); + let mut ingresses = Vec::new(); + + for mut log_stream in log_streams.into_iter() { + let mut stdout = String::new(); + log_stream.read_to_string(&mut stdout).await?; + + trace!( + "Received YAML from bootstrapping container in {app_name}: {}…", + stdout.lines().next().unwrap_or(&stdout) + ); + + for doc in serde_yaml::Deserializer::from_str(&stdout) { + match DynamicObject::deserialize(doc) { + Ok(mut dy) => { + dy.metadata.namespace = Some(app_name.to_rfc1123_namespace_id()); + dy.labels_mut() + .insert(APP_NAME_LABEL.to_string(), app_name.to_string()); + + let api_version = dy + .types + .as_ref() + .map(|t| t.api_version.as_str()) + .unwrap_or_default(); + let kind = dy + .types + .as_ref() + .map(|t| t.kind.as_str()) + .unwrap_or_default(); + + trace!( + "Parsed {} ({api_version}, {kind}) for {app_name} as a bootstrap application element.", + dy.metadata + .name + .as_deref() + .unwrap_or_default() + ); + + match (api_version, kind) { + (Role::API_VERSION, Role::KIND) => match dy.clone().try_parse::() + { + Ok(role) => { + roles.push(role); + } + Err(e) => { + error!("Cannot parse {:?} as Role: {e}", dy.metadata.name); + } + }, + + (RoleBinding::API_VERSION, RoleBinding::KIND) => { + match dy.clone().try_parse::() { + Ok(role_binding) => { + role_bindings.push(role_binding); + } + Err(e) => { + error!( + "Cannot parse {:?} as RoleBinding: {e}", + dy.metadata.name + ); + } + } + } + (StatefulSet::API_VERSION, StatefulSet::KIND) => { + match dy.clone().try_parse::() { + Ok(stateful_set) => { + stateful_sets.push(stateful_set); + } + Err(e) => { + error!( + "Cannot parse {:?} as StatefulSet: {e}", + dy.metadata.name + ); + } + } + } + (ConfigMap::API_VERSION, ConfigMap::KIND) => { + match dy.clone().try_parse::() { + Ok(config_map) => { + config_maps.push(config_map); + } + Err(e) => { + error!( + "Cannot parse {:?} as ConfigMap: {e}", + dy.metadata.name + ); + } + } + } + (Secret::API_VERSION, Secret::KIND) => { + if let serde_json::Value::Object(obj) = &mut dy.data { + obj.entry("data").and_modify(|obj| { + if let serde_json::Value::Object(obj) = obj { + for (_k, v) in obj.iter_mut() { + if let serde_json::Value::String(str) = v { + // replacing new lines here because it is assumed + // that the data is base64 encoded and thus there + // must be no new lines + *v = str.replace('\n', "").into(); + } + } + } + }); + } + + match dy.clone().try_parse::() { + Ok(secret) => { + secrets.push(secret); + } + Err(e) => { + error!( + "Cannot parse {:?} as Secret: {e}", + dy.metadata.name + ); + } + } + } + (PersistentVolumeClaim::API_VERSION, PersistentVolumeClaim::KIND) => { + match dy.clone().try_parse::() { + Ok(pvc) => { + pvcs.push(pvc); + } + Err(e) => { + error!( + "Cannot parse {:?} as PersistentVolumeClaim: {e}", + dy.metadata.name + ); + } + } + } + (Service::API_VERSION, Service::KIND) => { + match dy.clone().try_parse::() { + Ok(service) => { + services.push(service); + } + Err(e) => { + error!( + "Cannot parse {:?} as Service: {e}", + dy.metadata.name + ); + } + } + } + (Deployment::API_VERSION, Deployment::KIND) => { + match dy.clone().try_parse::() { + Ok(mut deployment) => { + let service_name = deployment + .labels() + .get("app.kubernetes.io/component") + .cloned() + .unwrap_or_else(|| { + deployment.metadata.name.clone().unwrap_or_default() + }); + + deployment + .labels_mut() + .insert(SERVICE_NAME_LABEL.to_string(), service_name); + deployment.labels_mut().insert( + CONTAINER_TYPE_LABEL.to_string(), + ContainerType::ApplicationCompanion.to_string(), + ); + + deployments.push(deployment); + } + Err(e) => { + error!( + "Cannot parse {:?} as Deployment: {e}", + dy.metadata.name + ); + } + } + } + (Pod::API_VERSION, Pod::KIND) => match dy.clone().try_parse::() { + Ok(pod) => { + pods.push(pod); + } + Err(e) => { + error!("Cannot parse {:?} as Pod: {e}", dy.metadata.name); + } + }, + (Job::API_VERSION, Job::KIND) => match dy.clone().try_parse::() { + Ok(job) => { + jobs.push(job); + } + Err(e) => { + error!("Cannot parse {:?} as Job: {e}", dy.metadata.name); + } + }, + (ServiceAccount::API_VERSION, ServiceAccount::KIND) => { + match dy.clone().try_parse::() { + Ok(service_account) => { + service_accounts.push(service_account); + } + Err(e) => { + error!( + "Cannot parse {:?} as ServiceAccount: {e}", + dy.metadata.name + ); + } + } + } + (Ingress::API_VERSION, Ingress::KIND) => { + match dy.clone().try_parse::() { + Ok(ingress) => { + ingresses.push(ingress); + } + Err(e) => { + error!( + "Cannot parse {:?} as Ingress: {e}", + dy.metadata.name + ); + } + } + } + _ => { + warn!( + "Cannot parse {name} ({api_version}, {kind}) for {app_name} because its kind is unknown", + name=dy.metadata.name.unwrap_or_default() + ); + } + } + } + Err(err) => { + warn!("The output of a bootstrap container for {app_name} could not be parsed: {stdout}"); + return Err(err.into()); + } + } + } + } + + let mut traefik_ingresses = Vec::new(); + let mut traefik_middlewares = Vec::new(); + + for ingress in ingresses { + let Ok((route, middlewares)) = convert_k8s_ingress_to_traefik_ingress( + ingress, + deployment_unit.app_base_route().clone(), + ) else { + continue; + }; + + traefik_ingresses.push(route); + traefik_middlewares.extend(middlewares); + } + + Ok(Self { + roles, + role_bindings, + stateful_sets, + config_maps, + secrets, + pvcs, + services, + pods, + deployments, + jobs, + service_accounts, + traefik_ingresses, + traefik_middlewares, + }) + } + + pub(super) fn merge( + &mut self, + secret: Option, + service: Service, + deployment: Deployment, + ingress: TraefikIngressRoute, + middlewares: Vec, + ) { + let mut deployment = deployment; + + let service_name = deployment + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get(SERVICE_NAME_LABEL)) + .expect("There must be label providing the service name"); + + let stateful_sets = self + .stateful_sets + .iter_mut() + .filter(|set| Some(service_name) == set.metadata().name.as_ref()) + .filter_map(|set| { + let spec = set.spec.as_mut()?; + Some(( + &mut set.metadata, + spec.template.metadata.as_mut(), + spec.template.spec.as_mut()?, + )) + }); + let deployments = self + .deployments + .iter_mut() + .filter(|set| Some(service_name) == set.metadata().name.as_ref()) + .filter_map(|deployment| { + let spec = deployment.spec.as_mut()?; + Some(( + &mut deployment.metadata, + spec.template.metadata.as_mut(), + spec.template.spec.as_mut()?, + )) + }); + let pods = self + .pods + .iter_mut() + .filter(|pod| Some(service_name) == pod.metadata().name.as_ref()) + .filter_map(|pod| Some((&mut pod.metadata, None, pod.spec.as_mut()?))); + + match stateful_sets.chain(deployments).chain(pods).next() { + Some((metadata, pod_meta, pod_spec)) => { + // Clean everything that might interfere with the original definitions of + // bootstrapped companion before calling merge_from down below. + deployment.metadata.name = None; + + metadata.merge_from(deployment.metadata); + + let mut deployment_spec = deployment + .spec + .expect("There should be a deployment spec created for the deployable service"); + + deployment_spec.selector = LabelSelector::default(); + + let template_to_be_merged = deployment_spec.template; + + if let Some(pod_meta) = pod_meta { + pod_meta.merge_from( + template_to_be_merged.metadata.expect( + "There should be a pod meta created for the deployable service", + ), + ); + } + + let mut pod_spec_to_be_merged = template_to_be_merged + .spec + .expect("There should be a pod spec created for the deployable service"); + pod_spec_to_be_merged.containers[0].name = pod_spec.containers[0].name.clone(); + pod_spec_to_be_merged.containers[0].ports = None; + + pod_spec.merge_from(pod_spec_to_be_merged); + + if let Some(secret) = secret { + self.secrets.push(secret); + } + // Ingress, Service, and Middlewares will be ignored because at this point it can + // be assumed that these configurations are covered by the Kubernetes objects that + // were used for bootstrapping the application. + } + None => { + self.secrets.extend(secret); + self.services.push(service); + self.deployments.push(deployment); + self.traefik_ingresses.push(ingress); + self.traefik_middlewares.extend(middlewares); + } + } + } + + /// This filters bootstrapped [Deployments](Deployment), [Stateful Sets](StatefulSet), or + /// [Pods](Pod) by the existing [services](Service) in already deployed application to avoid + /// that deployments of instances overwrite each other + pub(super) fn filter_by_instances_and_replicas( + &mut self, + services: &[crate::models::service::Service], + ) { + let service_not_to_be_retained = services + .iter() + .filter(|s| { + s.container_type() == &ContainerType::Instance + || s.container_type() == &ContainerType::Replica + }) + .map(|s| s.service_name()) + .collect::>(); + + self.deployments.retain(|deployment| { + let Some(service_name) = deployment + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get(SERVICE_NAME_LABEL)) + else { + return false; + }; + + !service_not_to_be_retained.contains(service_name) + }); + } + + fn images_of_pod_spec(spec: &PodSpec) -> HashSet { + let mut images = HashSet::new(); + + if let Some(init_containers) = &spec.init_containers { + for init_container in init_containers { + if let Some(image) = init_container + .image + .as_ref() + .and_then(|image| Image::from_str(image).ok()) + { + images.insert(image); + } + } + } + + for container in &spec.containers { + if let Some(image) = container + .image + .as_ref() + .and_then(|image| Image::from_str(image).ok()) + { + images.insert(image); + } + } + + images + } + + pub(super) fn images(&self) -> HashSet { + let mut images = HashSet::new(); + + for deployment in &self.deployments { + let Some(spec) = &deployment.spec else { + continue; + }; + let Some(spec) = &spec.template.spec else { + continue; + }; + + images.extend(Self::images_of_pod_spec(spec)); + } + for job in &self.jobs { + let Some(spec) = &job.spec else { + continue; + }; + let Some(spec) = &spec.template.spec else { + continue; + }; + + images.extend(Self::images_of_pod_spec(spec)); + } + for stateful_set in &self.stateful_sets { + let Some(spec) = &stateful_set.spec else { + continue; + }; + let Some(spec) = &spec.template.spec else { + continue; + }; + + images.extend(Self::images_of_pod_spec(spec)); + } + for pod in &self.pods { + let Some(spec) = &pod.spec else { + continue; + }; + + images.extend(Self::images_of_pod_spec(spec)); + } + + images + } + + pub(super) fn apply_image_pull_secret(&mut self, image_pull_secret: Secret) { + let pull_secret_reference = LocalObjectReference { + name: Some(image_pull_secret.metadata.name.clone().unwrap_or_default()), + }; + self.secrets.push(image_pull_secret); + + for deployment in self.deployments.iter_mut() { + let Some(spec) = &mut deployment.spec else { + continue; + }; + let Some(spec) = &mut spec.template.spec else { + continue; + }; + + spec.image_pull_secrets = Some(vec![pull_secret_reference.clone()]); + } + for job in self.jobs.iter_mut() { + let Some(spec) = &mut job.spec else { + continue; + }; + let Some(spec) = &mut spec.template.spec else { + continue; + }; + + spec.image_pull_secrets = Some(vec![pull_secret_reference.clone()]); + } + for stateful_set in self.stateful_sets.iter_mut() { + let Some(spec) = &mut stateful_set.spec else { + continue; + }; + let Some(spec) = &mut spec.template.spec else { + continue; + }; + + spec.image_pull_secrets = Some(vec![pull_secret_reference.clone()]); + } + for pod in self.pods.iter_mut() { + let Some(spec) = &mut pod.spec else { + continue; + }; + + spec.image_pull_secrets = Some(vec![pull_secret_reference.clone()]); + } + } + + pub(super) async fn deploy( + self, + client: Client, + app_name: &AppName, + ) -> Result, Error> { + let mut deployments = Vec::with_capacity(self.deployments.len()); + + for role in self.roles { + create_or_patch(client.clone(), app_name, role).await?; + } + for role_binding in self.role_bindings { + create_or_patch(client.clone(), app_name, role_binding).await?; + } + for config_map in self.config_maps { + create_or_patch(client.clone(), app_name, config_map).await?; + } + for secret in self.secrets { + create_or_patch(client.clone(), app_name, secret).await?; + } + for pvc in self.pvcs { + create_or_patch(client.clone(), app_name, pvc).await?; + } + for service in self.services { + create_or_patch(client.clone(), app_name, service).await?; + } + for service_account in self.service_accounts { + create_or_patch(client.clone(), app_name, service_account).await?; + } + for deployment in self.deployments { + let deployment = create_or_patch(client.clone(), app_name, deployment).await?; + deployments.push(deployment); + } + for job in self.jobs { + create_or_patch(client.clone(), app_name, job).await?; + } + for stateful_set in self.stateful_sets { + create_or_patch(client.clone(), app_name, stateful_set).await?; + } + for ingress in self.traefik_ingresses { + create_or_patch(client.clone(), app_name, ingress).await?; + } + for middleware in self.traefik_middlewares { + create_or_patch(client.clone(), app_name, middleware).await?; + } + for pod in self.pods { + create_or_patch(client.clone(), app_name, pod).await?; + } + + Ok(deployments) + } +} + +async fn create_or_patch(client: Client, app_name: &AppName, payload: T) -> Result +where + T: serde::Serialize + Clone + std::fmt::Debug + for<'a> serde::Deserialize<'a>, + T: kube::core::Resource, + ::DynamicType: std::default::Default, +{ + let api = Api::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()); + match api.create(&PostParams::default(), &payload).await { + Ok(result) => Ok(result), + Err(kube::error::Error::Api(kube::error::ErrorResponse { code, .. })) if code == 409 => { + let name = payload.meta().name.clone().unwrap_or_default(); + match api + .patch(&name, &PatchParams::default(), &Patch::Merge(&payload)) + .await + { + Ok(result) => Ok(result), + Err(_e) => { + // TODO: how to handle the case? e.g. patching a job may fails + Ok(payload) + } + } + } + Err(e) => Err(e.into()), + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::{deployment::deployment_unit::DeploymentUnitBuilder, models::ServiceBuilder}; + use k8s_openapi::api::{ + apps::v1::DeploymentSpec, + core::v1::{ContainerPort, EnvVar, PodTemplateSpec}, + }; + use std::collections::HashMap; + + async fn parse_unit(stdout: &'static str) -> K8sDeploymentUnit { + let log_streams = vec![stdout.as_bytes()]; + + let deployment_unit = DeploymentUnitBuilder::init(AppName::master(), Vec::new()) + .extend_with_config(&Default::default()) + .extend_with_templating_only_service_configs(Vec::new()) + .extend_with_image_infos(HashMap::new()) + .apply_templating() + .unwrap() + .apply_hooks(&Default::default()) + .await + .unwrap() + .apply_base_traefik_ingress_route( + crate::infrastructure::TraefikIngressRoute::with_app_only_defaults( + &AppName::master(), + ), + ) + .build(); + + K8sDeploymentUnit::parse_from_log_streams(&deployment_unit, log_streams) + .await + .unwrap() + } + + #[tokio::test] + async fn parse_unit_from_secret_stdout_where_value_is_base64_encoded() { + let unit = parse_unit( + r#" + apiVersion: v1 + kind: Secret + metadata: + name: secret-tls + type: kubernetes.io/tls + data: + # values are base64 encoded, which obscures them but does NOT provide + # any useful level of confidentiality + tls.crt: | + LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNVakNDQWJzQ0FnMytNQTBHQ1NxR1NJYjNE + UUVCQlFVQU1JR2JNUXN3Q1FZRFZRUUdFd0pLVURFT01Bd0cKQTFVRUNCTUZWRzlyZVc4eEVEQU9C + Z05WQkFjVEIwTm9kVzh0YTNVeEVUQVBCZ05WQkFvVENFWnlZVzVyTkVSRQpNUmd3RmdZRFZRUUxF + dzlYWldKRFpYSjBJRk4xY0hCdmNuUXhHREFXQmdOVkJBTVREMFp5WVc1ck5FUkVJRmRsCllpQkRR + VEVqTUNFR0NTcUdTSWIzRFFFSkFSWVVjM1Z3Y0c5eWRFQm1jbUZ1YXpSa1pDNWpiMjB3SGhjTk1U + TXcKTVRFeE1EUTFNVE01V2hjTk1UZ3dNVEV3TURRMU1UTTVXakJMTVFzd0NRWURWUVFHREFKS1VE + RVBNQTBHQTFVRQpDQXdHWEZSdmEzbHZNUkV3RHdZRFZRUUtEQWhHY21GdWF6UkVSREVZTUJZR0Ex + VUVBd3dQZDNkM0xtVjRZVzF3CmJHVXVZMjl0TUlHYU1BMEdDU3FHU0liM0RRRUJBUVVBQTRHSUFE + Q0JoQUo5WThFaUhmeHhNL25PbjJTbkkxWHgKRHdPdEJEVDFKRjBReTliMVlKanV2YjdjaTEwZjVN + Vm1UQllqMUZTVWZNOU1vejJDVVFZdW4yRFljV29IcFA4ZQpqSG1BUFVrNVd5cDJRN1ArMjh1bklI + QkphVGZlQ09PekZSUFY2MEdTWWUzNmFScG04L3dVVm16eGFLOGtCOWVaCmhPN3F1TjdtSWQxL2pW + cTNKODhDQXdFQUFUQU5CZ2txaGtpRzl3MEJBUVVGQUFPQmdRQU1meTQzeE15OHh3QTUKVjF2T2NS + OEtyNWNaSXdtbFhCUU8xeFEzazlxSGtyNFlUY1JxTVQ5WjVKTm1rWHYxK2VSaGcwTi9WMW5NUTRZ + RgpnWXcxbnlESnBnOTduZUV4VzQyeXVlMFlHSDYyV1hYUUhyOVNVREgrRlowVnQvRGZsdklVTWRj + UUFEZjM4aU9zCjlQbG1kb3YrcE0vNCs5a1h5aDhSUEkzZXZ6OS9NQT09Ci0tLS0tRU5EIENFUlRJ + RklDQVRFLS0tLS0K + # In this example, the key data is not a real PEM-encoded private key + tls.key: | + RXhhbXBsZSBkYXRhIGZvciB0aGUgVExTIGNydCBmaWVsZA== + "#, + ) + .await; + + assert_json_diff::assert_json_eq!( + unit.secrets, + serde_json::json!([{ + "apiVersion": "v1", + "kind": "Secret", + "metadata": { + "name": "secret-tls", + "namespace": "master", + "labels": { + APP_NAME_LABEL: "master" + } + }, + "type": "kubernetes.io/tls", + "data": { + "tls.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNVakNDQWJzQ0FnMytNQTBHQ1NxR1NJYjNEUUVCQlFVQU1JR2JNUXN3Q1FZRFZRUUdFd0pLVURFT01Bd0cKQTFVRUNCTUZWRzlyZVc4eEVEQU9CZ05WQkFjVEIwTm9kVzh0YTNVeEVUQVBCZ05WQkFvVENFWnlZVzVyTkVSRQpNUmd3RmdZRFZRUUxFdzlYWldKRFpYSjBJRk4xY0hCdmNuUXhHREFXQmdOVkJBTVREMFp5WVc1ck5FUkVJRmRsCllpQkRRVEVqTUNFR0NTcUdTSWIzRFFFSkFSWVVjM1Z3Y0c5eWRFQm1jbUZ1YXpSa1pDNWpiMjB3SGhjTk1UTXcKTVRFeE1EUTFNVE01V2hjTk1UZ3dNVEV3TURRMU1UTTVXakJMTVFzd0NRWURWUVFHREFKS1VERVBNQTBHQTFVRQpDQXdHWEZSdmEzbHZNUkV3RHdZRFZRUUtEQWhHY21GdWF6UkVSREVZTUJZR0ExVUVBd3dQZDNkM0xtVjRZVzF3CmJHVXVZMjl0TUlHYU1BMEdDU3FHU0liM0RRRUJBUVVBQTRHSUFEQ0JoQUo5WThFaUhmeHhNL25PbjJTbkkxWHgKRHdPdEJEVDFKRjBReTliMVlKanV2YjdjaTEwZjVNVm1UQllqMUZTVWZNOU1vejJDVVFZdW4yRFljV29IcFA4ZQpqSG1BUFVrNVd5cDJRN1ArMjh1bklIQkphVGZlQ09PekZSUFY2MEdTWWUzNmFScG04L3dVVm16eGFLOGtCOWVaCmhPN3F1TjdtSWQxL2pWcTNKODhDQXdFQUFUQU5CZ2txaGtpRzl3MEJBUVVGQUFPQmdRQU1meTQzeE15OHh3QTUKVjF2T2NSOEtyNWNaSXdtbFhCUU8xeFEzazlxSGtyNFlUY1JxTVQ5WjVKTm1rWHYxK2VSaGcwTi9WMW5NUTRZRgpnWXcxbnlESnBnOTduZUV4VzQyeXVlMFlHSDYyV1hYUUhyOVNVREgrRlowVnQvRGZsdklVTWRjUUFEZjM4aU9zCjlQbG1kb3YrcE0vNCs5a1h5aDhSUEkzZXZ6OS9NQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K", + "tls.key": "RXhhbXBsZSBkYXRhIGZvciB0aGUgVExTIGNydCBmaWVsZA==" + } + }]) + ) + } + + #[tokio::test] + async fn parse_unit_from_deployment_stdout() { + let unit = parse_unit( + r#" + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx-deployment + labels: + app: nginx + spec: + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + "#, + ) + .await; + + assert_json_diff::assert_json_eq!( + unit.deployments, + serde_json::json!([{ + "apiVersion": "apps/v1", + "kind": "Deployment", + "metadata": { + "name": "nginx-deployment", + "namespace": "master", + "labels": { + "app": "nginx", + APP_NAME_LABEL: "master", + SERVICE_NAME_LABEL: "nginx-deployment", + CONTAINER_TYPE_LABEL: "app-companion" + } + }, + "spec": { + "selector": { + "matchLabels": { + "app": "nginx" + } + }, + "template": { + "metadata": { + "labels": { + "app": "nginx" + } + }, + "spec": { + "containers": [{ + "name": "nginx", + "image": "nginx:1.14.2", + "ports": [{ + "containerPort": 80 + }] + }] + } + } + } + }]) + ) + } + + #[tokio::test] + async fn merge_deployment_into_bootstrapped_deployment() { + let mut unit = parse_unit( + r#" + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + labels: + app: nginx + spec: + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + "#, + ) + .await; + + unit.merge( + None, + Service { + ..Default::default() + }, + Deployment { + metadata: ObjectMeta { + name: Some(String::from("random-name")), + labels: Some(BTreeMap::from([ + (SERVICE_NAME_LABEL.to_string(), String::from("nginx")), + (CONTAINER_TYPE_LABEL.to_string(), String::from("instance")), + ])), + annotations: Some(BTreeMap::from([( + String::from("my-important-annotation"), + String::from("test data"), + )])), + ..Default::default() + }, + spec: Some(DeploymentSpec { + selector: LabelSelector { + match_labels: Some(BTreeMap::from([( + SERVICE_NAME_LABEL.to_string(), + String::from("random-name"), + )])), + ..Default::default() + }, + template: PodTemplateSpec { + metadata: Some(ObjectMeta { + annotations: Some(BTreeMap::from([( + String::from("date"), + String::from("2024-01-01"), + )])), + ..Default::default() + }), + spec: Some(PodSpec { + containers: vec![Container { + name: String::from("random-name"), + image: Some(String::from("nginx:1.29.0")), + env: Some(vec![EnvVar { + name: String::from("NGINX_HOST"), + value: Some(String::from("example.com")), + ..Default::default() + }]), + ports: Some(vec![ContainerPort { + container_port: 4711, + ..Default::default() + }]), + ..Default::default() + }], + ..Default::default() + }), + ..Default::default() + }, + ..Default::default() + }), + ..Default::default() + }, + TraefikIngressRoute { + metadata: Default::default(), + spec: Default::default(), + }, + Vec::new(), + ); + + assert!(unit.secrets.is_empty()); + assert!(unit.services.is_empty()); + assert!(unit.traefik_ingresses.is_empty()); + assert!(unit.traefik_middlewares.is_empty()); + assert_json_diff::assert_json_eq!( + unit.deployments, + serde_json::json!([{ + "apiVersion": "apps/v1", + "kind": "Deployment", + "metadata": { + "name": "nginx", + "namespace": "master", + "labels": { + "app": "nginx", + APP_NAME_LABEL: "master", + SERVICE_NAME_LABEL: "nginx", + CONTAINER_TYPE_LABEL: "instance" + }, + "annotations": { + "my-important-annotation": "test data" + } + }, + "spec": { + "selector": { + "matchLabels": { + "app": "nginx" + } + }, + "template": { + "metadata": { + "labels": { + "app": "nginx" + }, + "annotations": { + "date": "2024-01-01" + } + }, + "spec": { + "containers": [{ + "name": "nginx", + "image": "nginx:1.29.0", + "env": [{ + "name": "NGINX_HOST", + "value": "example.com" + }], + "ports": [{ + "containerPort": 80 + }] + }] + } + } + } + }]) + ) + } + + #[tokio::test] + async fn filter_by_instances_and_replicas() { + let mut unit = parse_unit( + r#" + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + labels: + app: nginx + spec: + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + "#, + ) + .await; + + unit.filter_by_instances_and_replicas(dbg!(&[ServiceBuilder::new() + .app_name(AppName::master().to_string()) + .id(String::from("test")) + .config(crate::sc!("nginx", "nginx:1.15")) + .build() + .unwrap()])); + + assert!(unit.deployments.is_empty()); + } + + #[tokio::test] + async fn filter_not_by_instances_and_replicas() { + let mut unit = parse_unit( + r#" + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + labels: + app: nginx + spec: + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + "#, + ) + .await; + + unit.filter_by_instances_and_replicas(dbg!(&[ServiceBuilder::new() + .app_name(AppName::master().to_string()) + .id(String::from("test")) + .config(crate::sc!("postgres", "postgres")) + .build() + .unwrap()])); + + assert!(!unit.deployments.is_empty()); + } +} diff --git a/api/src/infrastructure/kubernetes/infrastructure.rs b/api/src/infrastructure/kubernetes/infrastructure.rs index 56bca54c..503bacea 100644 --- a/api/src/infrastructure/kubernetes/infrastructure.rs +++ b/api/src/infrastructure/kubernetes/infrastructure.rs @@ -27,10 +27,11 @@ use super::super::{ APP_NAME_LABEL, CONTAINER_TYPE_LABEL, IMAGE_LABEL, REPLICATED_ENV_LABEL, SERVICE_NAME_LABEL, STORAGE_TYPE_LABEL, }; +use super::deployment_unit::K8sDeploymentUnit; use super::payloads::{ - deployment_payload, deployment_replicas_payload, image_pull_secret_payload, - ingress_route_payload, middleware_payload, namespace_payload, persistent_volume_claim_payload, - secrets_payload, service_payload, IngressRoute, + deployment_payload, image_pull_secret_payload, ingress_route_payload, middleware_payload, + namespace_payload, persistent_volume_claim_payload, secrets_payload, service_payload, + IngressRoute, Middleware, }; use crate::config::{Config as PREvantConfig, ContainerConfig, Runtime}; use crate::deployment::deployment_unit::{DeployableService, DeploymentUnit}; @@ -43,12 +44,13 @@ use crate::models::{ use async_trait::async_trait; use chrono::{DateTime, FixedOffset, Utc}; use failure::Error; -use futures::future::join_all; +use futures::stream::FuturesUnordered; +use futures::StreamExt; +use k8s_openapi::api::core::v1::PersistentVolumeClaim; use k8s_openapi::api::storage::v1::StorageClass; use k8s_openapi::api::{ apps::v1::Deployment as V1Deployment, core::v1::Namespace as V1Namespace, - core::v1::PersistentVolumeClaim, core::v1::Pod as V1Pod, core::v1::Secret as V1Secret, - core::v1::Service as V1Service, + core::v1::Pod as V1Pod, core::v1::Secret as V1Secret, core::v1::Service as V1Service, }; use kube::{ api::{Api, DeleteParams, ListParams, LogParams, Patch, PatchParams, PostParams}, @@ -62,7 +64,6 @@ use secstr::SecUtf8; use std::collections::{BTreeMap, HashMap}; use std::convert::{From, TryFrom}; use std::net::IpAddr; -use std::path::PathBuf; use std::str::FromStr; pub struct KubernetesInfrastructure { @@ -76,11 +77,6 @@ pub enum KubernetesInfrastructureError { internal_message )] UnexpectedError { internal_message: String }, - #[fail( - display = "The deployment {} does not provide a label for service name.", - deployment_name - )] - MissingServiceNameLabel { deployment_name: String }, #[fail( display = "The deployment {} does not provide a label for app name.", deployment_name @@ -118,34 +114,57 @@ impl KubernetesInfrastructure { }) } - async fn create_service_from( + async fn get_deployment_and_pod( &self, - deployment: V1Deployment, - ) -> Result { - let namespace = deployment.metadata.namespace.clone().unwrap_or_default(); - let mut builder = ServiceBuilder::try_from(deployment)?; + app_name: &AppName, + service_name: &str, + ) -> Result)>, KubernetesInfrastructureError> { + let client = self.client().await?; + let namespace = app_name.to_rfc1123_namespace_id(); let p = ListParams { - label_selector: Some(format!( - "{}={},{}={}", - APP_NAME_LABEL, - builder - .current_app_name() - .map_or_else(|| "", |name| name.as_str()), - SERVICE_NAME_LABEL, - builder - .current_config() - .map_or_else(|| "", |config| config.service_name()), - )), + label_selector: Some(format!("{SERVICE_NAME_LABEL}={service_name}",)), ..Default::default() }; - if let Some(pod) = Api::::namespaced(self.client().await?, &namespace) - .list(&p) - .await? - .items - .into_iter() - .next() - { + + let client_clone = client.clone(); + let deployment = async { + Api::::namespaced(client_clone, &namespace) + .list(&p) + .await + .map(|list| list.items.into_iter().next()) + }; + let pods = async { + Api::::namespaced(client, &namespace) + .list(&Default::default()) + .await + .map(|list| list.items) + }; + + let (deployment, pods) = futures::try_join!(deployment, pods)?; + + Ok(deployment.and_then(|deployment| { + let spec = deployment.spec.as_ref()?; + let matches_labels = spec.selector.match_labels.as_ref()?; + let pod = pods.into_iter().find(|pod| { + pod.metadata + .labels + .as_ref() + .map(|labels| matches_labels.iter().all(|(k, v)| labels.get(k) == Some(v))) + .unwrap_or(false) + }); + + Some((deployment, pod)) + })) + } + + fn create_service_from_deployment_and_pod( + deployment: V1Deployment, + pod: Option, + ) -> Result { + let mut builder = ServiceBuilder::try_from(deployment.clone())?; + + if let Some(pod) = pod { if let Some(container) = pod.spec.as_ref().and_then(|spec| spec.containers.first()) { builder = builder.started_at( pod.status @@ -179,20 +198,49 @@ impl KubernetesInfrastructure { &self, app_name: &AppName, ) -> Result, KubernetesInfrastructureError> { - let mut services = Vec::new(); - let futures = Api::::namespaced( - self.client().await?, - &app_name.to_rfc1123_namespace_id(), - ) - .list(&Default::default()) - .await? - .items - .into_iter() - .map(|deployment| self.create_service_from(deployment)) - .collect::>(); - - for create_service_result in join_all(futures).await { - let service = match create_service_result { + let client = self.client().await?; + + let namespace = app_name.to_rfc1123_namespace_id(); + let list_param = Default::default(); + let client_clone = client.clone(); + let deployments = async { + Api::::namespaced(client_clone, &namespace) + .list(&list_param) + .await + }; + let pods = async { + Api::::namespaced(client, &namespace) + .list(&list_param) + .await + }; + let (deployments, mut pods) = futures::try_join!(deployments, pods)?; + + let mut services = Vec::with_capacity(deployments.items.len()); + for deployment in deployments.into_iter() { + let pod = { + let Some(spec) = deployment.spec.as_ref() else { + continue; + }; + let Some(matches_labels) = spec.selector.match_labels.as_ref() else { + continue; + }; + + match pods.items.iter().position(|pod| { + pod.metadata + .labels + .as_ref() + .map(|labels| matches_labels.iter().all(|(k, v)| labels.get(k) == Some(v))) + .unwrap_or(false) + }) { + Some(pod_position) => { + let pod = pods.items.swap_remove(pod_position); + Some(pod) + } + None => None, + } + }; + + let service = match Self::create_service_from_deployment_and_pod(deployment, pod) { Ok(service) => service, Err(e) => { debug!("Deployment does not provide required data: {:?}", e); @@ -206,59 +254,6 @@ impl KubernetesInfrastructure { Ok(services) } - async fn get_service_of_app( - &self, - app_name: &AppName, - service_name: &str, - ) -> Result, KubernetesInfrastructureError> { - let p = ListParams { - label_selector: Some(format!("{SERVICE_NAME_LABEL}={service_name}")), - ..Default::default() - }; - - match Api::::namespaced( - self.client().await?, - &app_name.to_rfc1123_namespace_id(), - ) - .list(&p) - .await? - .items - .into_iter() - .next() - .map(|deployment| self.create_service_from(deployment)) - { - None => Ok(None), - Some(service) => Ok(Some(service.await?)), - } - } - - async fn post_service_and_custom_resource_definitions( - &self, - app_name: &AppName, - service: &DeployableService, - ) -> Result<(), KubernetesInfrastructureError> { - let client = self.client().await?; - - Api::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()) - .create(&PostParams::default(), &service_payload(app_name, service)) - .await?; - - Api::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()) - .create( - &PostParams::default(), - &ingress_route_payload(app_name, service), - ) - .await?; - - for middleware in middleware_payload(app_name, service) { - Api::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()) - .create(&PostParams::default(), &middleware) - .await?; - } - - Ok(()) - } - async fn create_namespace_if_necessary( &self, app_name: &AppName, @@ -291,15 +286,13 @@ impl KubernetesInfrastructure { } } - async fn create_pull_secrets_if_necessary( - &self, - app_name: &AppName, - service: &[DeployableService], - ) -> Result<(), KubernetesInfrastructureError> { - let registries_and_credentials: BTreeMap = service - .iter() - .filter_map(|strategy| { - strategy.image().registry().and_then(|registry| { + fn image_pull_secret<'a, I>(&self, app_name: &AppName, images: I) -> Option + where + I: Iterator, + { + let registries_and_credentials: BTreeMap = images + .filter_map(|image| { + image.registry().and_then(|registry| { self.config .registry_credentials(®istry) .map(|(username, password)| (registry, (username, password))) @@ -308,163 +301,49 @@ impl KubernetesInfrastructure { .collect(); if registries_and_credentials.is_empty() { - return Ok(()); + return None; } - match Api::namespaced(self.client().await?, &app_name.to_rfc1123_namespace_id()) - .create( - &PostParams::default(), - &image_pull_secret_payload(app_name, registries_and_credentials), - ) - .await - { - Ok(result) => { - debug!( - "Successfully created image pull secret {}", - result - .metadata - .name - .unwrap_or_else(|| String::from("")) - ); - Ok(()) - } - Err(KubeError::Api(ErrorResponse { code, .. })) if code == 409 => { - debug!("Secrets already exists for {}", app_name); - Ok(()) - } - Err(e) => { - error!("Cannot deploy namespace: {}", e); - Err(e.into()) - } - } + Some(image_pull_secret_payload( + app_name, + registries_and_credentials, + )) } - async fn deploy_service<'a>( + async fn create_payloads( &self, app_name: &AppName, - service: &'a DeployableService, + deployable_service: &DeployableService, container_config: &ContainerConfig, - ) -> Result<&'a DeployableService, KubernetesInfrastructureError> { - if let Some(files) = service.files() { - self.deploy_secret(app_name, service, files).await?; - } - - let client = self.client().await?; - - let persistence_volume_map = self - .create_persistent_volume_claim(app_name, service) - .await?; - - match Api::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()) - .create( - &PostParams::default(), - &deployment_payload( - app_name, - service, - container_config, - self.config - .registry_credentials(&service.image().registry().unwrap_or_default()) - .is_some(), - &persistence_volume_map, - ), - ) - .await - { - Ok(result) => { - debug!( - "Successfully deployed {}", - result - .metadata - .name - .unwrap_or_else(|| String::from("")) - ); - self.post_service_and_custom_resource_definitions(app_name, service) - .await?; - Ok(service) - } - - Err(KubeError::Api(ErrorResponse { code, .. })) if code == 409 => { - Api::::namespaced( - client.clone(), - &app_name.to_rfc1123_namespace_id(), - ) - .patch( - &format!( - "{}-{}-deployment", - app_name.to_rfc1123_namespace_id(), - service.service_name() - ), - &PatchParams::default(), - &Patch::Merge(deployment_payload( - app_name, - service, - container_config, - self.config - .registry_credentials(&service.image().registry().unwrap_or_default()) - .is_some(), - &persistence_volume_map, - )), - ) - .await?; - Ok(service) - } - Err(e) => { - error!("Cannot deploy service: {}", e); - Err(e.into()) - } - } - } - - async fn deploy_secret( - &self, - app_name: &AppName, - service_config: &ServiceConfig, - volumes: &BTreeMap, - ) -> Result<(), KubernetesInfrastructureError> { - debug!( - "Deploying volumes as secrets for {} in app {}", - service_config.service_name(), - app_name + ) -> Result< + ( + Option, + V1Service, + V1Deployment, + IngressRoute, + Vec, + ), + KubernetesInfrastructureError, + > { + let secret = deployable_service + .files() + .map(|files| secrets_payload(app_name, deployable_service, files)); + + let service = service_payload(app_name, deployable_service); + + let deployment = deployment_payload( + app_name, + deployable_service, + container_config, + &self + .create_persistent_volume_claim(app_name, deployable_service) + .await?, ); - let client = self.client().await?; + let ingress_route = ingress_route_payload(app_name, deployable_service); + let middlewares = middleware_payload(app_name, deployable_service.ingress_route()); - match Api::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()) - .create( - &PostParams::default(), - &secrets_payload(app_name, service_config, volumes), - ) - .await - { - Ok(result) => { - debug!( - "Successfully deployed {}", - result - .metadata - .name - .unwrap_or_else(|| String::from("")) - ); - Ok(()) - } - Err(KubeError::Api(ErrorResponse { code, .. })) if code == 409 => { - Api::::namespaced(client.clone(), &app_name.to_rfc1123_namespace_id()) - .patch( - &format!( - "{}-{}-secret", - app_name.to_rfc1123_namespace_id(), - service_config.service_name() - ), - &PatchParams::default(), - &Patch::Merge(secrets_payload(app_name, service_config, volumes)), - ) - .await?; - Ok(()) - } - Err(e) => { - error!("Cannot deploy secret: {}", e); - Err(e.into()) - } - } + Ok((secret, service, deployment, ingress_route, middlewares)) } async fn create_persistent_volume_claim<'a>( @@ -580,7 +459,7 @@ impl KubernetesInfrastructure { impl Infrastructure for KubernetesInfrastructure { async fn get_services(&self) -> Result, Error> { let client = self.client().await?; - let app_names = Api::::all(client.clone()) + let mut app_name_and_services = Api::::all(client.clone()) .list(&ListParams { label_selector: Some(APP_NAME_LABEL.to_string()), ..Default::default() @@ -597,12 +476,17 @@ impl Infrastructure for KubernetesInfrastructure { .filter_map(|ns| { AppName::from_str(ns.metadata.labels.as_ref()?.get(APP_NAME_LABEL)?).ok() }) - .collect::>(); + .map(|app_name| async { + self.get_services_of_app(&app_name) + .await + .map(|services| (app_name, services)) + }) + .map(Box::pin) + .collect::>(); let mut apps = MultiMap::new(); - - for app_name in app_names { - let services = self.get_services_of_app(&app_name).await?; + while let Some(res) = app_name_and_services.next().await { + let (app_name, services) = res?; apps.insert_many(app_name, services); } @@ -615,24 +499,52 @@ impl Infrastructure for KubernetesInfrastructure { deployment_unit: &DeploymentUnit, container_config: &ContainerConfig, ) -> Result, Error> { - let services = deployment_unit.services(); let app_name = deployment_unit.app_name(); - self.create_namespace_if_necessary(app_name).await?; - self.create_pull_secrets_if_necessary(app_name, services) - .await?; - let futures = services - .iter() - .map(|service| self.deploy_service(app_name, service, container_config)) - .collect::>(); + let client = self.client().await?; + + let bootstrap_image_pull_secret = self.image_pull_secret( + app_name, + self.config + .companion_bootstrapping_containers() + .iter() + .map(|bc| bc.image()), + ); + let mut k8s_deployment_unit = K8sDeploymentUnit::bootstrap( + deployment_unit, + client.clone(), + self.config.companion_bootstrapping_containers(), + bootstrap_image_pull_secret, + ) + .await?; + + let services = self.get_services_of_app(app_name).await?; + k8s_deployment_unit.filter_by_instances_and_replicas(&services); + + for deployable_service in deployment_unit.services() { + let (secret, service, deployment, ingress_route, middlewares) = self + .create_payloads(app_name, deployable_service, container_config) + .await?; + + k8s_deployment_unit.merge(secret, service, deployment, ingress_route, middlewares); + } - for deploy_result in join_all(futures).await { - trace!("deployed {:?}", deploy_result); - deploy_result?; + if let Some(image_pull_secret) = + self.image_pull_secret(app_name, k8s_deployment_unit.images().iter()) + { + k8s_deployment_unit.apply_image_pull_secret(image_pull_secret); + } + + let deployments = k8s_deployment_unit.deploy(client, app_name).await?; + let mut services = Vec::with_capacity(deployments.len()); + for deployment in deployments.into_iter() { + if let Ok(service) = Self::create_service_from_deployment_and_pod(deployment, None) { + services.push(service); + } } - Ok(self.get_services_of_app(app_name).await?) + Ok(services) } async fn stop_services( @@ -662,23 +574,13 @@ impl Infrastructure for KubernetesInfrastructure { from: &Option>, limit: usize, ) -> Result, String)>>, Error> { - let p = ListParams { - label_selector: Some(format!("{SERVICE_NAME_LABEL}={service_name}",)), - ..Default::default() - }; - let pod = match Api::::namespaced( - self.client().await?, - &app_name.to_rfc1123_namespace_id(), - ) - .list(&p) - .await? - .into_iter() - .next() - { - Some(pod) => pod, - None => { - return Ok(None); - } + let client = self.client().await?; + let namespace = app_name.to_rfc1123_namespace_id(); + + let Some((_deployment, Some(pod))) = + self.get_deployment_and_pod(app_name, service_name).await? + else { + return Ok(None); }; let p = LogParams { @@ -700,10 +602,9 @@ impl Infrastructure for KubernetesInfrastructure { ..Default::default() }; - let logs = - Api::::namespaced(self.client().await?, &app_name.to_rfc1123_namespace_id()) - .logs(&pod.metadata.name.unwrap(), &p) - .await?; + let logs = Api::::namespaced(client, &namespace) + .logs(&pod.metadata.name.unwrap(), &p) + .await?; let logs = logs .split('\n') @@ -736,24 +637,31 @@ impl Infrastructure for KubernetesInfrastructure { service_name: &str, status: ServiceStatus, ) -> Result, Error> { - let (service, replicas) = match self.get_service_of_app(app_name, service_name).await? { - Some(service) if service.status() == &status => return Ok(None), - Some(service) => match status { - ServiceStatus::Running => (service, 1), - ServiceStatus::Paused => (service, 0), - }, - None => return Ok(None), + let Some((mut deployment, pod)) = + self.get_deployment_and_pod(app_name, service_name).await? + else { + return Ok(None); + }; + + let service = Self::create_service_from_deployment_and_pod(deployment.clone(), pod)?; + if service.status() == &status { + return Ok(None); + } + + let Some(spec) = deployment.spec.as_mut() else { + return Ok(None); }; + spec.replicas = Some(match status { + ServiceStatus::Running => 1, + ServiceStatus::Paused => 0, + }); + Api::::namespaced(self.client().await?, &app_name.to_rfc1123_namespace_id()) .patch( - &format!( - "{}-{}-deployment", - app_name.to_rfc1123_namespace_id(), - service_name - ), + &deployment.metadata.name.clone().unwrap(), &PatchParams::default(), - &Patch::Merge(deployment_replicas_payload(app_name, &service, replicas)), + &Patch::Merge(deployment), ) .await?; @@ -866,9 +774,9 @@ impl TryFrom for ServiceBuilder { deployment .spec .as_ref() - .map(|spec| match (spec.paused, spec.replicas) { - (Some(true), _) => ServiceStatus::Paused, - (Some(false), Some(replicas)) if replicas <= 0 => ServiceStatus::Paused, + .map(|spec| match spec.replicas { + None => ServiceStatus::Paused, + Some(replicas) if replicas <= 0 => ServiceStatus::Paused, _ => ServiceStatus::Running, }) .unwrap_or(ServiceStatus::Paused), @@ -892,24 +800,24 @@ impl TryFrom<&V1Deployment> for ServiceConfig { &deployment.metadata.labels, &deployment.metadata.annotations, ) { - let service_name = match labels.get(SERVICE_NAME_LABEL) { - Some(service_name) => service_name, - None => { - return Err(KubernetesInfrastructureError::MissingServiceNameLabel { - deployment_name: deployment_name.clone(), - }); - } - }; + let service_name = labels.get(SERVICE_NAME_LABEL).unwrap_or(deployment_name); - let image = annotations + let image = match annotations .get(IMAGE_LABEL) - .map(|image| { - Image::from_str(image) - .expect("Kubernetes API should provide valid image string") - }) - .ok_or_else(|| KubernetesInfrastructureError::MissingImageLabel { - deployment_name: deployment_name.clone(), - })?; + .and_then(|image| Image::from_str(image).ok()) + { + Some(img) => img, + None => deployment + .spec + .as_ref() + .and_then(|spec| spec.template.spec.as_ref()) + .and_then(|pod_spec| pod_spec.containers.first()) + .and_then(|container| container.image.as_ref()) + .and_then(|image| Image::from_str(image).ok()) + .ok_or_else(|| KubernetesInfrastructureError::MissingImageLabel { + deployment_name: deployment_name.clone(), + })?, + }; let mut config = ServiceConfig::new(service_name.clone(), image); @@ -1111,7 +1019,7 @@ mod tests { } #[test] - fn should_not_parse_service_from_deployment_spec_missing_service_name_label() { + fn should_parse_service_from_deployment_spec_with_missing_service_name_label() { let deployment = deployment_object!( "master-nginx", Some(String::from("master")), @@ -1120,13 +1028,11 @@ mod tests { None, ); - let err = ServiceBuilder::try_from(deployment).unwrap_err(); - assert_eq!( - err, - KubernetesInfrastructureError::MissingServiceNameLabel { - deployment_name: "master-nginx".to_string() - } - ); + let service = ServiceBuilder::try_from(deployment) + .unwrap() + .build() + .unwrap(); + assert_eq!(service.service_name(), "master-nginx"); } #[test] diff --git a/api/src/infrastructure/kubernetes/mod.rs b/api/src/infrastructure/kubernetes/mod.rs index 38e22459..2f2d1744 100644 --- a/api/src/infrastructure/kubernetes/mod.rs +++ b/api/src/infrastructure/kubernetes/mod.rs @@ -25,5 +25,6 @@ */ pub use infrastructure::KubernetesInfrastructure; +mod deployment_unit; mod infrastructure; mod payloads; diff --git a/api/src/infrastructure/kubernetes/payloads.rs b/api/src/infrastructure/kubernetes/payloads.rs index aac3830b..7a4b567d 100644 --- a/api/src/infrastructure/kubernetes/payloads.rs +++ b/api/src/infrastructure/kubernetes/payloads.rs @@ -31,17 +31,17 @@ use crate::config::{Config, ContainerConfig}; use crate::deployment::deployment_unit::{DeployableService, DeploymentStrategy}; use crate::infrastructure::traefik::TraefikMiddleware; use crate::infrastructure::{TraefikIngressRoute, TraefikRouterRule}; -use crate::models::service::Service; use crate::models::{AppName, ServiceConfig}; use base64::{engine::general_purpose, Engine}; use bytesize::ByteSize; use chrono::Utc; use k8s_openapi::api::apps::v1::DeploymentSpec; use k8s_openapi::api::core::v1::{ - Container, ContainerPort, EnvVar, KeyToPath, LocalObjectReference, PersistentVolumeClaim, - PersistentVolumeClaimSpec, PersistentVolumeClaimVolumeSource, PodSpec, PodTemplateSpec, - ResourceRequirements, SecretVolumeSource, Volume, VolumeMount, + Container, ContainerPort, EnvVar, KeyToPath, PersistentVolumeClaim, PersistentVolumeClaimSpec, + PersistentVolumeClaimVolumeSource, PodSpec, PodTemplateSpec, ResourceRequirements, + SecretVolumeSource, Volume, VolumeMount, }; +use k8s_openapi::api::networking::v1::Ingress; use k8s_openapi::api::{ apps::v1::Deployment as V1Deployment, core::v1::Namespace as V1Namespace, core::v1::Secret as V1Secret, core::v1::Service as V1Service, @@ -56,8 +56,10 @@ use schemars::JsonSchema; use secstr::SecUtf8; use serde::{Deserialize, Serialize}; use serde_json::{Map, Value}; +use std::collections::hash_map::DefaultHasher; use std::collections::{BTreeMap, HashMap, HashSet}; use std::convert::TryFrom; +use std::hash::Hasher; use std::iter::FromIterator; use std::path::{Component, PathBuf}; use std::str::FromStr; @@ -72,7 +74,7 @@ use std::string::ToString; )] #[serde(rename_all = "camelCase")] pub struct IngressRouteSpec { - pub entrypoints: Option>, + pub entry_points: Option>, pub routes: Option>, pub tls: Option, } @@ -94,13 +96,13 @@ pub struct TraefikRuleService { #[derive(Clone, Debug, Default, Deserialize, Serialize, JsonSchema)] pub struct TraefikRuleMiddleware { - name: String, + pub name: String, } #[derive(Clone, Debug, Default, Deserialize, Serialize, JsonSchema)] #[serde(rename_all = "camelCase")] pub struct TraefikTls { - cert_resolver: Option, + pub cert_resolver: Option, } #[derive(CustomResource, Clone, Debug, Deserialize, Serialize, JsonSchema)] @@ -111,7 +113,7 @@ pub struct TraefikTls { namespaced )] #[serde(rename_all = "camelCase")] -pub struct MiddlewareSpec(Value); +pub struct MiddlewareSpec(pub Value); macro_rules! secret_name_from_path { ($path:expr) => {{ @@ -139,14 +141,23 @@ macro_rules! secret_name_from_name { } impl TryFrom for TraefikIngressRoute { - type Error = &'static str; + type Error = String; fn try_from(value: IngressRoute) -> Result { - let k8s_route = value.spec.routes.unwrap().into_iter().next().unwrap(); - let rule = TraefikRouterRule::from_str(&k8s_route.r#match).unwrap(); + let Some(routes) = value.spec.routes else { + return Err(String::from( + "The ingress route does not provide any routes", + )); + }; + let Some(k8s_route) = routes.into_iter().next() else { + return Err(String::from( + "The ingress route does not provide any routes", + )); + }; + let rule = TraefikRouterRule::from_str(&k8s_route.r#match)?; Ok(TraefikIngressRoute::with_existing_routing_rules( - value.spec.entrypoints.unwrap_or_default(), + value.spec.entry_points.unwrap_or_default(), rule, k8s_route .middlewares @@ -159,6 +170,136 @@ impl TryFrom for TraefikIngressRoute { } } +pub fn convert_k8s_ingress_to_traefik_ingress( + ingress: Ingress, + base_route: TraefikIngressRoute, +) -> Result<(IngressRoute, Option), &'static str> { + let Some(spec) = ingress.spec else { + return Err("Ingress does not provide spec"); + }; + let Some(rules) = spec.rules else { + return Err("Ingress' spec does not provide rules"); + }; + + let Some(path) = rules + .into_iter() + .filter_map(|rule| rule.http) + .find_map(|http| http.paths.into_iter().next()) + else { + return Err("Ingress' rule does not a provide http paths object"); + }; + + let Some(path_value) = path.path else { + return Err("Ingress' path does not provide a HTTP path value"); + }; + + let (rule, middleware) = match &spec.ingress_class_name { + Some(ingress_class_name) if ingress_class_name == "nginx" => { + let middleware = ingress + .metadata + .annotations + .as_ref() + .filter(|annotations| { + annotations.get("nginx.ingress.kubernetes.io/use-regex") + == Some(&String::from("true")) + }) + .and_then(|annotations| { + annotations + .get("nginx.ingress.kubernetes.io/rewrite-target") + .cloned() + }) + .and_then(|_rewrite_target| { + let hir = regex_syntax::parse(&path_value).ok()?; + let got = regex_syntax::hir::literal::Extractor::new().extract(&hir); + let prefixes = got + .literals()? + .iter() + .map(|l| String::from_utf8_lossy(l.as_bytes()).to_string()) + .map(serde_json::Value::from) + .collect::>(); + + Some(Middleware { + metadata: kube::core::ObjectMeta { + name: Some(uuid::Uuid::new_v4().to_string()), + ..Default::default() + }, + spec: MiddlewareSpec(serde_json::json!({ + "stripPrefix": { + "prefixes": serde_json::Value::from(prefixes) + } + })), + }) + }); + + (None, middleware) + } + _ => { + // TODO warn that ingress class is unknown + ( + Some(TraefikIngressRoute::with_rule( + TraefikRouterRule::path_prefix_rule([path_value.clone()]), + )), + None, + ) + } + }; + + let mut route = base_route; + if let Some(rule) = rule { + route.merge_with(rule); + } + + let mut middlewares = route + .routes() + .iter() + .flat_map(|route| route.middlewares().iter()) + .filter_map(|middleware| match middleware { + crate::infrastructure::traefik::TraefikMiddleware::Ref(name) => { + Some(TraefikRuleMiddleware { name: name.clone() }) + } + crate::infrastructure::traefik::TraefikMiddleware::Spec { .. } => None, + }) + .collect::>(); + middlewares.extend(middleware.as_ref().map(|m| TraefikRuleMiddleware { + name: m.metadata.name.clone().unwrap_or_default(), + })); + + let routes = vec![TraefikRuleSpec { + kind: String::from("Rule"), + r#match: route.routes()[0].rule().to_string(), + middlewares: Some(middlewares), + services: vec![TraefikRuleService { + kind: Some(String::from("Service")), + name: path.backend.service.clone().unwrap().name, + port: Some( + path.backend + .service + .as_ref() + .and_then(|service| service.port.as_ref()) + .and_then(|port| port.number) + .map(|p| p as u16) + // TODO: for now it is okay to assume that if the port is missing, port 80 is a + // good default. However, in the future there should be some better error + // handling. + .unwrap_or(80), + ), + }], + }]; + + let route = IngressRoute { + metadata: ingress.metadata, + spec: IngressRouteSpec { + routes: Some(routes), + entry_points: Some(route.entry_points().clone()), + tls: route.tls().as_ref().map(|tls| TraefikTls { + cert_resolver: Some(tls.cert_resolver.clone()), + }), + }, + }; + + Ok((route, middleware)) +} + /// Creates a JSON payload suitable for [Kubernetes' /// Namespaces](https://kubernetes.io/docs/tasks/administer-cluster/namespaces/) pub fn namespace_payload(app_name: &AppName, config: &Config) -> V1Namespace { @@ -202,7 +343,6 @@ pub fn deployment_payload( app_name: &AppName, service: &DeployableService, container_config: &ContainerConfig, - use_image_pull_secret: bool, persistent_volume_map: &Option>, ) -> V1Deployment { let env = service.env().map(|env| { @@ -346,7 +486,7 @@ pub fn deployment_payload( template: PodTemplateSpec { metadata: Some(ObjectMeta { labels: Some(labels), - annotations: Some(deployment_annotations(service)), + annotations: Some(deployment_annotations(service.strategy())), ..Default::default() }), spec: Some(PodSpec { @@ -364,16 +504,6 @@ pub fn deployment_payload( resources, ..Default::default() }], - image_pull_secrets: if use_image_pull_secret { - Some(vec![LocalObjectReference { - name: Some(format!( - "{}-image-pull-secret", - app_name.to_rfc1123_namespace_id() - )), - }]) - } else { - None - }, ..Default::default() }), }, @@ -389,8 +519,8 @@ pub fn deployment_payload( /// For example, this [popular workaround](https://stackoverflow.com/a/55221174/5088458) will be /// applied to ensure that a pod will be recreated everytime a deployment with /// [`DeploymentStrategy::RedeployAlways`] has been initiated. -fn deployment_annotations(service: &DeployableService) -> BTreeMap { - match service.strategy() { +fn deployment_annotations(strategy: &DeploymentStrategy) -> BTreeMap { + match strategy { DeploymentStrategy::RedeployOnImageUpdate(image_id) => { BTreeMap::from([(String::from("imageHash"), image_id.clone())]) } @@ -401,37 +531,6 @@ fn deployment_annotations(service: &DeployableService) -> BTreeMap V1Deployment { - serde_json::from_value(serde_json::json!({ - "apiVersion": "apps/v1", - "kind": "Deployment", - "metadata": { - "name": format!("{}-{}-deployment", app_name.to_rfc1123_namespace_id(), service.service_name()), - "namespace": app_name.to_rfc1123_namespace_id(), - "labels": { - APP_NAME_LABEL: app_name, - SERVICE_NAME_LABEL: service.service_name(), - CONTAINER_TYPE_LABEL: service.container_type().to_string() - } - }, - "spec": { - "replicas": replicas, - "selector": { - "matchLabels": { - APP_NAME_LABEL: app_name, - SERVICE_NAME_LABEL: service.service_name(), - CONTAINER_TYPE_LABEL: service.container_type().to_string() - } - } - } - })) - .expect("Cannot convert value to apps/v1/Deployment") -} - /// Creates a JSON payload suitable for [Kubernetes' Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) pub fn secrets_payload( app_name: &AppName, @@ -468,6 +567,14 @@ pub fn image_pull_secret_payload( app_name: &AppName, registries_and_credentials: BTreeMap, ) -> V1Secret { + // Hashing over all registries ensures that the same secret name will be generated for the same + // registries. Thus, password or user can change and will be updated. Additionally, it will be + // idempontent to the Kubernetes API. + let mut registry_hasher = DefaultHasher::new(); + for registry in registries_and_credentials.keys() { + registry_hasher.write(registry.as_bytes()); + } + let data = ByteString( serde_json::json!({ "auths": @@ -490,8 +597,9 @@ pub fn image_pull_secret_payload( V1Secret { metadata: ObjectMeta { name: Some(format!( - "{}-image-pull-secret", - app_name.to_rfc1123_namespace_id() + "{}-image-pull-secret-{:#010x}", + app_name.to_rfc1123_namespace_id(), + registry_hasher.finish() )), namespace: Some(app_name.to_rfc1123_namespace_id()), labels: Some(BTreeMap::from([( @@ -542,8 +650,9 @@ pub fn service_payload(app_name: &AppName, service_config: &ServiceConfig) -> V1 /// See [Traefik Routers](https://docs.traefik.io/v2.0/user-guides/crd-acme/#traefik-routers) /// for more information. pub fn ingress_route_payload(app_name: &AppName, service: &DeployableService) -> IngressRoute { - let rules = service - .ingress_route() + let route = service.ingress_route(); + + let rules = route .routes() .iter() .map(|route| { @@ -606,18 +715,21 @@ pub fn ingress_route_payload(app_name: &AppName, service: &DeployableService) -> }, spec: IngressRouteSpec { routes: Some(rules), - ..Default::default() + entry_points: Some(route.entry_points().clone()), + tls: route.tls().as_ref().map(|tls| TraefikTls { + cert_resolver: Some(tls.cert_resolver.clone()), + }), }, } } -/// Creates a payload that ensures that Traefik strips out the path prefix. -/// /// See [Traefik Routers](https://docs.traefik.io/v2.0/user-guides/crd-acme/#traefik-routers) /// for more information. -pub fn middleware_payload(app_name: &AppName, service: &DeployableService) -> Vec { - service - .ingress_route() +pub fn middleware_payload( + app_name: &AppName, + ingress_route: &TraefikIngressRoute, +) -> Vec { + ingress_route .routes() .iter() .flat_map(|r| { @@ -758,7 +870,6 @@ mod tests { Vec::new(), ), &ContainerConfig::default(), - false, &None, ); @@ -837,7 +948,6 @@ mod tests { Vec::new(), ), &ContainerConfig::default(), - false, &None, ); @@ -919,7 +1029,6 @@ mod tests { Vec::new(), ), &ContainerConfig::default(), - false, &None, ); @@ -1002,7 +1111,6 @@ mod tests { Vec::new(), ), &ContainerConfig::default(), - false, &None, ); @@ -1157,15 +1265,11 @@ mod tests { #[test] fn should_create_middleware_with_default_prefix() { let app_name = AppName::master(); - let config = sc!("db", "mariadb:10.3.17"); - let service = DeployableService::new( - config, - DeploymentStrategy::RedeployAlways, - TraefikIngressRoute::with_defaults(&app_name, "db"), - Vec::new(), - ); - let payload = middleware_payload(&app_name, &service); + let payload = middleware_payload( + &app_name, + &TraefikIngressRoute::with_defaults(&app_name, "db"), + ); assert_json_diff::assert_json_include!( actual: payload, @@ -1190,15 +1294,11 @@ mod tests { #[test] fn should_create_middleware_with_default_prefix_with_name_rfc1123_app_name() { let app_name = AppName::from_str("MY-APP").unwrap(); - let config = sc!("db", "mariadb:10.3.17"); - let service = DeployableService::new( - config, - DeploymentStrategy::RedeployAlways, - TraefikIngressRoute::with_defaults(&app_name, "db"), - Vec::new(), - ); - let payload = middleware_payload(&app_name, &service); + let payload = middleware_payload( + &app_name, + &TraefikIngressRoute::with_defaults(&app_name, "db"), + ); assert_json_diff::assert_json_include!( actual: payload, @@ -1260,7 +1360,6 @@ mod tests { vec![String::from("/var/lib/data")], ), &ContainerConfig::default(), - false, &Some(HashMap::from([( &String::from("/var/lib/data"), persistent_volume_claim, @@ -1359,7 +1458,6 @@ mod tests { Vec::new(), ), &ContainerConfig::default(), - false, &None, ); @@ -1489,4 +1587,48 @@ mod tests { } ); } + + #[test] + fn create_image_pull_secrets() { + let payload = image_pull_secret_payload( + &AppName::from_str("MY-APP").unwrap(), + BTreeMap::from([( + String::from("registry.gitlab.com"), + ("oauth2", &SecUtf8::from_str("some-random-token").unwrap()), + )]), + ); + + assert_eq!( + payload, + V1Secret { + metadata: ObjectMeta { + name: Some(String::from("my-app-image-pull-secret-0x7a2952c7a89d3fd0")), + namespace: Some(String::from("my-app")), + labels: Some(BTreeMap::from([( + String::from("com.aixigo.preview.servant.app-name"), + String::from("MY-APP") + )])), + ..Default::default() + }, + immutable: Some(true), + data: Some(BTreeMap::from([( + String::from(".dockerconfigjson"), + ByteString( + serde_json::json!({ + "auths": { + "registry.gitlab.com": { + "username": "oauth2", + "password": "some-random-token" + } + } + }) + .to_string() + .into_bytes() + ) + )])), + type_: Some(String::from("kubernetes.io/dockerconfigjson")), + ..Default::default() + } + ) + } } diff --git a/api/src/infrastructure/traefik.rs b/api/src/infrastructure/traefik.rs index dd3edd3d..cc06f641 100644 --- a/api/src/infrastructure/traefik.rs +++ b/api/src/infrastructure/traefik.rs @@ -13,10 +13,18 @@ pub struct TraefikIngressRoute { } impl TraefikIngressRoute { + pub fn entry_points(&self) -> &Vec { + &self.entry_points + } + pub fn routes(&self) -> &Vec { &self.routes } + pub fn tls(&self) -> &Option { + &self.tls + } + #[cfg(test)] pub fn empty() -> Self { Self { @@ -26,14 +34,37 @@ impl TraefikIngressRoute { } } + pub fn with_app_only_defaults(app_name: &AppName) -> Self { + let mut prefixes = BTreeMap::new(); + prefixes.insert( + Value::String(String::from("prefixes")), + Value::Seq(vec![Value::String(format!("/{app_name}/",))]), + ); + + let mut middlewares = BTreeMap::new(); + middlewares.insert( + Value::String(String::from("stripPrefix")), + Value::Map(prefixes), + ); + + Self { + entry_points: Vec::new(), + routes: vec![TraefikRoute { + rule: TraefikRouterRule::path_prefix_rule([app_name.as_str()]), + middlewares: vec![TraefikMiddleware::Spec { + name: format!("{app_name}-middleware"), + spec: Value::Map(middlewares), + }], + }], + tls: None, + } + } + pub fn with_defaults(app_name: &AppName, service_name: &str) -> Self { let mut prefixes = BTreeMap::new(); prefixes.insert( Value::String(String::from("prefixes")), - Value::Seq(vec![Value::String(format!( - "/{}/{}/", - app_name, service_name - ))]), + Value::Seq(vec![Value::String(format!("/{app_name}/{service_name}/",))]), ); let mut middlewares = BTreeMap::new(); @@ -55,7 +86,6 @@ impl TraefikIngressRoute { } } - #[cfg(test)] pub fn with_rule(rule: TraefikRouterRule) -> Self { Self::with_existing_routing_rules(Vec::new(), rule, Vec::new(), None) } @@ -83,7 +113,7 @@ impl TraefikIngressRoute { } pub fn merge_with(&mut self, other: Self) { - self.entry_points.extend(other.entry_points.into_iter()); + self.entry_points.extend(other.entry_points); // FIXME: at the moment there is no handling of multiple routes which needs to be addessed // in the future when it is required. @@ -109,6 +139,47 @@ impl TraefikIngressRoute { (Some(_), Some(tls)) => Some(tls), }; } + + pub fn to_url(&self) -> Option { + let mut domain = None; + let mut path = None; + + match self.routes.first() { + Some(route) => { + let rule = &route.rule; + for m in &rule.matches { + match m { + Matcher::Host { domains } => { + domain = Some(&domains[0]); + } + Matcher::PathPrefix { paths } => { + path = Some(&paths[0]); + } + _ => {} + } + } + } + None => return None, + } + + let scheme = if self.tls.is_some() + || self + .entry_points + .iter() + .any(|entry_point| entry_point == "websecure") + { + "https" + } else { + "http" + }; + + Url::parse(&format!( + "{scheme}://{}{}", + domain?, + path.as_ref().map(|p| p.as_str()).unwrap_or_default() + )) + .ok() + } } #[derive(Clone, Debug, Eq, PartialEq)] @@ -361,7 +432,7 @@ impl Display for TraefikRouterRule { #[derive(Clone, Debug, Eq, PartialEq)] pub struct TraefikTLS { - cert_resolver: String, + pub cert_resolver: String, } #[cfg(test)] @@ -652,8 +723,7 @@ mod test { cert_resolver: String::from("letsencrypt"), }), }; - let route2 = - TraefikIngressRoute::with_defaults(&AppName::from_str("master").unwrap(), "whoami"); + let route2 = TraefikIngressRoute::with_defaults(&AppName::master(), "whoami"); route1.merge_with(route2); @@ -701,8 +771,7 @@ mod test { cert_resolver: String::from("letsencrypt"), }), }; - let mut route2 = - TraefikIngressRoute::with_defaults(&AppName::from_str("master").unwrap(), "whoami"); + let mut route2 = TraefikIngressRoute::with_defaults(&AppName::master(), "whoami"); route2.merge_with(route1); @@ -753,13 +822,13 @@ mod test { let mut route1 = TraefikIngressRoute::empty(); route1.merge_with(TraefikIngressRoute::with_defaults( - &AppName::from_str("master").unwrap(), + &AppName::master(), "test", )); assert_eq!( route1, - TraefikIngressRoute::with_defaults(&AppName::from_str("master").unwrap(), "test",) + TraefikIngressRoute::with_defaults(&AppName::master(), "test",) ); } @@ -787,4 +856,66 @@ mod test { } ); } + + mod to_url { + use super::*; + + #[test] + fn empty_route() { + assert_eq!(TraefikIngressRoute::empty().to_url(), None); + } + + #[test] + fn with_host_rule() { + let url = + TraefikIngressRoute::with_rule(TraefikRouterRule::host_rule(vec![String::from( + "example.com", + )])) + .to_url(); + + assert_eq!(url, Url::parse("http://example.com").ok()); + } + + #[test] + fn with_host_and_path_rule() { + let url = TraefikIngressRoute::with_rule( + TraefikRouterRule::from_str( + "PathPrefix(`/master/whoami/`) && Host(`prevant.example.com`)", + ) + .unwrap(), + ) + .to_url(); + + assert_eq!( + url, + Url::parse("http://prevant.example.com/master/whoami/").ok() + ); + } + + #[test] + fn with_host_rule_and_tls() { + let mut route = + TraefikIngressRoute::with_rule(TraefikRouterRule::host_rule(vec![String::from( + "example.com", + )])); + route.tls = Some(TraefikTLS { + cert_resolver: String::from("first"), + }); + let url = route.to_url(); + + assert_eq!(url, Url::parse("https://example.com").ok()); + } + + #[test] + fn with_host_rule_and_websecure_entrypoint() { + let mut route = + TraefikIngressRoute::with_rule(TraefikRouterRule::host_rule(vec![String::from( + "example.com", + )])); + route.entry_points.push(String::from("websecure")); + let url = route.to_url(); + + assert_eq!(url, Url::parse("https://example.com").ok()); + } + } } diff --git a/api/src/models/app_name.rs b/api/src/models/app_name.rs index 1f87ccf6..d04f31c9 100644 --- a/api/src/models/app_name.rs +++ b/api/src/models/app_name.rs @@ -65,6 +65,12 @@ impl std::fmt::Display for AppName { } } +impl AsRef for AppName { + fn as_ref(&self) -> &str { + self.0.as_str() + } +} + impl FromStr for AppName { type Err = AppNameError; diff --git a/api/src/models/service.rs b/api/src/models/service.rs index 9023e62d..6c36d27c 100644 --- a/api/src/models/service.rs +++ b/api/src/models/service.rs @@ -232,14 +232,6 @@ impl ServiceBuilder { self } - pub fn current_app_name(&self) -> Option<&String> { - self.app_name.as_ref() - } - - pub fn current_config(&self) -> Option<&ServiceConfig> { - self.config.as_ref() - } - pub fn started_at(mut self, started_at: DateTime) -> Self { self.started_at = Some(started_at); self diff --git a/assets/bootstrap-companions.svg b/assets/bootstrap-companions.svg new file mode 100644 index 00000000..2082e8b1 --- /dev/null +++ b/assets/bootstrap-companions.svg @@ -0,0 +1,1148 @@ + + + + + + image/svg+xml + + in-a-nutshell + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + in-a-nutshell + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/companions.md b/docs/companions.md new file mode 100644 index 00000000..94860bb1 --- /dev/null +++ b/docs/companions.md @@ -0,0 +1,192 @@ +# Companion Configuration + +Have a look at the [basic terminology](../README.md) to understand what a +companion is. For these use cases the following sections provide example +configurations. + +A simple, but limited configuration of companions can be done via the +`config.toml` file for [application companions](#application-wide) and [service +companions](#service-based). More complex companions can be created via +[bootstrapping](#bootstrapping-from-the-infrastructure-backend). + +## Static Configuration + +### Application Wide + +If you want to include an OpenID provider for every application, you could use +the following configuration: + +```toml +[companions.openid] +type = 'application' +image = 'private.example.com/library/openid:latest' +env = [ 'KEY=VALUE' ] +``` + +The provided values of `serviceName` and `env` can include the [handlebars syntax][handlebars] in order to access dynamic values. + +Additionally, you could mount files that are generated from handlebars templates (example contains a properties generation): + +```toml +[companions.openid.volumes] +"/path/to/volume.properties" = """ +remote.services={{#each services~}} + {{~#if (eq type 'instance')~}} + {{name}}:{{port}}, + {{~/if~}} +{{~/each~}} +""" +``` + +Furthermore, you can provide labels through handlebars templating: + +```toml +[companions.openid.labels] +"com.github.prevant" = "bar-{{application.name}}" +``` + +#### Template Variables + +The list of available handlebars variables: + +- `application`: The companion's application information + - `name`: The application name +- `services`: An array of the services of the application. Each element has the + following structure: + - `name`: The service name which is equivalent to the network alias + - `port`: The exposed port of the service + - `type`: The type of service. For example, `instance`, `replica`, `app-companion`, or `service-companion`. + +#### Handlebar Helpers + +PREvant provides some handlebars helpers which can be used to generate more complex configuration files. See handlerbar's [block helper documentation](https://handlebarsjs.com/block_helpers.html) for more details. + +- `{{#isCompanion }}` A conditional handlerbars block helper that checks if the given service type matches any companion type. +- `isNotCompanion ` A conditional handlerbars block helper that checks if the given service type does not match any companion type. + +### Service Based + +The service-based companions work the in the same way as the application-based +services. Make sure, that the `serviceName` is unique by using handlebars +templating. + +```toml +[companions.service-name] +serviceName = '{{service.name}}-db' +image = 'postgres:11' +env = [ 'KEY=VALUE' ] + +[companions.service-name.postgres.volumes] +"/path/to/volume.properties" == "…" +[companions.openid.labels] +"com.github.prevant" = "bar-{{application.name}}" +``` + + +#### Template Variables + +The list of available handlebars variables: + +- `application`: The companion's application information + - `name`: The application name +- `service`: The companion's service containing the following fields: + - `name`: The service name which is equivalent to the network alias + - `port`: The exposed port of the service + - `type`: The type of service. For example, `instance`, `replica`, `app-companion`, or `service-companion`. + +### Deployment Strategy + +Companions offer different deployment strategies so that a companion could be restarted or not under certain conditions. Therefore, PREvant offers following configuration flags: + +```toml +[companions.openid] +type = 'application' +image = 'private.example.com/library/openid:latest' +deploymentStrategy = 'redeploy-on-image-update' +``` + +`deploymentStrategy` offers the following values and if a companion exists for an app following strategy will be applied: + +- `redeploy-always` (_default_): Re-deploys the companion every time there is a new deployment request. +- `redeploy-on-image-update`: Re-deploys the companion if there is a more rescent image available. +- `redeploy-never`: Even if there is a new deployment request the companion won't be redeployed and stays running. + +### Storage Strategy + +Companions may have varying storage requirements and storage strategies cater to these by offering the below configuration flags: + +```toml +[companions.postgres] +type = 'application' +image = 'postgres:latest' +storageStrategy = 'mount-declared-image-volumes' +``` + +`storageStrategy` offers the following values to determine how storage is managed for a companion: + +- `none` (_default_): Companion is deployed without persistent storage. +- `mount-declared-image-volumes`: Mounts the volume paths declared within the image, providing persistent storage for the companion. + +## Bootstrapping From the Infrastructure Backend + +When the [static configuration](#static-configuration) is insufficient for your +use case, then PREvant can utilize the underlying infrastructure to bootstrap +the companion configuration from the stdout of containers that are run once +within the infrastructure (depicted by the following image). PREvant's static +companion configuration might be insufficient if services of the application +rely on volume sharing among services (see [#123][persistent-data-issue]) or +when operations are required to be run at the application's start up, e.g. +importing test data. + +![](../assets/bootstrap-companions.svg "Illustration how bootstrapping of companions work") + +In the depicted images PREvant will start one or more containers on the +infrastructure backend that are expected to generate output on standard out +(stdout) that will be parsed by PREvant that needs to be native to the +underlying infrastructure. + +- When PREvant uses Kubernetes as the infrastructure runtime, the bootstrap + containers need to output [Kubernetes manifests][k8s-manifest]. + + Make sure to output YAML that is compatible with 1.1 and 1.2 + (For example, bitnami helm charts have been adjusted in part in that regard, + see [here][zookeeper-yaml-1.2-pr] and [here][kafka-yaml-1.2-pr]) +- Docker: not yet implemented but the aim is to support [Docker + compose][docker-compose] files. + +Then before deploying these bootstrapped companions PREvant merges them with +the objects generated from the HTTP request payload (all bootstrapped +companions will be considered as application companions). Thus you can add or +overwrite configurations. For example, you can change the image used or an +environment variable. If you overwrite any configuration the companion will be +turned into an instance (as PREvant did before). + +The following configuration block depicts that an image +`registry.example.com/user/bootstrap-helm-chart:lastet` is based on a [Helm +chart][helm-chart] that generates an OpenID provider for your application with +[Keycloak][keycloak]. Additionally, it is possible to pass additional arguments +to the container than can be templated with [Handlebars][handlebars]. + +```toml +[[companions.bootstrapping.containers]] +image = "registry.example.com/user/bootstrap-helm-chart:latest" +args = [ + "--set", "keycloak.httpRelativePath=/{{application.name}}/keycloak/", + "--set", "keycloak.redirectUris[0]={{application.baseUrl}}oauth_redir" +] +``` + +The list of available handlebars variables for bootstrap container arguments: + +- `application`: The companion's application information + - `name`: The application name + - `baseUrl`: The URL that all services in the application share + +[docker-compose]: https://docs.docker.com/compose/ +[handlebars]: https://handlebarsjs.com/ +[helm-chart]: https://helm.sh/docs/topics/charts/ +[k8s-manifest]: https://kubernetes.io/docs/reference/glossary/?all=true#term-manifest +[keycloak]: https://www.keycloak.org/ +[persistent-data-issue]: https://github.com/aixigo/PREvant/issues/123 +[zookeeper-yaml-1.2-pr]: https://github.com/bitnami/charts/pull/21081 +[kafka-yaml-1.2-pr]: https://github.com/bitnami/charts/pull/21086