Cloud-proxy container fails to startup and throws an error. May be fixed by https://github.com/pixie-io/pixie/pull/2016 #2017
Comments
JerrinAndrei
changed the title
JerrinAndrei
changed the title
ddelnano
added a commit
that referenced
this issue
Sep 16, 2024
…igmaps for easier runtime overrides (#2018) Summary: Remove nginx config files from cloud proxy container in favor of Configmaps for easier runtime overrides This is an alternative approach to #2014 and #2016. While this doesn't provide an environment variable for configuring the intended behavior, this approach is more flexible since many Nginx directives don't work with variables (`server_name`, `resolver`, among others ). Because nginx prohibits variables in these directives, it makes it very difficult to provide environment variable based settings without our previous `sed` approach. The `sed` approach also has its problems since it requires [hacks](https://github.com/pixie-io/pixie/pull/2014/files#diff-5ec7ca8d0f624fe1f4eb3778cc96dcee2f999bf39bad422807b67b15ce2f8e7bR27) to support configuration removals. Rather than trying to solve all potential use cases, this PR opts to make the configuration easy to swap out via the `pl-proxy-nginx-config` Configmap. I plan to update the self hosted cloud docs to call out that this Configmap exists and should be used if custom nginx configuration is needed outside of the upstream defaults. Relevant Issues: #2017 Type of change: /kind feature Test Plan: Deployed to a cloud environment and verified that the upstream defaults and `PL_DOMAIN_NAME` apply as expected Changelog Message: Removed nginx configuration from the container image into `pl-proxy-nginx-config` Configmap for easier runtime overrides --------- Signed-off-by: Dom Del Nano <[email protected]>
|
This is covered by #2018 and will be available in the next cloud release. |
ddelnano
added a commit
that referenced
this issue
Sep 18, 2024
…igmap mount directory) (#2027) Summary: Fix cloud proxy entrypoint by avoiding modifying a RO directory (Configmap mount directory) This bug was introduced between 0a44b36 and c3e0fba on #2018 when the individual file mounts were changed to a directory mount. Deploying the cloud proxy from main results in the following error: ``` $ kubectl -n plc logs cloud-proxy-5df85487bf-hrglr Defaulted container "cloud-proxy-server" out of: cloud-proxy-server, envoy /scripts/entrypoint.sh: line 20: can't create /usr/local/openresty/nginx/conf/nginx.conf: Read-only file system ``` When I originally tested the final change, I must have only looked at the resulting directory and missed that the pod was crashing. This issue was detected during the 0.1.8 cloud prerelease testing. Relevant Issues: #2017 #2013 Type of change: /kind bugfix Test Plan: Verified that the cloud proxy image starts up successfully Signed-off-by: Dom Del Nano <[email protected]>
ddelnano
added a commit
to ddelnano/pixie
that referenced
this issue
Sep 23, 2024
…igmaps for easier runtime overrides (pixie-io#2018) Summary: Remove nginx config files from cloud proxy container in favor of Configmaps for easier runtime overrides This is an alternative approach to pixie-io#2014 and pixie-io#2016. While this doesn't provide an environment variable for configuring the intended behavior, this approach is more flexible since many Nginx directives don't work with variables (`server_name`, `resolver`, among others ). Because nginx prohibits variables in these directives, it makes it very difficult to provide environment variable based settings without our previous `sed` approach. The `sed` approach also has its problems since it requires [hacks](https://github.com/pixie-io/pixie/pull/2014/files#diff-5ec7ca8d0f624fe1f4eb3778cc96dcee2f999bf39bad422807b67b15ce2f8e7bR27) to support configuration removals. Rather than trying to solve all potential use cases, this PR opts to make the configuration easy to swap out via the `pl-proxy-nginx-config` Configmap. I plan to update the self hosted cloud docs to call out that this Configmap exists and should be used if custom nginx configuration is needed outside of the upstream defaults. Relevant Issues: pixie-io#2017 Type of change: /kind feature Test Plan: Deployed to a cloud environment and verified that the upstream defaults and `PL_DOMAIN_NAME` apply as expected Changelog Message: Removed nginx configuration from the container image into `pl-proxy-nginx-config` Configmap for easier runtime overrides --------- Signed-off-by: Dom Del Nano <[email protected]> GitOrigin-RevId: 9b5f295
ddelnano
added a commit
to ddelnano/pixie
that referenced
this issue
Sep 23, 2024
…igmap mount directory) (pixie-io#2027) Summary: Fix cloud proxy entrypoint by avoiding modifying a RO directory (Configmap mount directory) This bug was introduced between 0a44b36 and c3e0fba on pixie-io#2018 when the individual file mounts were changed to a directory mount. Deploying the cloud proxy from main results in the following error: ``` $ kubectl -n plc logs cloud-proxy-5df85487bf-hrglr Defaulted container "cloud-proxy-server" out of: cloud-proxy-server, envoy /scripts/entrypoint.sh: line 20: can't create /usr/local/openresty/nginx/conf/nginx.conf: Read-only file system ``` When I originally tested the final change, I must have only looked at the resulting directory and missed that the pod was crashing. This issue was detected during the 0.1.8 cloud prerelease testing. Relevant Issues: pixie-io#2017 pixie-io#2013 Type of change: /kind bugfix Test Plan: Verified that the cloud proxy image starts up successfully Signed-off-by: Dom Del Nano <[email protected]> GitOrigin-RevId: 1f96cff
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
Currently the nginx.conf expects the a kubernetes dns service named kube-dns.kube-system.svc.cluster.local to be existing in the cluster. If this dns service does not exist the cloud-proxy pod gets terminated on startup.
To Reproduce
Steps to reproduce the behavior:
kustomize build k8s/cloud/public/ | kubectl apply -f -
The cloud proxy pod fails to start with error
"host not found in resolver kube-dns.kube-system.svc.cluster.local in /usr/local/openresty/nginx/conf/nginx.conf:43"
Expected behavior
Cloud proxy Pod should startup successfully with existing kube dns if any.
Every kubernetes cluster can have different kube dns service names. Also some clusters can have core-dns. So the cloud proxy service should look up for the dns service specific to the cluster in which its being setup. The control for defining this this should be given to the user.
This might be fixed in #2016
The text was updated successfully, but these errors were encountered: