-
Notifications
You must be signed in to change notification settings - Fork 344
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix docs for v0.57.2 #1987
Open
nikhita
wants to merge
4
commits into
vmware-tanzu:main
Choose a base branch
from
nikhita:release-process-fix
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Fix docs for v0.57.2 #1987
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,298 @@ | ||
--- | ||
version: v0.57.2 | ||
cascade: | ||
layout: docs | ||
gh: https://github.com/vmware-tanzu/sonobuoy/tree/v0.57.2 | ||
--- | ||
# ![Sonobuoy Logo](img/sonobuoy-logo.png) | ||
|
||
[![Test](https://github.com/vmware-tanzu/sonobuoy/actions/workflows/ci-test.yaml/badge.svg)](https://github.com/vmware-tanzu/sonobuoy/actions/workflows/ci-test.yaml/badge.svg) | ||
[![Lint](https://github.com/vmware-tanzu/sonobuoy/actions/workflows/ci-lint.yaml/badge.svg)](https://github.com/vmware-tanzu/sonobuoy/actions/workflows/ci-lint.yaml/badge.svg) | ||
|
||
## [Overview][oview] | ||
|
||
Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of | ||
plugins (including [Kubernetes][k8s] conformance tests) in an accessible and non-destructive manner. It is a | ||
customizable, extendable, and cluster-agnostic way to generate clear, informative reports about your cluster. | ||
|
||
Its selective data dumps of Kubernetes resource objects and cluster nodes allow for the following use cases: | ||
|
||
* Integrated end-to-end (e2e) [conformance-testing][e2ePlugin] | ||
* Workload debugging | ||
* Custom data collection via extensible plugins | ||
|
||
Starting v0.20, Sonobuoy supports Kubernetes v1.17 or later. Sonobuoy releases will be independent of Kubernetes | ||
release, while ensuring that new releases continue to work functionally across different versions of Kubernetes. Read | ||
more about the new release cycles in [our blog][decoupling-sonobuoy-k8s]. | ||
|
||
> Note: You can skip this version enforcement by running Sonobuoy with the `--skip-preflight` flag. | ||
|
||
## Prerequisites | ||
|
||
* Access to an up-and-running Kubernetes cluster. If you do not have a cluster, we recommend either: | ||
* following the [AWS Quickstart for Kubernetes][quickstart] instructions. | ||
* setting up a local cluster using [KinD][kind] | ||
|
||
* An admin `kubeconfig` file, and the KUBECONFIG environment variable set. | ||
|
||
* For some advanced workflows it may be required to have `kubectl` installed. | ||
See [installing via Homebrew (MacOS)][brew] or [building the binary (Linux)][linux]. | ||
|
||
* The `sonobuoy images` subcommand requires [Docker](https://www.docker.com) to be installed. | ||
See [installing Docker][docker]. | ||
|
||
## Installation | ||
|
||
The following methods exist for installing Sonobuoy: | ||
|
||
### Install binary | ||
|
||
1. Download the [latest release][releases] for your client platform. | ||
2. Extract the tarball: | ||
|
||
``` | ||
tar -xvf <RELEASE_TARBALL_NAME>.tar.gz | ||
``` | ||
|
||
Move the extracted `sonobuoy` executable to somewhere on your `PATH`. | ||
|
||
### Install with Hombrew (MacOS) | ||
|
||
1. Run the command: | ||
|
||
``` | ||
brew install sonobuoy | ||
``` | ||
|
||
## Getting Started | ||
|
||
To launch conformance tests (ensuring [CNCF][cncf] conformance) and wait until they are finished run: | ||
|
||
```bash | ||
sonobuoy run --wait | ||
``` | ||
|
||
> Note: Using `--mode quick` will significantly shorten the runtime of Sonobuoy. It runs just a single test, helping to quickly validate your Sonobuoy and Kubernetes configuration. | ||
|
||
Get the results from the plugins (e.g. e2e test results): | ||
|
||
```bash | ||
results=$(sonobuoy retrieve) | ||
``` | ||
|
||
Inspect results for test failures. This will list the number of tests failed and their names: | ||
|
||
```bash | ||
sonobuoy results $results | ||
``` | ||
|
||
> Note: The `results` command has lots of useful options for various situations. See the [results page][results] for more details. | ||
|
||
You can also extract the entire contents of the file to get much more [detailed data][snapshot] about your cluster. | ||
|
||
Sonobuoy creates a few resources in order to run and expects to run within its own namespace. | ||
|
||
Deleting Sonobuoy entails removing its namespace as well as a few cluster scoped resources. | ||
|
||
```bash | ||
sonobuoy delete --wait | ||
``` | ||
|
||
> Note: The --wait option ensures the Kubernetes namespace is deleted, avoiding conflicts if another Sonobuoy run is started quickly. | ||
|
||
If you have an issue with permissions in your cluster but you still want to run Sonobuoy, you can use `--aggregator-permissions` flag. Read more details about it [here][aggregator-permissions]. | ||
|
||
### Other Tests | ||
|
||
By default, `sonobuoy run` runs the Kubernetes conformance tests but this can easily be configured. The same plugin that | ||
has the conformance tests has all the Kubernetes end-to-end tests which include other tests such as: | ||
|
||
* tests for specific storage features | ||
* performance tests | ||
* scaling tests | ||
* provider specific tests | ||
* and many more | ||
|
||
To modify which tests you want to run, checkout our page on the [e2e plugin][e2ePlugin]. | ||
|
||
If you want to run other tests or tools which are not a part of the Kubernetes end-to-end suite, refer to our | ||
documentation on [custom plugins][customPlugins]. | ||
|
||
### Monitoring Sonobuoy during a run | ||
|
||
You can check on the status of each of the plugins running with: | ||
|
||
```bash | ||
sonobuoy status | ||
``` | ||
|
||
You can also inspect the logs of all Sonobuoy containers: | ||
|
||
```bash | ||
sonobuoy logs | ||
``` | ||
|
||
## Troubleshooting | ||
|
||
If you encounter any problems that the documentation does not address, [file an issue][issue]. | ||
|
||
## Docker Hub rate limit | ||
|
||
This year, Docker has started rate limiting image pulls from Docker Hub. We're planning a future release with a better | ||
user interface to work around this. Until then, this is the recommended approach. | ||
|
||
### Sonobuoy Pod | ||
|
||
Sonobuoy by default pulls from Docker Hub for [`sonobuoy/sonobuoy` image](https://hub.docker.com/r/sonobuoy/sonobuoy). | ||
If you're encountering rate limit on this, you can use VMware-provided mirror with: | ||
|
||
```bash | ||
sonobuoy run --sonobuoy-image projects.registry.vmware.com/sonobuoy/sonobuoy:<VERSION> | ||
``` | ||
|
||
### Conformance | ||
|
||
Kubernetes end-to-end conformance test pulls several images from Docker Hub as part of testing. To override this, you | ||
will need to create a registry manifest file locally (e.g. `conformance-image-config.yaml`) containing the following: | ||
|
||
```yaml | ||
dockerLibraryRegistry: mirror.gcr.io/library | ||
``` | ||
|
||
Then on running conformance: | ||
|
||
```bash | ||
sonobuoy run --sonobuoy-image projects.registry.vmware.com/sonobuoy/sonobuoy:<VERSION> --e2e-repo-config conformance-image-config.yaml | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. probably change this to sonobuoy/sonobuoy https://hub.docker.com/r/sonobuoy/sonobuoy/tags |
||
``` | ||
|
||
Technically `dockerGluster` is also a registry pulling from Docker Hub, but it's not part of Conformance test suite at | ||
the moment, so overriding `dockerLibraryRegistry` should be enough. | ||
|
||
## Known Issues | ||
|
||
### Leaked End-to-end namespaces | ||
|
||
There are some Kubernetes e2e tests that may leak resources. Sonobuoy can help clean those up as well by deleting all | ||
namespaces prefixed with `e2e`: | ||
|
||
```bash | ||
sonobuoy delete --all | ||
``` | ||
|
||
### Run on Google Cloud Platform (GCP) | ||
|
||
Sonobuoy requires admin permissions which won't be automatic if you are running via Google Kubernetes Engine (GKE) | ||
cluster. You must first create an admin role for the user under which you run Sonobuoy: | ||
|
||
```bash | ||
kubectl create clusterrolebinding <your-user-cluster-admin-binding> --clusterrole=cluster-admin --user=<[email protected]> | ||
``` | ||
|
||
### Run on Kubernetes for Docker Desktop | ||
|
||
We don't recommend running via a cluster set up via Docker Desktop. Known issues include: | ||
|
||
- `kubectl logs` will not function | ||
- `sonobuoy logs` will not function | ||
- `sonobuoy retrieve` will not function | ||
- `systemd-logs` plugin will hang | ||
|
||
Most of these issues revolve around issues with kube-proxy on Docker Desktop so if you know of how to resolve these | ||
issues, let us know. | ||
|
||
### Certified-Conformance bug (versions v0.53.0 and v0.53.1) | ||
|
||
These versions of Sonobuoy have a bug that runs the wrong set of tests without additional actions. See more | ||
details [here][issue1388]. The simplest way to avoid this is to update your version of Sonobuoy to >= v0.53.2. | ||
|
||
## Strategy Document | ||
|
||
See our current [strategy document][strategy] and [roadmap][roadmap] for context on what our highest priority use cases and work items | ||
will be. Feel free to make comments on Github or start conversations in Slack. | ||
|
||
## Contributing | ||
|
||
Thanks for taking the time to join our community and start contributing! We welcome pull requests. Feel free to dig | ||
through the [issues][issue] and jump in. | ||
|
||
The most common build/test functions are called via the Makefile: | ||
|
||
``` | ||
// Build the binary | ||
$ make build | ||
|
||
// Run local unit tests | ||
$ make test | ||
``` | ||
|
||
If you make changes which change output, you may fail tests which utilize the golden file testing pattern (e.g. correct data is stored in external files), update them by running: | ||
``` | ||
$ make golden | ||
``` | ||
|
||
In most cases, running integration tests is more simply done in CI when you open a pull request. | ||
You can dig into scripts/build_funcs.sh and our .github/workflows/ci-test.yaml for exact details of existing test flows. | ||
|
||
### Before you start | ||
|
||
* Please familiarize yourself with the [Code of Conduct][coc] before contributing. | ||
* See [CONTRIBUTING.md][contrib] for instructions on the developer certificate of origin that we require. | ||
* There is a [Slack channel][slack] if you want to interact with other members of the community | ||
|
||
## Changelog | ||
|
||
See [the list of releases][releases] to find out about feature changes. | ||
|
||
[decoupling-sonobuoy-k8s]: https://sonobuoy.io/decoupling-sonobuoy-and-kubernetes | ||
|
||
[airgap]: airgap | ||
|
||
[brew]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-with-homebrew-on-macos | ||
|
||
[cncf]: https://github.com/cncf/k8s-conformance#certified-kubernetes | ||
|
||
[coc]: https://github.com/vmware-tanzu/sonobuoy/blob/main/CODE_OF_CONDUCT.md | ||
|
||
[contrib]: https://github.com/vmware-tanzu/sonobuoy/blob/main/CONTRIBUTING.md | ||
|
||
[docker]: https://docs.docker.com/get-docker/ | ||
|
||
[docs]: https://sonobuoy.io/docs/v0.57.2 | ||
|
||
[e2ePlugin]: e2eplugin | ||
|
||
[customPlugins]: plugins | ||
|
||
[gen]: gen | ||
|
||
[issue]: https://github.com/vmware-tanzu/sonobuoy/issues | ||
|
||
[issue1388]: issue1388 | ||
|
||
[k8s]: https://github.com/kubernetes/kubernetes | ||
|
||
[kind]: https://github.com/kubernetes-sigs/kind | ||
|
||
[linux]: https://kubernetes.io/docs/tasks/tools/install-kubectl/#tabset-1 | ||
|
||
[oview]: https://youtu.be/8QK-Hg2yUd4 | ||
|
||
[plugins]: plugins | ||
|
||
[quickstart]: https://aws.amazon.com/quickstart/architecture/vmware-kubernetes/ | ||
|
||
[releases]: https://github.com/vmware-tanzu/sonobuoy/releases | ||
|
||
[results]: results | ||
|
||
[slack]: https://kubernetes.slack.com/messages/sonobuoy | ||
|
||
[snapshot]:snapshot | ||
|
||
[sonobuoyconfig]: sonobuoy-config | ||
|
||
[strategy]: strategy | ||
|
||
[aggregator-permissions]: aggregator-permissions | ||
|
||
[roadmap]: https://github.com/vmware-tanzu/sonobuoy/wiki |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# Aggregator Permissions | ||
|
||
By default, the Sonobuoy aggregator is given very elevated permissions in order to successfully run the Kubernetes end-to-end tests. In some situations you may want to (or need to) limit the permissions of the aggregator so that the aggregator and the pods that it creates do not have such wide-reaching permissions. You can always customize the exact permissions of the ServiceAccount via editing `sonobuoy gen` output manually, but Sonobuoy also provides useful presets via the CLI flag, `--aggregator-permissions`. | ||
|
||
## Type of Aggregator Permissions | ||
|
||
Allowable values are `[namespaced, clusterAdmin, clusterRead]`, `clusterAdmin` is default value. | ||
|
||
### clusterAdmin | ||
|
||
- `clusterAdmin` is the default value. With this value Sonobuoy can do pretty much everything in the run, it does not implement any restrictions. Most of these are required for the e2e conformance tests to work since they create/destroy namespaces, pods etc. | ||
|
||
### namespaceAdmin | ||
|
||
namespaceAdmin is the most restrictive preset permissions Sonobuoy provides and ensures that Sonobuoy and its plugins do not impact other namespaces at all. | ||
|
||
Due to these limitations there are a number of things to note: | ||
- Sonobuoy does not create the namespace so it needs to already exist | ||
- You must provide `--skip-preflight` to avoid Sonobuoy from complaining about the preexisting namespace | ||
- The `e2e` plugin (conformance tests) will not work in this mode and won't even start up due to severely limited permissions | ||
- Daemonset plugins will not work in this mode because Sonobuoy monitors them on a per-node basis. Since Sonobuoy can't query the list of nodes in the cluster, it can't properly monitor or gather results from them. At this time, Daemonset plugins will simply be ignored. | ||
|
||
### clusterRead | ||
|
||
`clusterRead` is a compromise between `namespaceAdmin` and `clusterAdmin`. It adds ability to GET any resource from the API so that the Sonobuoy queries work OK, it is able to get nodes so daemonsets run fine, and e2e tests can technically start. Sonobuoy can't create namespaces so e2e tests can't run in this mode in any useful manner either. However, this may be a more reasonable mode to run less intrusive, custom plugins in. In this mode Sonobuoy don't create the namespace either so it has to be created first and sonobuoy run with the `--skip-preflight` flag. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't have latest version on this might need to manually sync from dockerhub or update dockerhub ref