From 3d6d35e97619f369182373bf25962494634fe653 Mon Sep 17 00:00:00 2001 From: J Stickler Date: Mon, 4 Nov 2024 10:23:29 -0500 Subject: [PATCH] docs: update admonitions to new shortcode format (#14735) --- docs/sources/alert/_index.md | 8 +-- docs/sources/community/contributing.md | 4 +- .../2021-01-Ordering-Constraint-Removal.md | 8 +-- .../release/create-release-branch.md | 12 ++-- .../release/update-version-numbers.md | 4 +- docs/sources/get-started/deployment-modes.md | 8 +-- docs/sources/get-started/labels/_index.md | 4 +- .../get-started/labels/structured-metadata.md | 8 +-- docs/sources/operations/authentication.md | 4 +- .../operations/automatic-stream-sharding.md | 4 +- docs/sources/operations/blocking-queries.md | 8 +-- docs/sources/operations/bloom-filters.md | 4 +- docs/sources/operations/recording-rules.md | 8 +-- docs/sources/operations/scalability.md | 4 +- docs/sources/operations/storage/_index.md | 4 +- .../operations/storage/boltdb-shipper.md | 20 +++---- .../operations/storage/legacy-storage.md | 8 +-- .../operations/storage/logs-deletion.md | 4 +- docs/sources/operations/storage/retention.md | 40 ++++++------- .../storage/table-manager/_index.md | 8 +-- docs/sources/query/log_queries/_index.md | 4 +- docs/sources/query/logcli.md | 8 +-- docs/sources/query/query_accceleration.md | 4 +- docs/sources/reference/loki-http-api.md | 20 +++---- docs/sources/release-notes/cadence.md | 4 +- docs/sources/release-notes/v2-3.md | 4 +- docs/sources/send-data/_index.md | 4 +- .../sources/send-data/docker-driver/_index.md | 8 +-- .../send-data/docker-driver/configuration.md | 16 ++--- docs/sources/send-data/fluentd/_index.md | 4 +- .../send-data/lambda-promtail/_index.md | 12 ++-- docs/sources/send-data/otel/_index.md | 4 +- .../send-data/promtail/configuration.md | 4 +- .../sources/send-data/promtail/stages/drop.md | 4 +- .../sources/send-data/promtail/stages/json.md | 4 +- .../send-data/promtail/stages/match.md | 4 +- .../promtail/stages/structured_metadata.md | 4 +- .../setup/migrate/migrate-to-tsdb/_index.md | 4 +- docs/sources/setup/upgrade/_index.md | 60 +++++++++---------- 39 files changed, 174 insertions(+), 174 deletions(-) diff --git a/docs/sources/alert/_index.md b/docs/sources/alert/_index.md index e12e073c3b88..b43b78e092b4 100644 --- a/docs/sources/alert/_index.md +++ b/docs/sources/alert/_index.md @@ -202,18 +202,18 @@ Another great use case is alerting on high cardinality sources. These are things Creating these alerts in LogQL is attractive because these metrics can be extracted at _query time_, meaning we don't suffer the cardinality explosion in our metrics store. -{{% admonition type="note" %}} +{{< admonition type="note" >}} As an example, we can use LogQL v2 to help Loki to monitor _itself_, alerting us when specific tenants have queries that take longer than 10s to complete! To do so, we'd use the following query: `sum by (org_id) (rate({job="loki-prod/query-frontend"} |= "metrics.go" | logfmt | duration > 10s [1m])`. -{{% /admonition %}} +{{< /admonition >}} ## Interacting with the Ruler ### Lokitool Because the rule files are identical to Prometheus rule files, we can interact with the Loki Ruler via `lokitool`. -{{% admonition type="note" %}} +{{< admonition type="note" >}} lokitool is intended to run against multi-tenant Loki. The commands need an `--id=` flag set to the Loki instance ID or set the environment variable `LOKI_TENANT_ID`. If Loki is running in single tenant mode, the required ID is `fake`. -{{% /admonition %}} +{{< /admonition >}} An example workflow is included below: diff --git a/docs/sources/community/contributing.md b/docs/sources/community/contributing.md index f5918dfe82f4..e8553756ca34 100644 --- a/docs/sources/community/contributing.md +++ b/docs/sources/community/contributing.md @@ -30,10 +30,10 @@ $ git commit -m "docs: fix spelling error" $ git push -u fork HEAD ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} If you downloaded Loki using `go get`, the message `package github.com/grafana/loki: no Go files in /go/src/github.com/grafana/loki` is normal and requires no actions to resolve. -{{% /admonition %}} +{{< /admonition >}} ### Building diff --git a/docs/sources/community/design-documents/2021-01-Ordering-Constraint-Removal.md b/docs/sources/community/design-documents/2021-01-Ordering-Constraint-Removal.md index 64e59e21d07d..f50d9000343c 100644 --- a/docs/sources/community/design-documents/2021-01-Ordering-Constraint-Removal.md +++ b/docs/sources/community/design-documents/2021-01-Ordering-Constraint-Removal.md @@ -129,9 +129,9 @@ The performance losses against the current approach includes: Loki regularly combines multiple blocks into a chunk and "flushes" it to storage. In order to ensure that reads over flushed chunks remain as performant as possible, we will re-order a possibly-overlapping set of blocks into a set of blocks that maintain monotonically increasing order between them. From the perspective of the rest of Loki’s components (queriers/rulers fetching chunks from storage), nothing has changed. -{{% admonition type="note" %}} +{{< admonition type="note" >}} **In the case that data for a stream is ingested in order, this is effectively a no-op, making it well optimized for in-order writes (which is both the requirement and default in Loki currently). Thus, this should have little performance impact on ordered data while enabling Loki to ingest unordered data.** -{{% /admonition %}} +{{< /admonition >}} #### Chunk Durations @@ -153,9 +153,9 @@ The second is simple to implement and an effective way to ensure Loki can ingest We also cut chunks according to the `sync_period`. The first timestamp ingested past this bound will trigger a cut. This process aids in increasing chunk determinism and therefore our deduplication ratio in object storage because chunks are [content addressed](https://en.wikipedia.org/wiki/Content-addressable_storage). With the removal of our ordering constraint, it's possible that in some cases the synchronization method will not be as effective, such as during concurrent writes to the same stream across this bound. -{{% admonition type="note" %}} +{{< admonition type="note" >}} **It's important to mention that this is possible today with the current ordering constraint, but we'll be increasing the likelihood by removing it.** -{{% /admonition %}} +{{< /admonition >}} ``` Figure 5 diff --git a/docs/sources/community/maintaining/release/create-release-branch.md b/docs/sources/community/maintaining/release/create-release-branch.md index 0c33d61d26e4..fa4d87db66a6 100644 --- a/docs/sources/community/maintaining/release/create-release-branch.md +++ b/docs/sources/community/maintaining/release/create-release-branch.md @@ -21,18 +21,18 @@ branch is then used for all the Stable Releases, and all Patch Releases for that The name of the release branch should be `release-VERSION_PREFIX`, such as `release-2.9.x`. - {{% admonition type="note" %}} + {{< admonition type="note" >}} Branches are only made for VERSION_PREFIX; do not create branches for the full VERSION such as `release-2.9.1`. - {{% /admonition %}} + {{< /admonition >}} - {{% admonition type="note" %}} + {{< admonition type="note" >}} Don't create any other branches that are prefixed with `release` when creating PRs or those branches will collide with our automated release build publish rules. - {{% /admonition %}} + {{< /admonition >}} 1. Create a label to make backporting PRs to this branch easy. The name of the label should be `backport release-VERSION_PREFIX`, such as `backport release-2.9.x`. - {{% admonition type="note" %}} + {{< admonition type="note" >}} Note there is space in the label name. The label name must follow this naming convention to trigger CI related jobs. - {{% /admonition %}} + {{< /admonition >}} diff --git a/docs/sources/community/maintaining/release/update-version-numbers.md b/docs/sources/community/maintaining/release/update-version-numbers.md index 17a43ca1d6c7..79d1c9c0e2c1 100644 --- a/docs/sources/community/maintaining/release/update-version-numbers.md +++ b/docs/sources/community/maintaining/release/update-version-numbers.md @@ -24,6 +24,6 @@ Upgrade the Loki version to the new release version in documents, examples, json LOKI_NEW_VERSION=$VERSION ./tools/release_update_tags.sh ``` - {{% admonition type="note" %}} + {{< admonition type="note" >}} Do not upgrade the version numbers in the `operator/` directory as @periklis and team have a different process to upgrade the Operator version. - {{% /admonition %}} + {{< /admonition >}} diff --git a/docs/sources/get-started/deployment-modes.md b/docs/sources/get-started/deployment-modes.md index 60d3ccc74fbe..7094dc4deb72 100644 --- a/docs/sources/get-started/deployment-modes.md +++ b/docs/sources/get-started/deployment-modes.md @@ -32,9 +32,9 @@ Query parallelization is limited by the number of instances and the setting `max The simple scalable deployment is the default configuration installed by the [Loki Helm Chart]({{< relref "../setup/install/helm" >}}). This deployment mode is the easiest way to deploy Loki at scale. It strikes a balance between deploying in [monolithic mode](#monolithic-mode) or deploying each component as a [separate microservice](#microservices-mode). -{{% admonition type="note" %}} +{{< admonition type="note" >}} This deployment mode is sometimes referred to by the acronym SSD for simple scalable deployment, not to be confused with solid state drives. Loki uses an object store. -{{% /admonition %}} +{{< /admonition >}} Loki’s simple scalable deployment mode separates execution paths into read, write, and backend targets. These targets can be scaled independently, letting you customize your Loki deployment to meet your business needs for log ingestion and log query so that your infrastructure costs better match how you use Loki. @@ -79,13 +79,13 @@ For release 3.2 the components are: - Ruler - Table Manager (deprecated) -{{% admonition type="tip" %}} +{{< admonition type="tip" >}} You can see the complete list of targets for your version of Loki by running Loki with the flag `-list-targets`, for example: ```bash docker run docker.io/grafana/loki:3.2.1 -config.file=/etc/loki/local-config.yaml -list-targets ``` -{{% /admonition %}} +{{< /admonition >}} ![Microservices mode diagram](../microservices-mode.png "Microservices mode") diff --git a/docs/sources/get-started/labels/_index.md b/docs/sources/get-started/labels/_index.md index 96625b2b13ab..c9650befa70c 100644 --- a/docs/sources/get-started/labels/_index.md +++ b/docs/sources/get-started/labels/_index.md @@ -19,10 +19,10 @@ Labels in Loki perform a very important task: They define a stream. More specifi If you are familiar with Prometheus, the term used there is series; however, Prometheus has an additional dimension: metric name. Loki simplifies this in that there are no metric names, just labels, and we decided to use streams instead of series. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Structured metadata do not define a stream, but are metadata attached to a log line. See [structured metadata]({{< relref "./structured-metadata" >}}) for more information. -{{% /admonition %}} +{{< /admonition >}} ## Format diff --git a/docs/sources/get-started/labels/structured-metadata.md b/docs/sources/get-started/labels/structured-metadata.md index ac91f58e0c87..f1877139fef7 100644 --- a/docs/sources/get-started/labels/structured-metadata.md +++ b/docs/sources/get-started/labels/structured-metadata.md @@ -5,9 +5,9 @@ description: Describes how to enable structure metadata for logs and how to quer --- # What is structured metadata -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} Structured metadata was added to chunk format V4 which is used if the schema version is greater or equal to `13`. See [Schema Config](https://grafana.com/docs/loki//configure/storage/#schema-config) for more details about schema versions. -{{% /admonition %}} +{{< /admonition >}} Selecting proper, low cardinality labels is critical to operating and querying Loki effectively. Some metadata, especially infrastructure related metadata, can be difficult to embed in log lines, and is too high cardinality to effectively store as indexed labels (and therefore reducing performance of the index). @@ -36,7 +36,7 @@ See the [Promtail: Structured metadata stage](https://grafana.com/docs/loki//send-data/logstash/). -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} Structured metadata size is taken into account while asserting ingestion rate limiting. Along with that, there are separate limits on how much structured metadata can be attached per log line. ``` @@ -48,7 +48,7 @@ Along with that, there are separate limits on how much structured metadata can b # CLI flag: -limits.max-structured-metadata-entries-count [max_structured_metadata_entries_count: | default = 128] ``` -{{% /admonition %}} +{{< /admonition >}} ## Querying structured metadata diff --git a/docs/sources/operations/authentication.md b/docs/sources/operations/authentication.md index 11949a1a9811..44ef07c9bc39 100644 --- a/docs/sources/operations/authentication.md +++ b/docs/sources/operations/authentication.md @@ -18,11 +18,11 @@ A list of open-source reverse proxies you can use: - [OAuth2 proxy](https://github.com/oauth2-proxy/oauth2-proxy) - [HAProxy](https://www.haproxy.org/) -{{% admonition type="note" %}} +{{< admonition type="note" >}} When using Loki in multi-tenant mode, Loki requires the HTTP header `X-Scope-OrgID` to be set to a string identifying the tenant; the responsibility of populating this value should be handled by the authenticating reverse proxy. -For more information, read the [multi-tenancy]({{< relref "./multi-tenancy" >}}) documentation.{{% /admonition %}} +For more information, read the [multi-tenancy]({{< relref "./multi-tenancy" >}}) documentation.{{< /admonition >}} For information on authenticating Promtail, see the documentation for [how to configure Promtail]({{< relref "../send-data/promtail/configuration" >}}). diff --git a/docs/sources/operations/automatic-stream-sharding.md b/docs/sources/operations/automatic-stream-sharding.md index 04a46fbfe082..2e876c89197f 100644 --- a/docs/sources/operations/automatic-stream-sharding.md +++ b/docs/sources/operations/automatic-stream-sharding.md @@ -30,9 +30,9 @@ per-stream rate limit. ``` 1. Optionally enable `logging_enabled` for debugging stream sharding. - {{% admonition type="note" %}} + {{< admonition type="note" >}} This may affect the ingestion performance of Loki. - {{% /admonition %}} + {{< /admonition >}} ```yaml limits_config: diff --git a/docs/sources/operations/blocking-queries.md b/docs/sources/operations/blocking-queries.md index 1a88342bd3f1..ac286bbb3f84 100644 --- a/docs/sources/operations/blocking-queries.md +++ b/docs/sources/operations/blocking-queries.md @@ -35,9 +35,9 @@ overrides: - hash: 2943214005 # hash of {stream="stdout",pod="loki-canary-9w49x"} types: filter,limited ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} Changes to these configurations **do not require a restart**; they are defined in the [runtime configuration file](https://grafana.com/docs/loki//configure/#runtime-configuration-file). -{{% /admonition %}} +{{< /admonition >}} The available query types are: @@ -53,9 +53,9 @@ is logged with every query request in the `query-frontend` and `querier` logs, f level=info ts=2023-03-30T09:08:15.2614555Z caller=metrics.go:152 component=frontend org_id=29 latency=fast query="{stream=\"stdout\",pod=\"loki-canary-9w49x\"}" query_hash=2943214005 query_type=limited range_type=range ... ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} The order of patterns is preserved, so the first matching pattern will be used. -{{% /admonition %}} +{{< /admonition >}} ## Observing blocked queries diff --git a/docs/sources/operations/bloom-filters.md b/docs/sources/operations/bloom-filters.md index a62e67efbc90..7394e26ec4f0 100644 --- a/docs/sources/operations/bloom-filters.md +++ b/docs/sources/operations/bloom-filters.md @@ -12,9 +12,9 @@ aliases: # Bloom filters (Experimental) -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} This feature is an [experimental feature](/docs/release-life-cycle/). Engineering and on-call support is not available. No SLA is provided. -{{% /admonition %}} +{{< /admonition >}} Loki leverages [bloom filters](https://en.wikipedia.org/wiki/Bloom_filter) to speed up queries by reducing the amount of data Loki needs to load from the store and iterate through. Loki is often used to run "needle in a haystack" queries; these are queries where a large number of log lines are searched, but only a few log lines match the query. diff --git a/docs/sources/operations/recording-rules.md b/docs/sources/operations/recording-rules.md index cd7bdd2d301c..8c335740d5af 100644 --- a/docs/sources/operations/recording-rules.md +++ b/docs/sources/operations/recording-rules.md @@ -30,11 +30,11 @@ is that Prometheus will, for example, reject a remote-write request with 100 sam When the `ruler` starts up, it will load the WALs for the tenants who have recording rules. These WAL files are stored on disk and are loaded into memory. -{{% admonition type="note" %}} +{{< admonition type="note" >}} WALs are loaded one at a time upon start-up. This is a current limitation of the Loki ruler. For this reason, it is adviseable that the number of rule groups serviced by a ruler be kept to a reasonable size, since _no rule evaluation occurs while WAL replay is in progress (this includes alerting rules)_. -{{% /admonition %}} +{{< /admonition >}} ### Truncation @@ -56,10 +56,10 @@ excessively large due to truncation. See Mimir's guide for [configuring Grafana Mimir hash rings](/docs/mimir/latest/configure/configure-hash-rings/) for scaling the ruler using a ring. -{{% admonition type="note" %}} +{{< admonition type="note" >}} The `ruler` shards by rule _group_, not by individual rules. This is an artifact of the fact that Prometheus recording rules need to run in order since one recording rule can reuse another - but this is not possible in Loki. -{{% /admonition %}} +{{< /admonition >}} ## Deployment diff --git a/docs/sources/operations/scalability.md b/docs/sources/operations/scalability.md index f7117ff10347..1cc6e87d1264 100644 --- a/docs/sources/operations/scalability.md +++ b/docs/sources/operations/scalability.md @@ -66,9 +66,9 @@ this will result in far lower `ruler` resource usage because the majority of the The LogQL queries coming from the `ruler` will be executed against the given `query-frontend` service. Requests will be load-balanced across all `query-frontend` IPs if the `dns:///` prefix is used. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Queries that fail to execute are _not_ retried. -{{% /admonition %}} +{{< /admonition >}} ### Limits and Observability diff --git a/docs/sources/operations/storage/_index.md b/docs/sources/operations/storage/_index.md index 74fa7620d908..3b6bef94f9dc 100644 --- a/docs/sources/operations/storage/_index.md +++ b/docs/sources/operations/storage/_index.md @@ -77,9 +77,9 @@ See the [AWS deployment section](https://grafana.com/docs/loki//co ### DynamoDB -{{% admonition type="note" %}} +{{< admonition type="note" >}} DynamoDB support is deprecated and will be removed in a future release. -{{% /admonition %}} +{{< /admonition >}} When using DynamoDB for the index, the following permissions are needed: diff --git a/docs/sources/operations/storage/boltdb-shipper.md b/docs/sources/operations/storage/boltdb-shipper.md index 8fd7b9b085bf..b43b8967df62 100644 --- a/docs/sources/operations/storage/boltdb-shipper.md +++ b/docs/sources/operations/storage/boltdb-shipper.md @@ -6,19 +6,19 @@ weight: 200 --- # Single Store BoltDB (boltdb-shipper) -{{% admonition type="note" %}} +{{< admonition type="note" >}} Single store BoltDB Shipper is a legacy storage option recommended for Loki 2.0 through 2.7.x and is not recommended for new deployments. The [TSDB](https://grafana.com/docs/loki//operations/storage/tsdb/) is the recommended index for Loki 2.8 and newer. -{{% /admonition %}} +{{< /admonition >}} BoltDB Shipper lets you run Grafana Loki without any dependency on NoSQL stores for storing index. It locally stores the index in BoltDB files instead and keeps shipping those files to a shared object store i.e the same object store which is being used for storing chunks. It also keeps syncing BoltDB files from shared object store to a configured local directory for getting index entries created by other services of same Loki cluster. This helps run Loki with one less dependency and also saves costs in storage since object stores are likely to be much cheaper compared to cost of a hosted NoSQL store or running a self hosted instance of Cassandra. -{{% admonition type="note" %}} +{{< admonition type="note" >}} BoltDB shipper works best with 24h periodic index files. It is a requirement to have the index period set to 24h for either active or upcoming usage of boltdb-shipper. If boltdb-shipper already has created index files with 7 days period, and you want to retain previous data, add a new schema config using boltdb-shipper with a future date and index files period set to 24h. -{{% /admonition %}} +{{< /admonition >}} ## Example Configuration @@ -76,9 +76,9 @@ they both having shipped files for day `18371` and `18372` with prefix `loki_ind ... ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} Loki also adds a timestamp to names of the files to randomize the names to avoid overwriting files when running Ingesters with same name and not have a persistent storage. Timestamps not shown here for simplification. -{{% /admonition %}} +{{< /admonition >}} Let us talk about more in depth about how both Ingesters and Queriers work when running them with BoltDB Shipper. @@ -89,9 +89,9 @@ and the BoltDB Shipper looks for new and updated files in that directory at 1 mi When running Loki in microservices mode, there could be multiple ingesters serving write requests. Each ingester generates BoltDB files locally. -{{% admonition type="note" %}} +{{< admonition type="note" >}} To avoid any loss of index when an ingester crashes, we recommend running ingesters as a StatefulSet (when using Kubernetes) with a persistent storage for storing index files. -{{% /admonition %}} +{{< /admonition >}} When chunks are flushed, they are available for reads in the object store instantly. The index is not available instantly, since we upload every 15 minutes with the BoltDB shipper. Ingesters expose a new RPC for letting queriers query the ingester's local index for chunks which were recently flushed, but its index might not be available yet with queriers. @@ -137,9 +137,9 @@ While using `boltdb-shipper` avoid configuring WriteDedupe cache since it is use Compactor is a BoltDB Shipper specific service that reduces the index size by deduping the index and merging all the files to a single file per table. We recommend running a Compactor since a single Ingester creates 96 files per day which include a lot of duplicate index entries and querying multiple files per table adds up the overall query latency. -{{% admonition type="note" %}} +{{< admonition type="note" >}} There should be only one compactor instance running at a time that otherwise could create problems and may lead to data loss. -{{% /admonition %}} +{{< /admonition >}} Example compactor configuration with GCS: diff --git a/docs/sources/operations/storage/legacy-storage.md b/docs/sources/operations/storage/legacy-storage.md index 5ec085983365..e474d5ec4061 100644 --- a/docs/sources/operations/storage/legacy-storage.md +++ b/docs/sources/operations/storage/legacy-storage.md @@ -6,11 +6,11 @@ weight: 1000 --- # Legacy storage -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} The concepts described on this page are considered legacy and pre-date the single store storage introduced in Loki 2.0. The usage of legacy storage for new installations is highly discouraged and documentation is meant for informational purposes in case of upgrade to a single store. -{{% /admonition %}} +{{< /admonition >}} The **chunk store** is the Loki long-term data store, designed to support interactive querying and sustained writing without the need for background @@ -27,11 +27,11 @@ maintenance tasks. It consists of: - [Amazon S3](https://aws.amazon.com/s3) - [Google Cloud Storage](https://cloud.google.com/storage/) -{{% admonition type="note" %}} +{{< admonition type="note" >}} Unlike the other core components of Loki, the chunk store is not a separate service, job, or process, but rather a library embedded in the two services that need to access Loki data: the [ingester]({{< relref "../../get-started/components#ingester" >}}) and [querier]({{< relref "../../get-started/components#querier" >}}). -{{% /admonition %}} +{{< /admonition >}} The chunk store relies on a unified interface to the "[NoSQL](https://en.wikipedia.org/wiki/NoSQL)" stores (DynamoDB, Bigtable, and diff --git a/docs/sources/operations/storage/logs-deletion.md b/docs/sources/operations/storage/logs-deletion.md index eef99609ffd3..ddbb23d8d6dd 100644 --- a/docs/sources/operations/storage/logs-deletion.md +++ b/docs/sources/operations/storage/logs-deletion.md @@ -22,9 +22,9 @@ Log entry deletion relies on configuration of the custom logs retention workflow Enable log entry deletion by setting `retention_enabled` to true in the compactor's configuration and setting and `deletion_mode` to `filter-only` or `filter-and-delete` in the runtime config. `delete_request_store` also needs to be configured when retention is enabled to process delete requests, this determines the storage bucket that stores the delete requests. -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} Be very careful when enabling retention. It is strongly recommended that you also enable versioning on your objects in object storage to allow you to recover from accidental misconfiguration of a retention setting. If you want to enable deletion but not not want to enforce retention, configure the `retention_period` setting with a value of `0s`. -{{% /admonition %}} +{{< /admonition >}} Because it is a runtime configuration, `deletion_mode` can be set per-tenant, if desired. diff --git a/docs/sources/operations/storage/retention.md b/docs/sources/operations/storage/retention.md index d21110978638..2ef39b7a3b93 100644 --- a/docs/sources/operations/storage/retention.md +++ b/docs/sources/operations/storage/retention.md @@ -9,24 +9,24 @@ weight: 600 Retention in Grafana Loki is achieved through the [Compactor](#compactor). By default the `compactor.retention-enabled` flag is not set, so the logs sent to Loki live forever. -{{% admonition type="note" %}} +{{< admonition type="note" >}} If you have a lifecycle policy configured on the object store, please ensure that it is longer than the retention period. -{{% /admonition %}} +{{< /admonition >}} Granular retention policies to apply retention at per tenant or per stream level are also supported by the Compactor. -{{% admonition type="note" %}} +{{< admonition type="note" >}} The Compactor does not support retention on [legacy index types](https://grafana.com/docs/loki//configure/storage/#index-storage). Please use the [Table Manager](https://grafana.com/docs/loki//operations/storage/table-manager/) when using legacy index types. Both the Table manager and legacy index types are deprecated and may be removed in future major versions of Loki. -{{% /admonition %}} +{{< /admonition >}} ## Compactor The Compactor is responsible for compaction of index files and applying log retention. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Run the Compactor as a singleton (a single instance). -{{% /admonition %}} +{{< /admonition >}} The Compactor loops to apply compaction and retention at every `compactor.compaction-interval`, or as soon as possible if running behind. Both compaction and retention are idempotent. If the Compactor restarts, it will continue from where it left off. @@ -49,9 +49,9 @@ Chunks cannot be deleted immediately for the following reasons: - It provides a short window of time in which to cancel chunk deletion in the case of a configuration mistake. Marker files should be stored on a persistent disk to ensure that the chunks pending for deletion are processed even if the Compactor process restarts. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Grafana Labs recommends running Compactor as a stateful deployment (StatefulSet when using Kubernetes) with a persistent storage for storing marker files. -{{% /admonition %}} +{{< /admonition >}} ### Retention Configuration @@ -82,9 +82,9 @@ storage_config: bucket_name: loki ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} Retention is only available if the index period is 24h. Single store TSDB and single store BoltDB require 24h index period. -{{% /admonition %}} +{{< /admonition >}} `retention_enabled` should be set to true. Without this, the Compactor will only compact tables. @@ -107,9 +107,9 @@ There are two ways of setting retention policies: - `retention_period` which is applied globally for all log streams. - `retention_stream` which is only applied to log streams matching the selector. -{{% admonition type="note" %}} +{{< admonition type="note" >}} The minimum retention period is 24h. -{{% /admonition %}} +{{< /admonition >}} This example configures global retention that applies to all tenants (unless overridden by configuring per-tenant overrides): @@ -125,9 +125,9 @@ limits_config: ... ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} You can only use label matchers in the `selector` field of a `retention_stream` definition. Arbitrary LogQL expressions are not supported. -{{% /admonition %}} +{{< /admonition >}} Per tenant retention can be defined by configuring [runtime overrides](https://grafana.com/docs/loki//configure/#runtime-configuration-file). For example: @@ -156,9 +156,9 @@ Retention period for a given stream is decided based on the first match in this 4. The global `retention_period` will be applied if none of the above match. 5. If no global `retention_period` is specified, the default value of `744h` (30days) retention is used. -{{% admonition type="note" %}} +{{< admonition type="note" >}} The larger the priority value, the higher the priority. -{{% /admonition %}} +{{< /admonition >}} Stream matching uses the same syntax as Prometheus label matching: @@ -194,16 +194,16 @@ Alternatively, the `table-manager.retention-period` and provided retention period needs to be a duration represented as a string that can be parsed using the Prometheus common model [ParseDuration](https://pkg.go.dev/github.com/prometheus/common/model#ParseDuration). Examples: `7d`, `1w`, `168h`. -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} The retention period must be a multiple of the index and chunks table `period`, configured in the [`period_config`](https://grafana.com/docs/loki//configure/#period_config) block. See the [Table Manager](https://grafana.com/docs/loki//operations/storage/table-manager/#retention) documentation for more information. -{{% /admonition %}} +{{< /admonition >}} -{{% admonition type="note" %}} +{{< admonition type="note" >}} To avoid querying of data beyond the retention period,`max_query_lookback` config in [`limits_config`](https://grafana.com/docs/loki//configure/#limits_config) must be set to a value less than or equal to what is set in `table_manager.retention_period`. -{{% /admonition %}} +{{< /admonition >}} When using S3 or GCS, the bucket storing the chunks needs to have the expiry policy set correctly. For more details check diff --git a/docs/sources/operations/storage/table-manager/_index.md b/docs/sources/operations/storage/table-manager/_index.md index 6016b6bb12b9..5e3061cc9f8f 100644 --- a/docs/sources/operations/storage/table-manager/_index.md +++ b/docs/sources/operations/storage/table-manager/_index.md @@ -6,9 +6,9 @@ weight: --- # Table manager -{{% admonition type="note" %}} +{{< admonition type="note" >}} Table manager is only needed if you are using a multi-store [backend](https://grafana.com/docs/loki//configure/storage/). If you are using either TSDB (recommended), or BoltDB (deprecated) you do not need the Table Manager. -{{% /admonition %}} +{{< /admonition >}} Grafana Loki supports storing indexes and chunks in table-based data storages. When such a storage type is used, multiple tables are created over the time: each @@ -141,11 +141,11 @@ number_of_tables_to_keep = floor(retention_period / table_period) + 1 {{< figure alt="retention" align="center" src="./table-manager-retention.png" >}} -{{% admonition type="note" %}} +{{< admonition type="note" >}} It's important to note that - due to the internal implementation - the table `period` and `retention_period` **must** be multiples of `24h` in order to get the expected behavior. -{{% /admonition %}} +{{< /admonition >}} For detailed information on configuring the retention, refer to the [Loki Storage Retention]({{< relref "../retention" >}}) diff --git a/docs/sources/query/log_queries/_index.md b/docs/sources/query/log_queries/_index.md index 5e429b2f3b86..52650aa75329 100644 --- a/docs/sources/query/log_queries/_index.md +++ b/docs/sources/query/log_queries/_index.md @@ -661,9 +661,9 @@ the result will be The `| keep` expression will keep only the specified labels in the pipeline and drop all the other labels. -{{% admonition type="note" %}} +{{< admonition type="note" >}} The keep stage will not drop the __error__ or __error_details__ labels added by Loki at query time. To drop these labels, refer to [drop](#drop-labels-expression) stage. -{{% /admonition %}} +{{< /admonition >}} Query examples: diff --git a/docs/sources/query/logcli.md b/docs/sources/query/logcli.md index ee282ac73a6d..97f5f9f3d5ad 100644 --- a/docs/sources/query/logcli.md +++ b/docs/sources/query/logcli.md @@ -70,11 +70,11 @@ without needing a username and password: export LOKI_ADDR=http://localhost:3100 ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} If you are running Loki behind a proxy server and you have authentication configured, you will also have to pass in LOKI_USERNAME and LOKI_PASSWORD, LOKI_BEARER_TOKEN or LOKI_BEARER_TOKEN_FILE accordingly. -{{% /admonition %}} +{{< /admonition >}} ```bash $ logcli labels job @@ -860,9 +860,9 @@ You can consume log lines from your `stdin` instead of Loki servers. Say you have log files in your local, and just want to do run some LogQL queries for that, `--stdin` flag can help. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Currently it doesn't support any type of metric queries. -{{% /admonition %}} +{{< /admonition >}} You may have to use `stdin` flag for several reasons 1. Quick way to check and validate a LogQL expressions. diff --git a/docs/sources/query/query_accceleration.md b/docs/sources/query/query_accceleration.md index ab377b828243..48998fffb9dd 100644 --- a/docs/sources/query/query_accceleration.md +++ b/docs/sources/query/query_accceleration.md @@ -10,9 +10,9 @@ keywords: # Query acceleration (Experimental) -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} Query acceleration using blooms is an [experimental feature](/docs/release-life-cycle/). Engineering and on-call support is not available. No SLA is provided. -{{% /admonition %}} +{{< /admonition >}} If [bloom filters][] are enabled, you can write LogQL queries using [structured metadata][] to benefit from query acceleration. diff --git a/docs/sources/reference/loki-http-api.md b/docs/sources/reference/loki-http-api.md index 55d724160abd..dbf0d38a0aed 100644 --- a/docs/sources/reference/loki-http-api.md +++ b/docs/sources/reference/loki-http-api.md @@ -12,10 +12,10 @@ weight: 500 Loki exposes an HTTP API for pushing, querying, and tailing log data, as well as for viewing and managing cluster information. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Note that authorization is not part of the Loki API. Authorization needs to be done separately, for example, using an open-source load-balancer such as NGINX. -{{% /admonition %}} +{{< /admonition >}} ## Endpoints @@ -30,9 +30,9 @@ A [list of clients]({{< relref "../send-data" >}}) can be found in the clients d ### Query endpoints -{{% admonition type="note" %}} +{{< admonition type="note" >}} Requests sent to the query endpoints must use valid LogQL syntax. For more information, see the [LogQL]({{< relref "../query" >}}) section of the documentation. -{{% /admonition %}} +{{< /admonition >}} These HTTP endpoints are exposed by the `querier`, `query-frontend`, `read`, and `all` components: @@ -112,10 +112,10 @@ These HTTP endpoints are exposed by all individual components: ### Deprecated endpoints -{{% admonition type="note" %}} +{{< admonition type="note" >}} The following endpoints are deprecated.While they still exist and work, they should not be used for new deployments. Existing deployments should upgrade to use the supported endpoints. -{{% /admonition %}} +{{< /admonition >}} | Deprecated | Replacement | | ---------- | ----------- | @@ -154,9 +154,9 @@ The API accepts several formats for timestamps: - A floating point number is a Unix timestamp with fractions of a second. - A string in `RFC3339` and `RFC3339Nano` format, as supported by Go's [time](https://pkg.go.dev/time) package. -{{% admonition type="note" %}} +{{< admonition type="note" >}} When using `/api/v1/push`, you must send the timestamp as a string and not a number, otherwise the endpoint will return a 400 error. -{{% /admonition %}} +{{< /admonition >}} ### Statistics @@ -1440,9 +1440,9 @@ Query parameters: - `request_id=`: Identifies the delete request to cancel; IDs are found using the `delete` endpoint. - `force=`: When the `force` query parameter is true, partially completed delete requests will be canceled. - {{% admonition type="note" %}} + {{< admonition type="note" >}} some data from the request may still be deleted and the deleted request will be listed as 'processed'. - {{% /admonition %}} + {{< /admonition >}} A 204 response indicates success. diff --git a/docs/sources/release-notes/cadence.md b/docs/sources/release-notes/cadence.md index ef6fbcaf072f..1059fb7f95cf 100644 --- a/docs/sources/release-notes/cadence.md +++ b/docs/sources/release-notes/cadence.md @@ -15,12 +15,12 @@ naming scheme: `MAJOR`.`MINOR`.`PATCH`. - `MINOR` (roughly once a quarter): these releases include new features which generally do not break backwards-compatibility, but from time to time we might introduce _minor_ breaking changes, and we will specify these in our upgrade docs. - `PATCH` (roughly once or twice a month): these releases include bug and security fixes which do not break backwards-compatibility. -{{% admonition type="note" %}} +{{< admonition type="note" >}} While our naming scheme resembles [Semantic Versioning](https://semver.org/), at this time we do not strictly follow its guidelines to the letter. Our goal is to provide regular releases that are as stable as possible, and we take backwards-compatibility seriously. As with any software, always read the [release notes](https://grafana.com/docs/loki//release-notes/) and the [upgrade guide](https://grafana.com/docs/loki//setup/upgrade/) whenever choosing a new version of Loki to install. -{{% /admonition %}} +{{< /admonition >}} New releases are based of a [weekly release](#weekly-releases) which we have vetted for stability over a number of weeks. diff --git a/docs/sources/release-notes/v2-3.md b/docs/sources/release-notes/v2-3.md index 49ae7c76a798..87aff23ed0e9 100644 --- a/docs/sources/release-notes/v2-3.md +++ b/docs/sources/release-notes/v2-3.md @@ -78,8 +78,8 @@ List of security fixes for 2.3.x. * [4020](https://github.com/grafana/loki/pull/4020) **simonswine**: Restrict path segments in TenantIDs (CVE-2021-36156 CVE-2021-36157). -{{% admonition type="note" %}} -Exploitation of this vulnerability requires the ability for an attacker to craft and send directly to Loki an `X-Scope-OrgID` header, end users should not have the ability to create and send this header directly to Loki as it controls access to tenants and is important to control setting of this header for proper tenant isolation and security. We always recommend having a proxy or gateway be responsible for setting the `X-Scope-OrgID`.{{% /admonition %}} +{{< admonition type="note" >}} +Exploitation of this vulnerability requires the ability for an attacker to craft and send directly to Loki an `X-Scope-OrgID` header, end users should not have the ability to create and send this header directly to Loki as it controls access to tenants and is important to control setting of this header for proper tenant isolation and security. We always recommend having a proxy or gateway be responsible for setting the `X-Scope-OrgID`.{{< /admonition >}} ## Bug fixes diff --git a/docs/sources/send-data/_index.md b/docs/sources/send-data/_index.md index 0ef9432d3caf..411f0f436704 100644 --- a/docs/sources/send-data/_index.md +++ b/docs/sources/send-data/_index.md @@ -37,9 +37,9 @@ For more information, see [Ingesting logs to Loki using OpenTelemetry Collector] The following clients have been developed by the Loki community or other third-parties and can be used to send log data to Loki. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Grafana Labs cannot provide support for third-party clients. Once an issue has been determined to be with the client and not Loki, it is the responsibility of the customer to work with the associated vendor or project for bug fixes to these clients. -{{% /admonition %}} +{{< /admonition >}} The following are popular third-party Loki clients: diff --git a/docs/sources/send-data/docker-driver/_index.md b/docs/sources/send-data/docker-driver/_index.md index f262829d58fd..092857732c80 100644 --- a/docs/sources/send-data/docker-driver/_index.md +++ b/docs/sources/send-data/docker-driver/_index.md @@ -12,9 +12,9 @@ Grafana Loki officially supports a Docker plugin that will read logs from Docker containers and ship them to Loki. The plugin can be configured to send the logs to a private Loki instance or [Grafana Cloud](/oss/loki). -{{% admonition type="note" %}} +{{< admonition type="note" >}} Docker plugins are not supported on Windows; see the [Docker Engine managed plugin system](https://docs.docker.com/engine/extend) documentation for more information. -{{% /admonition %}} +{{< /admonition >}} Documentation on configuring the Loki Docker Driver can be found on the [configuration page]({{< relref "./configuration" >}}). @@ -58,9 +58,9 @@ docker plugin upgrade loki grafana/loki-docker-driver:2.9.2 --grant-all-permissi docker plugin enable loki systemctl restart docker ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} Update the version number to the appropriate version. -{{% /admonition %}} +{{< /admonition >}} ## Uninstall the Docker driver client diff --git a/docs/sources/send-data/docker-driver/configuration.md b/docs/sources/send-data/docker-driver/configuration.md index afa982b0b57c..b2038c97335d 100644 --- a/docs/sources/send-data/docker-driver/configuration.md +++ b/docs/sources/send-data/docker-driver/configuration.md @@ -33,11 +33,11 @@ docker run --log-driver=loki \ --log-opt loki-batch-size=400 \ grafana/grafana ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} The Loki logging driver still uses the json-log driver in combination with sending logs to Loki, this is mainly useful to keep the `docker logs` command working. You can adjust file size and rotation using the respective log option `max-size` and `max-file`. Keep in mind that default values for these options are not taken from json-log configuration. You can deactivate this behavior by setting the log option `no-file` to true. -{{% /admonition %}} +{{< /admonition >}} ## Change the default logging driver @@ -65,11 +65,11 @@ Options for the logging driver can also be configured with `log-opts` in the } } ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} log-opt configuration options in daemon.json must be provided as > strings. Boolean and numeric values (such as the value for loki-batch-size in > the example above) must therefore be enclosed in quotes (`"`). -{{% /admonition %}} +{{< /admonition >}} After changing `daemon.json`, restart the Docker daemon for the changes to take effect. All **newly created** containers from that host will then send logs to Loki via the driver. @@ -104,9 +104,9 @@ docker-compose -f docker-compose.yaml up Once deployed, the Grafana service will send its logs to Loki. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Stack name and service name for each swarm service and project name and service name for each compose service are automatically discovered and sent as Loki labels, this way you can filter by them in Grafana. -{{% /admonition %}} +{{< /admonition >}} ## Labels @@ -150,9 +150,9 @@ services: - "3000:3000" ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} Note the `loki-pipeline-stages: |` letting you keep the indentation correct. -{{% /admonition %}} +{{< /admonition >}} When using docker run you can also pass the value via a string parameter like such: diff --git a/docs/sources/send-data/fluentd/_index.md b/docs/sources/send-data/fluentd/_index.md index c712b0bb166b..35a682e92dff 100644 --- a/docs/sources/send-data/fluentd/_index.md +++ b/docs/sources/send-data/fluentd/_index.md @@ -71,9 +71,9 @@ services: ## Usage -{{% admonition type="note" %}} +{{< admonition type="note" >}} Use either `` or `extra_labels` to set at least one label. -{{% /admonition %}} +{{< /admonition >}} In your Fluentd configuration, add `@type loki`. Additional configuration is optional. Default values would look like this: diff --git a/docs/sources/send-data/lambda-promtail/_index.md b/docs/sources/send-data/lambda-promtail/_index.md index 49b1795a7c30..4f2b41bd713d 100644 --- a/docs/sources/send-data/lambda-promtail/_index.md +++ b/docs/sources/send-data/lambda-promtail/_index.md @@ -99,9 +99,9 @@ Ephemeral jobs can quite easily run afoul of cardinality best practices. During For those using Cloudwatch and wishing to test out Loki in a low-risk way, this workflow allows piping Cloudwatch logs to Loki regardless of the event source (EC2, Kubernetes, Lambda, ECS, etc) without setting up a set of Promtail daemons across their infrastructure. However, running Promtail as a daemon on your infrastructure is the best-practice deployment strategy in the long term for flexibility, reliability, performance, and cost. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Propagating logs from Cloudwatch to Loki means you'll still need to _pay_ for Cloudwatch. -{{% /admonition %}} +{{< /admonition >}} ### VPC Flow logs @@ -165,9 +165,9 @@ Incoming logs can have seven special labels assigned to them which can be used i ### Promtail labels -{{% admonition type="note" %}} +{{< admonition type="note" >}} This section is relevant if running Promtail between lambda-promtail and the end Loki deployment and was used to circumvent `out of order` problems prior to the v2.4 Loki release which removed the ordering constraint. -{{% /admonition %}} +{{< /admonition >}} As stated earlier, this workflow moves the worst case stream cardinality from `number_of_log_streams` -> `number_of_log_groups` * `number_of_promtails`. For this reason, each Promtail must have a unique label attached to logs it processes (ideally via something like `--client.external-labels=promtail=${HOSTNAME}`) and it's advised to run a small number of Promtails behind a load balancer according to your throughput and redundancy needs. @@ -195,9 +195,9 @@ The provided Terraform and CloudFormation files are meant to cover the default u ## Example Promtail Config -{{% admonition type="note" %}} +{{< admonition type="note" >}} This should be run in conjunction with a Promtail-specific label attached, ideally via a flag argument like `--client.external-labels=promtail=${HOSTNAME}`. It will receive writes via the push-api on ports `3500` (http) and `3600` (grpc). -{{% /admonition %}} +{{< /admonition >}} ```yaml server: diff --git a/docs/sources/send-data/otel/_index.md b/docs/sources/send-data/otel/_index.md index 690df4f5b544..176cd0afd410 100644 --- a/docs/sources/send-data/otel/_index.md +++ b/docs/sources/send-data/otel/_index.md @@ -93,9 +93,9 @@ Since the OpenTelemetry protocol differs from the Loki storage model, here is ho - service.name - service.namespace - {{% admonition type="note" %}} + {{< admonition type="note" >}} Because Loki has a default limit of 15 index labels, we recommend storing only select resource attributes as index labels. Although the default config selects more than 15 Resource Attributes, it should be fine since a few are mutually exclusive. - {{% /admonition %}} + {{< /admonition >}} - Timestamp: One of `LogRecord.TimeUnixNano` or `LogRecord.ObservedTimestamp`, based on which one is set. If both are not set, the ingestion timestamp will be used. diff --git a/docs/sources/send-data/promtail/configuration.md b/docs/sources/send-data/promtail/configuration.md index 32e4d01a70e3..0cd4d4eccc43 100644 --- a/docs/sources/send-data/promtail/configuration.md +++ b/docs/sources/send-data/promtail/configuration.md @@ -73,12 +73,12 @@ ${VAR:-default_value} Where default_value is the value to use if the environment variable is undefined. -{{% admonition type="note" %}} +{{< admonition type="note" >}} With `expand-env=true` the configuration will first run through [envsubst](https://pkg.go.dev/github.com/drone/envsubst) which will replace double backslashes with single backslashes. Because of this every use of a backslash `\` needs to be replaced with a double backslash `\\`. -{{% /admonition %}} +{{< /admonition >}} ### Generic placeholders diff --git a/docs/sources/send-data/promtail/stages/drop.md b/docs/sources/send-data/promtail/stages/drop.md index 77b66020bb23..0c6c15a1feda 100644 --- a/docs/sources/send-data/promtail/stages/drop.md +++ b/docs/sources/send-data/promtail/stages/drop.md @@ -126,9 +126,9 @@ Would drop this log line: #### Drop old log lines -{{% admonition type="note" %}} +{{< admonition type="note" >}} For `older_than` to work, you must be using the [timestamp]({{< relref "./timestamp" >}}) stage to set the timestamp from the ingested log line _before_ applying the `drop` stage. -{{% /admonition %}} +{{< /admonition >}} Given the pipeline: diff --git a/docs/sources/send-data/promtail/stages/json.md b/docs/sources/send-data/promtail/stages/json.md index 6babe1f60700..79a4c2438fb8 100644 --- a/docs/sources/send-data/promtail/stages/json.md +++ b/docs/sources/send-data/promtail/stages/json.md @@ -134,7 +134,7 @@ The following key-value pairs would be created in the set of extracted data: - `stream`: `stderr` - `timestamp`: `2019-04-30T02:12:41.8443515` -{{% admonition type="note" %}} +{{< admonition type="note" >}} Referring to `grpc.stream` without the combination of double quotes wrapped in single quotes will not work properly. -{{% /admonition %}} +{{< /admonition >}} diff --git a/docs/sources/send-data/promtail/stages/match.md b/docs/sources/send-data/promtail/stages/match.md index e9351350f4b3..c76a3a8dc834 100644 --- a/docs/sources/send-data/promtail/stages/match.md +++ b/docs/sources/send-data/promtail/stages/match.md @@ -14,9 +14,9 @@ or drop entries when a log entry matches a configurable LogQL [stream selector]({{< relref "../../../query/log_queries#log-stream-selector" >}}) and [filter expressions]({{< relref "../../../query/log_queries#line-filter-expression" >}}). -{{% admonition type="note" %}} +{{< admonition type="note" >}} The filters do not include label filter expressions such as `| label == "foobar"`. -{{% /admonition %}} +{{< /admonition >}} ## Schema diff --git a/docs/sources/send-data/promtail/stages/structured_metadata.md b/docs/sources/send-data/promtail/stages/structured_metadata.md index 369ceecfba3f..129dd757c4a2 100644 --- a/docs/sources/send-data/promtail/stages/structured_metadata.md +++ b/docs/sources/send-data/promtail/stages/structured_metadata.md @@ -8,11 +8,11 @@ description: The 'structured_metadata' Promtail pipeline stage The `structured_metadata` stage is an action stage that takes data from the extracted map and modifies the [structured metadata]({{< relref "../../../get-started/labels/structured-metadata" >}}) that is sent to Loki with the log entry. -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} Structured metadata will be rejected by Loki unless you enable the `allow_structured_metadata` per tenant configuration (in the `limits_config`). Structured metadata was added to chunk format V4 which is used if the schema version is greater or equal to **13**. (See Schema Config for more details about schema versions. ) -{{% /admonition %}} +{{< /admonition >}} ## Schema diff --git a/docs/sources/setup/migrate/migrate-to-tsdb/_index.md b/docs/sources/setup/migrate/migrate-to-tsdb/_index.md index 5be15a42b22d..b67e19b37196 100644 --- a/docs/sources/setup/migrate/migrate-to-tsdb/_index.md +++ b/docs/sources/setup/migrate/migrate-to-tsdb/_index.md @@ -20,9 +20,9 @@ we strongly recommend migrating to TSDB. To begin the migration, add a new [period_config](https://grafana.com/docs/loki//configure/#period_config) entry in your [schema_config](https://grafana.com/docs/loki//configure/#schema_config). You can read more about schema config [here](https://grafana.com/docs/loki//configure/storage/#schema-config). -{{% admonition type="note" %}} +{{< admonition type="note" >}} You must roll out the new `period_config` change to all Loki components in order for it to take effect. -{{% /admonition %}} +{{< /admonition >}} This example adds a new `period_config` which configures Loki to start using the TSDB index for the data ingested starting from `2023-10-20`. diff --git a/docs/sources/setup/upgrade/_index.md b/docs/sources/setup/upgrade/_index.md index 07cb5863c4db..ae4a80e8909a 100644 --- a/docs/sources/setup/upgrade/_index.md +++ b/docs/sources/setup/upgrade/_index.md @@ -48,10 +48,10 @@ Loki changes the default value of `-ruler.alertmanager-use-v2` from `false` to ` ### Experimental Bloom Filters -{{% admonition type="note" %}} +{{< admonition type="note" >}} Experimental features are subject to rapid change and/or removal, which can introduce breaking changes even between minor version. They also don't follow the deprecation lifecycle of regular features. -{{% /admonition %}} +{{< /admonition >}} The bloom compactor component, which builds bloom filter blocks for query acceleration, has been removed in favor of two new components: bloom planner and bloom builder. Please consult the [Query Acceleration with Blooms](https://grafana.com/docs/loki//operations/query-acceleration-blooms/) docs for more information. @@ -78,11 +78,11 @@ All other CLI arguments (and their YAML counterparts) prefixed with `-bloom-comp ## 3.0.0 -{{% admonition type="note" %}} +{{< admonition type="note" >}} If you have questions about upgrading to Loki 3.0, please join us on the [community Slack](https://slack.grafana.com/) in the `#loki-3` channel. Or leave a comment on this [Github Issue](https://github.com/grafana/loki/issues/12506). -{{% /admonition %}} +{{< /admonition >}} {{< admonition type="tip" >}} If you have not yet [migrated to TSDB](https://grafana.com/docs/loki//setup/migrate/migrate-to-tsdb/), do so before you upgrade to Loki 3.0. @@ -165,11 +165,11 @@ This enforces chunks and index files to reside together in the same storage buck We are removing the shared store setting in an effort to simplify storage configuration and reduce the possibility for misconfiguration. -{{% admonition type="warning" %}} +{{< admonition type="warning" >}} With this change Loki no longer allows storing chunks and indexes for a given period in different storage buckets. This is a breaking change for setups that store chunks and indexes in different storage buckets by setting `-boltdb.shipper.shared-store` or `-tsdb.shipper.shared-store` to a value different from `object_store` in `period_config`. -{{% /admonition %}} +{{< /admonition >}} - If you have not configured `-boltdb.shipper.shared-store`,`-tsdb.shipper.shared-store` or their corresponding YAML setting before, no changes are required as part of the upgrade. - If you have configured `-boltdb.shipper.shared-store` or its YAML setting: @@ -194,9 +194,9 @@ period_config: period: 24h ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} `path_prefix` only applies to TSDB and BoltDB indexes. This setting has no effect on [legacy indexes](https://grafana.com/docs/loki//configure/storage/#index-storage). -{{% /admonition %}} +{{< /admonition >}} `path_prefix` defaults to `index/` which is same as the default value of the removed configurations. @@ -290,7 +290,7 @@ This new metric will provide a more clear signal that there is an issue with ing #### Changes to default configuration values in 3.0 -{{% responsive-table %}} +{{< responsive-table >}} | configuration | new default | old default | notes | | ------------------------------------------------------ | ----------- | ----------- | -------- | `compactor.delete-max-interval` | 24h | 0 | splits the delete requests into intervals no longer than `delete_max_interval` | @@ -310,7 +310,7 @@ This new metric will provide a more clear signal that there is an issue with ing | `query-scheduler.max-outstanding-requests-per-tenant` | 32000 | 100 | - | | `validation.max-label-names-per-series` | 15 | 30 | - | | `legacy-read-mode` | false | true | Deprecated. It will be removed in the next minor release. | -{{% /responsive-table %}} +{{< /responsive-table >}} #### Automatic stream sharding is enabled by default @@ -529,10 +529,10 @@ limits_config: retention_period: 744h ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} In previous versions, the zero value of `0` or `0s` will result in **immediate deletion of all logs**, only in 2.8 and forward releases does the zero value disable retention. -{{% /admonition %}} +{{< /admonition >}} #### metrics.go log line `subqueries` replaced with `splits` and `shards` @@ -546,9 +546,9 @@ In 2.8 we no longer include `subqueries` in metrics.go, it does still exist in t Instead, now you can use `splits` to see how many split by time intervals were created and `shards` to see the total number of shards created for a query. -{{% admonition type="note" %}} +{{< admonition type="note" >}} Currently not every query can be sharded and a shards value of zero is a good indicator the query was not able to be sharded. -{{% /admonition %}} +{{< /admonition >}} ### Promtail @@ -579,9 +579,9 @@ ruler: #### query-frontend Kubernetes headless service changed to load balanced service -{{% admonition type="note" %}} +{{< admonition type="note" >}} This is relevant only if you are using [jsonnet for deploying Loki in Kubernetes](/docs/loki//installation/tanka/). -{{% /admonition %}} +{{< /admonition >}} The `query-frontend` Kubernetes service was previously headless and was used for two purposes: * Distributing the Loki query requests amongst all the available Query Frontend pods. @@ -1114,9 +1114,9 @@ In Loki 2.2 we changed the internal version of our chunk format from v2 to v3, t This makes it important to first upgrade to 2.0, 2.0.1, or 2.1 **before** upgrading to 2.2 so that if you need to rollback for any reason you can do so easily. -{{% admonition type="note" %}} +{{< admonition type="note" >}} 2.0 and 2.0.1 are identical in every aspect except 2.0.1 contains the code necessary to read the v3 chunk format. Therefor if you are on 2.0 and ugrade to 2.2, if you want to rollback, you must rollback to 2.0.1. -{{% /admonition %}} +{{< /admonition >}} ### Loki Config @@ -1260,14 +1260,14 @@ This likely only affects a small portion of tanka users because the default sche } ``` -{{% admonition type="note" %}} +{{< admonition type="note" >}} If you had set `index_period_hours` to a value other than 168h (the previous default) you must update this in the above config `period:` to match what you chose. -{{% /admonition %}} +{{< /admonition >}} -{{% admonition type="note" %}} +{{< admonition type="note" >}} We have changed the default index store to `boltdb-shipper` it's important to add `using_boltdb_shipper: false,` until you are ready to change (if you want to change) -{{% /admonition %}} +{{< /admonition >}} Changing the jsonnet config to use the `boltdb-shipper` type is the same as [below](#upgrading-schema-to-use-boltdb-shipper-andor-v11-schema) where you need to add a new schema section. @@ -1309,9 +1309,9 @@ _THIS BEING SAID_ we are not expecting problems, our testing so far has not unco Report any problems via GitHub issues or reach us on the #loki slack channel. -{{% admonition type="note" %}} +{{< admonition type="note" >}} If are using boltdb-shipper and were running with high availability and separate filesystems, this was a poorly documented and even more experimental mode we toyed with using boltdb-shipper. For now we removed the documentation and also any kind of support for this mode. -{{% /admonition %}} +{{< /admonition >}} To use boltdb-shipper in 2.0 you need a shared storage (S3, GCS, etc), the mode of running with separate filesystem stores in HA using a ring is not officially supported. @@ -1454,9 +1454,9 @@ schema_config: ``` If you are not on `schema: v11` this would be a good opportunity to make that change _in the new schema config_ also. -{{% admonition type="note" %}} +{{< admonition type="note" >}} If the current time in your timezone is after midnight UTC already, set the date one additional day forward. -{{% /admonition %}} +{{< /admonition >}} There was also a significant overhaul to how boltdb-shipper internals, this should not be visible to a user but as this feature is experimental and under development bug are possible! @@ -1515,9 +1515,9 @@ Defaulting to `gcs,bigtable` was confusing for anyone using ksonnet with other s ## 1.5.0 -{{% admonition type="note" %}} +{{< admonition type="note" >}} The required upgrade path outlined for version 1.4.0 below is still true for moving to 1.5.0 from any release older than 1.4.0 (e.g. 1.3.0 -> 1.5.0 needs to also look at the 1.4.0 upgrade requirements). -{{% /admonition %}} +{{< /admonition >}} ### Breaking config changes! @@ -1571,9 +1571,9 @@ Not every environment will allow this capability however, it's possible to restr #### Filesystem -{{% admonition type="note" %}} +{{< admonition type="note" >}} The location Loki is looking for files with the provided config in the docker image has changed. -{{% /admonition %}} +{{< /admonition >}} In 1.4.0 and earlier the included config file in the docker container was using directories: