From 6151ae8c8d37fe8496fc60171a3173343d9168bc Mon Sep 17 00:00:00 2001 From: Denis Date: Mon, 14 Aug 2023 15:21:48 +0200 Subject: [PATCH] Recommend debug logs while migrating from log to filestream input (#36302) All the migration-related logs are reported at the debug level. This makes it hard to investigate if something goes wrong and the user has not enabled the debug level. The migration guide is now updated to mention the need of debug-level logs. --- filebeat/docs/howto/migrate-to-filestream.asciidoc | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/filebeat/docs/howto/migrate-to-filestream.asciidoc b/filebeat/docs/howto/migrate-to-filestream.asciidoc index ce68033803e..a57105adb3e 100644 --- a/filebeat/docs/howto/migrate-to-filestream.asciidoc +++ b/filebeat/docs/howto/migrate-to-filestream.asciidoc @@ -97,6 +97,9 @@ defined `log` input, we need to add `take_over: true` to each new `filestream`. that the new `filestream` inputs will continue ingesting files from the same offset where the `log` inputs stopped. +NOTE: It's recommended to enable debug-level logs for Filebeat in order to follow the migration process. +After the first run with `take_over: true` the setting can be removed. + WARNING: The `take over` mode is in beta. IMPORTANT: If this parameter is not set, all the files will be re-ingested from the beginning @@ -104,6 +107,8 @@ and this will lead to data duplication. Please, double-check that this parameter [source,yaml] ---- +logging: + level: debug filebeat.inputs: - type: filestream enabled: true @@ -235,9 +240,10 @@ If for whatever reason you'd like to revert the configuration after running the and return to old `log` inputs the files that were taken by `filestream` inputs, you need to do the following: 1. Stop Filebeat as soon as possible -2. Find your <> -3. Find the created backup files, they have the `.bak` suffix. If you have multiple backups for the same file, choose the one with the more recent timestamp. -4. Replace the files with their backups, e.g. `log.json` should be replaced by `log.json-1674152412247684000.bak` -5. Run Filebeat with the old configuration (no `filestream` inputs with `take_over: true`). +2. Save its debug-level logs for further investigation +3. Find your <> +4. Find the created backup files, they have the `.bak` suffix. If you have multiple backups for the same file, choose the one with the more recent timestamp. +5. Replace the files with their backups, e.g. `log.json` should be replaced by `log.json-1674152412247684000.bak` +6. Run Filebeat with the old configuration (no `filestream` inputs with `take_over: true`). NOTE: Reverting to backups might cause some events to repeat, depends on the amount of time the new configuration was running.