diff --git a/doc/04-Upgrading.md b/doc/04-Upgrading.md index 5a085e027..7c683d8ac 100644 --- a/doc/04-Upgrading.md +++ b/doc/04-Upgrading.md @@ -3,6 +3,54 @@ Specific version upgrades are described below. Please note that version upgrades are incremental. If you are upgrading across multiple versions, make sure to follow the steps for each of them. +## Upgrading to Icinga DB v1.1.2 + +Please apply the `1.1.2.sql` upgrade script to your database. For package installations, you can find this file at +`/usr/share/icingadb/schema/mysql/upgrades/` or `/usr/share/icingadb/schema/pgsql/upgrades/`, depending on your +database vendor. + +!!! warning + + In order to fix a loading performance issue of the `icingadb/history` view, this upgrade script adds an appropriate + index on the `history` table. However, this may hang for a relatively considerable time and block all Icinga DB + queries on all `*_history` tables and the `history` table inclusively if there is an ongoing, long-running query + on the `history` table. One way of causing this to happen is if an Icinga Web user accesses the `icingadb/history` + view at the exact moment you are running this script. Depending on how many entries you have in the history table, + Icinga DB Web may take quite a long time to load, until your web servers timeout kicks in. + See [unblock history tables](#unblock-history-tables) for details. + +### Unblock History Tables + +When you observe that the upgrade script has been taking an unusually long to complete, you can perform the following +analysis on another console and unblock it if necessary. It is important to note though that the script may need some +time to perform the reindexing on the `history` table even if it is not blocked. Nonetheless, you can use the +`show processlist` command to determine whether an excessive number of queries have been stuck in a waiting state. + +``` +MariaDB [icingadb]> show processlist; ++------+-----+-----+----------+-----+------+---------------------------------+------------------------------------+-----+ +| Id | ... | ... | db | ... | Time | State | Info | ... | ++------+-----+-----+----------+-----+------+---------------------------------+------------------------------------+-----+ +| 1475 | ... | ... | icingadb | ... | 1222 | Waiting for table metadata lock | INSERT INTO "notification_history" | ... | +| 1485 | ... | ... | icingadb | ... | 1262 | Creating sort index | SELECT history.id, history.... | ... | +| 1494 | ... | ... | icingadb | ... | 1224 | Waiting for table metadata lock | ALTER TABLE history ADD INDEX ... | ... | ++------+-----+-----+----------+-----+------+---------------------------------+------------------------------------+-----+ +39 rows in set (0.001 sec) +``` + +In the above output you can see that there are approximately 39 queries, including the `ALTER TABLE history ADD INDEX` +query from the upgrade script, waiting for a metadata lock, I just minimised those to the bare essentials. +Unfortunately, only one of these queries is holding the `table metadata lock` that everyone else is now waiting for, +which in this case is a `SELECT` statement initiated by Icinga DB Web in the `icingadb/history` view, which takes an +unimaginably long time. + +You can now either just wait for the `SELECT` statement to finish by itself and let it block the upgrade script and all +Icinga DB queries on all `*_history` tables or forcibly terminate it and let the remaining queries do their work. +Cancelling that one blocking query will let the upgrade script continue normally without blocking any other queries. +``` +MariaDB [icingadb]> kill 1485; +``` + ## Upgrading to Icinga DB v1.1.1 Please apply the `1.1.1.sql` upgrade script to your database.