diff --git a/SOURCES/0001-backport-of-ccd121cc248d79b749a63d4ad099e6d5f4b8b588.patch b/SOURCES/0001-backport-of-ccd121cc248d79b749a63d4ad099e6d5f4b8b588.patch index 577319c..a2b7013 100644 --- a/SOURCES/0001-backport-of-ccd121cc248d79b749a63d4ad099e6d5f4b8b588.patch +++ b/SOURCES/0001-backport-of-ccd121cc248d79b749a63d4ad099e6d5f4b8b588.patch @@ -1,7 +1,7 @@ From 705a6d9deed2ba862577681fc54df144060f3816 Mon Sep 17 00:00:00 2001 From: Mark Syms Date: Thu, 20 May 2021 17:40:06 +0100 -Subject: [PATCH 01/23] backport of ccd121cc248d79b749a63d4ad099e6d5f4b8b588: +Subject: [PATCH 001/177] backport of ccd121cc248d79b749a63d4ad099e6d5f4b8b588: CA-354692: check for device parameter in create/probe calls Signed-off-by: Mark Syms diff --git a/SOURCES/0002-Update-xs-sm.service-s-description-for-XCP-ng.patch b/SOURCES/0002-Update-xs-sm.service-s-description-for-XCP-ng.patch index c4c897c..9196a6b 100644 --- a/SOURCES/0002-Update-xs-sm.service-s-description-for-XCP-ng.patch +++ b/SOURCES/0002-Update-xs-sm.service-s-description-for-XCP-ng.patch @@ -1,7 +1,7 @@ From fd898b82d880619f0acd9d1b8f1d55b3967bc339 Mon Sep 17 00:00:00 2001 From: Samuel Verschelde Date: Thu, 13 Aug 2020 15:22:17 +0200 -Subject: [PATCH 02/23] Update xs-sm.service's description for XCP-ng +Subject: [PATCH 002/177] Update xs-sm.service's description for XCP-ng This was a patch added to the sm RPM git repo before we had this forked git repo for sm in the xcp-ng github organisation. diff --git a/SOURCES/0003-Add-TrueNAS-multipath-config.patch b/SOURCES/0003-Add-TrueNAS-multipath-config.patch index 6dd0f7d..964e197 100644 --- a/SOURCES/0003-Add-TrueNAS-multipath-config.patch +++ b/SOURCES/0003-Add-TrueNAS-multipath-config.patch @@ -1,7 +1,7 @@ From bd709621897bda8d9f14820fb3d52eecfe51facb Mon Sep 17 00:00:00 2001 From: Samuel Verschelde Date: Thu, 13 Aug 2020 15:26:43 +0200 -Subject: [PATCH 03/23] Add TrueNAS multipath config +Subject: [PATCH 003/177] Add TrueNAS multipath config This was a patch added to the sm RPM git repo before we had this forked git repo for sm in the xcp-ng github organisation. diff --git a/SOURCES/0004-feat-drivers-add-CephFS-GlusterFS-and-XFS-drivers.patch b/SOURCES/0004-feat-drivers-add-CephFS-GlusterFS-and-XFS-drivers.patch index bf8bfd2..b9155b8 100644 --- a/SOURCES/0004-feat-drivers-add-CephFS-GlusterFS-and-XFS-drivers.patch +++ b/SOURCES/0004-feat-drivers-add-CephFS-GlusterFS-and-XFS-drivers.patch @@ -1,7 +1,7 @@ From 574897552a11af6fc92e145df0bacb42a2ac6b53 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Mon, 20 Jul 2020 16:26:42 +0200 -Subject: [PATCH 04/23] feat(drivers): add CephFS, GlusterFS and XFS drivers +Subject: [PATCH 004/177] feat(drivers): add CephFS, GlusterFS and XFS drivers --- Makefile | 3 + diff --git a/SOURCES/0005-feat-drivers-add-ZFS-driver-to-avoid-losing-VDI-meta.patch b/SOURCES/0005-feat-drivers-add-ZFS-driver-to-avoid-losing-VDI-meta.patch index f5f8db2..a72163d 100644 --- a/SOURCES/0005-feat-drivers-add-ZFS-driver-to-avoid-losing-VDI-meta.patch +++ b/SOURCES/0005-feat-drivers-add-ZFS-driver-to-avoid-losing-VDI-meta.patch @@ -1,7 +1,7 @@ From 3a948fe3ff75146c3a9c960768ca4791c109a6b6 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Wed, 12 Aug 2020 11:14:33 +0200 -Subject: [PATCH 05/23] feat(drivers): add ZFS driver to avoid losing VDI +Subject: [PATCH 005/177] feat(drivers): add ZFS driver to avoid losing VDI metadata (xcp-ng/xcp#401) --- diff --git a/SOURCES/0006-Re-add-the-ext4-driver-for-users-who-need-to-transit.patch b/SOURCES/0006-Re-add-the-ext4-driver-for-users-who-need-to-transit.patch index 02aa908..8e6b31c 100644 --- a/SOURCES/0006-Re-add-the-ext4-driver-for-users-who-need-to-transit.patch +++ b/SOURCES/0006-Re-add-the-ext4-driver-for-users-who-need-to-transit.patch @@ -1,7 +1,8 @@ From 8b1962f5d6d8b092525e94241df033e4a78872b1 Mon Sep 17 00:00:00 2001 From: Samuel Verschelde Date: Thu, 13 Aug 2020 17:10:12 +0200 -Subject: [PATCH 06/23] Re-add the ext4 driver for users who need to transition +Subject: [PATCH 006/177] Re-add the ext4 driver for users who need to + transition The driver is needed to transition to the ext driver. Users who upgrade from XCP-ng <= 8.0 need a working driver so that they diff --git a/SOURCES/0007-feat-drivers-add-LinstorSR-driver.patch b/SOURCES/0007-feat-drivers-add-LinstorSR-driver.patch index 044f0db..0922edc 100644 --- a/SOURCES/0007-feat-drivers-add-LinstorSR-driver.patch +++ b/SOURCES/0007-feat-drivers-add-LinstorSR-driver.patch @@ -1,7 +1,7 @@ From 9b836e9e503354796a7f975bb44f10a0920f33e3 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Mon, 16 Mar 2020 15:39:44 +0100 -Subject: [PATCH 07/23] feat(drivers): add LinstorSR driver +Subject: [PATCH 007/177] feat(drivers): add LinstorSR driver Some important points: diff --git a/SOURCES/0008-feat-tests-add-unit-tests-concerning-ZFS-close-xcp-n.patch b/SOURCES/0008-feat-tests-add-unit-tests-concerning-ZFS-close-xcp-n.patch index 72f9b4a..3423d8c 100644 --- a/SOURCES/0008-feat-tests-add-unit-tests-concerning-ZFS-close-xcp-n.patch +++ b/SOURCES/0008-feat-tests-add-unit-tests-concerning-ZFS-close-xcp-n.patch @@ -1,7 +1,7 @@ From c756b29ce80da0cd38f88bbaaddf4e1130962af9 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Tue, 27 Oct 2020 15:04:36 +0100 -Subject: [PATCH 08/23] feat(tests): add unit tests concerning ZFS (close +Subject: [PATCH 008/177] feat(tests): add unit tests concerning ZFS (close xcp-ng/xcp#425) - Check if "create" doesn't succeed without zfs packages diff --git a/SOURCES/0009-If-no-NFS-ACLs-provided-assume-everyone.patch b/SOURCES/0009-If-no-NFS-ACLs-provided-assume-everyone.patch index 99d794c..40daa32 100644 --- a/SOURCES/0009-If-no-NFS-ACLs-provided-assume-everyone.patch +++ b/SOURCES/0009-If-no-NFS-ACLs-provided-assume-everyone.patch @@ -1,7 +1,7 @@ From c4402ded6d0dd748435c621c7d7840cb7886bc3f Mon Sep 17 00:00:00 2001 From: BenjiReis Date: Thu, 25 Feb 2021 09:54:52 +0100 -Subject: [PATCH 09/23] If no NFS ACLs provided, assume everyone: +Subject: [PATCH 009/177] If no NFS ACLs provided, assume everyone: Some QNAP devices do not provide ACL when fetching NFS mounts. In this case the assumed ACL should be: "*". diff --git a/SOURCES/0010-Added-SM-Driver-for-MooseFS.patch b/SOURCES/0010-Added-SM-Driver-for-MooseFS.patch index ae390a5..3eeb282 100644 --- a/SOURCES/0010-Added-SM-Driver-for-MooseFS.patch +++ b/SOURCES/0010-Added-SM-Driver-for-MooseFS.patch @@ -1,7 +1,7 @@ From ba6e2bace64965d7d9f89e6dd9a94bf22ae075a2 Mon Sep 17 00:00:00 2001 From: Aleksander Wieliczko Date: Fri, 29 Jan 2021 15:21:23 +0100 -Subject: [PATCH 10/23] Added SM Driver for MooseFS +Subject: [PATCH 010/177] Added SM Driver for MooseFS Co-authored-by: Piotr Robert Konopelko Signed-off-by: Aleksander Wieliczko diff --git a/SOURCES/0011-Avoid-usage-of-umount-in-ISOSR-when-legacy_mode-is-u.patch b/SOURCES/0011-Avoid-usage-of-umount-in-ISOSR-when-legacy_mode-is-u.patch index 8183ea8..742662f 100644 --- a/SOURCES/0011-Avoid-usage-of-umount-in-ISOSR-when-legacy_mode-is-u.patch +++ b/SOURCES/0011-Avoid-usage-of-umount-in-ISOSR-when-legacy_mode-is-u.patch @@ -1,7 +1,7 @@ From dc7619735961b1fc0b7075cdc02ba375b76aa011 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Thu, 2 Dec 2021 09:28:37 +0100 -Subject: [PATCH 11/23] Avoid usage of `umount` in `ISOSR` when `legacy_mode` +Subject: [PATCH 011/177] Avoid usage of `umount` in `ISOSR` when `legacy_mode` is used `umount` should not be called when `legacy_mode` is enabled, otherwise a mounted dir diff --git a/SOURCES/0012-MooseFS-SR-uses-now-UUID-subdirs-for-each-SR.patch b/SOURCES/0012-MooseFS-SR-uses-now-UUID-subdirs-for-each-SR.patch index c24c4c2..b585471 100644 --- a/SOURCES/0012-MooseFS-SR-uses-now-UUID-subdirs-for-each-SR.patch +++ b/SOURCES/0012-MooseFS-SR-uses-now-UUID-subdirs-for-each-SR.patch @@ -1,7 +1,7 @@ From a1341e33251ec40a12e0971ffeda8b935af0be93 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Wed, 18 May 2022 17:28:09 +0200 -Subject: [PATCH 12/23] MooseFS SR uses now UUID subdirs for each SR +Subject: [PATCH 012/177] MooseFS SR uses now UUID subdirs for each SR A sm-config boolean param `subdir` is available to configure where to store the VHDs: - In a subdir with the SR UUID, the new behavior diff --git a/SOURCES/0013-Fix-is_open-call-for-many-drivers-25.patch b/SOURCES/0013-Fix-is_open-call-for-many-drivers-25.patch index 51705ab..08603e9 100644 --- a/SOURCES/0013-Fix-is_open-call-for-many-drivers-25.patch +++ b/SOURCES/0013-Fix-is_open-call-for-many-drivers-25.patch @@ -1,7 +1,7 @@ From 58c1e5a82d1b943a94a1524e52fd8bf43e916e3e Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Thu, 23 Jun 2022 10:36:36 +0200 -Subject: [PATCH 13/23] Fix is_open call for many drivers (#25) +Subject: [PATCH 013/177] Fix is_open call for many drivers (#25) Ensure all shared drivers are imported in `_is_open` definition to register them in the driver list. Otherwise this function always fails with a SRUnknownType exception. diff --git a/SOURCES/0014-Remove-SR_CACHING-capability-for-many-SR-types-24.patch b/SOURCES/0014-Remove-SR_CACHING-capability-for-many-SR-types-24.patch index 18b0d8f..26dba93 100644 --- a/SOURCES/0014-Remove-SR_CACHING-capability-for-many-SR-types-24.patch +++ b/SOURCES/0014-Remove-SR_CACHING-capability-for-many-SR-types-24.patch @@ -1,7 +1,7 @@ From 9d45a0e68acbcaec82a36c74f91dfa176b4fc7bf Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Thu, 23 Jun 2022 10:37:07 +0200 -Subject: [PATCH 14/23] Remove SR_CACHING capability for many SR types (#24) +Subject: [PATCH 014/177] Remove SR_CACHING capability for many SR types (#24) SR_CACHING offers the capacity to use IntelliCache, but this feature is only available using NFS SR. diff --git a/SOURCES/0015-Remove-SR_PROBE-from-ZFS-capabilities-37.patch b/SOURCES/0015-Remove-SR_PROBE-from-ZFS-capabilities-37.patch index adf250b..6495bd2 100644 --- a/SOURCES/0015-Remove-SR_PROBE-from-ZFS-capabilities-37.patch +++ b/SOURCES/0015-Remove-SR_PROBE-from-ZFS-capabilities-37.patch @@ -1,7 +1,7 @@ From ee2433b00bea733c8ec0d294ffb476e04e7e5bdb Mon Sep 17 00:00:00 2001 From: BenjiReis Date: Fri, 4 Aug 2023 12:10:08 +0200 -Subject: [PATCH 15/23] Remove `SR_PROBE` from ZFS capabilities (#37) +Subject: [PATCH 015/177] Remove `SR_PROBE` from ZFS capabilities (#37) The probe method is not implemented so we shouldn't advertise it. diff --git a/SOURCES/0016-Fix-vdi-ref-when-static-vdis-are-used.patch b/SOURCES/0016-Fix-vdi-ref-when-static-vdis-are-used.patch index c6fcd8a..a7d03f4 100644 --- a/SOURCES/0016-Fix-vdi-ref-when-static-vdis-are-used.patch +++ b/SOURCES/0016-Fix-vdi-ref-when-static-vdis-are-used.patch @@ -1,7 +1,7 @@ From a14e7dbb656f0c722e85cacf4ad658695e086502 Mon Sep 17 00:00:00 2001 From: Guillaume Date: Wed, 16 Aug 2023 13:42:21 +0200 -Subject: [PATCH 16/23] Fix vdi-ref when static vdis are used +Subject: [PATCH 016/177] Fix vdi-ref when static vdis are used When static vdis are used there is no snapshots and we don't want to call method from XAPI. diff --git a/SOURCES/0017-Tell-users-not-to-edit-multipath.conf-directly.patch b/SOURCES/0017-Tell-users-not-to-edit-multipath.conf-directly.patch index 0c51ba1..9c34993 100644 --- a/SOURCES/0017-Tell-users-not-to-edit-multipath.conf-directly.patch +++ b/SOURCES/0017-Tell-users-not-to-edit-multipath.conf-directly.patch @@ -1,7 +1,7 @@ From 1cf0c93a4e8c1de3688115954a8d4a8d4c25830d Mon Sep 17 00:00:00 2001 From: Samuel Verschelde Date: Fri, 27 Jan 2023 12:03:15 +0100 -Subject: [PATCH 17/23] Tell users not to edit multipath.conf directly +Subject: [PATCH 017/177] Tell users not to edit multipath.conf directly This file is meant to remain unchanged and regularly updated along with the SM component. Users can create a custom configuration file in diff --git a/SOURCES/0018-Add-custom.conf-multipath-configuration-file.patch b/SOURCES/0018-Add-custom.conf-multipath-configuration-file.patch index da6c3a1..3f7b1db 100644 --- a/SOURCES/0018-Add-custom.conf-multipath-configuration-file.patch +++ b/SOURCES/0018-Add-custom.conf-multipath-configuration-file.patch @@ -1,7 +1,7 @@ From c8e7d59b6c9b68f9e5ef89d0ad87c157bb35c736 Mon Sep 17 00:00:00 2001 From: Samuel Verschelde Date: Fri, 27 Jan 2023 12:23:13 +0100 -Subject: [PATCH 18/23] Add custom.conf multipath configuration file +Subject: [PATCH 018/177] Add custom.conf multipath configuration file Meant to be installed as /etc/multipath/conf.d/custom.conf for users to have an easy entry point for editing, as well as information on what diff --git a/SOURCES/0019-Install-etc-multipath-conf.d-custom.conf.patch b/SOURCES/0019-Install-etc-multipath-conf.d-custom.conf.patch index 4f9af7e..7828c76 100644 --- a/SOURCES/0019-Install-etc-multipath-conf.d-custom.conf.patch +++ b/SOURCES/0019-Install-etc-multipath-conf.d-custom.conf.patch @@ -1,7 +1,7 @@ From 81b847dca6a48d73936592820a1a7b88ae1ff5cc Mon Sep 17 00:00:00 2001 From: Samuel Verschelde Date: Fri, 25 Aug 2023 17:47:34 +0200 -Subject: [PATCH 19/23] Install /etc/multipath/conf.d/custom.conf +Subject: [PATCH 019/177] Install /etc/multipath/conf.d/custom.conf Update Makefile so that the file is installed along with sm. diff --git a/SOURCES/0020-Backport-NFS4-only-support.patch b/SOURCES/0020-Backport-NFS4-only-support.patch index 3b32047..617581e 100644 --- a/SOURCES/0020-Backport-NFS4-only-support.patch +++ b/SOURCES/0020-Backport-NFS4-only-support.patch @@ -1,7 +1,7 @@ From fe5a7b355ea6820036cfeef3847b2bec28567632 Mon Sep 17 00:00:00 2001 From: Benjamin Reis Date: Mon, 18 Dec 2023 10:22:26 +0100 -Subject: [PATCH 20/23] Backport NFS4 only support +Subject: [PATCH 020/177] Backport NFS4 only support See: https://github.com/xapi-project/sm/pull/617 diff --git a/SOURCES/0021-Backport-probe-for-NFS4-when-rpcinfo-does-not-includ.patch b/SOURCES/0021-Backport-probe-for-NFS4-when-rpcinfo-does-not-includ.patch index 44e6b03..cb61536 100644 --- a/SOURCES/0021-Backport-probe-for-NFS4-when-rpcinfo-does-not-includ.patch +++ b/SOURCES/0021-Backport-probe-for-NFS4-when-rpcinfo-does-not-includ.patch @@ -1,7 +1,7 @@ From eea13526d4e202e395d1a4289f837b0171a73b8e Mon Sep 17 00:00:00 2001 From: Benjamin Reis Date: Mon, 18 Dec 2023 10:35:46 +0100 -Subject: [PATCH 21/23] Backport probe for NFS4 when rpcinfo does not include +Subject: [PATCH 021/177] Backport probe for NFS4 when rpcinfo does not include it See: https://github.com/xapi-project/sm/pull/655 diff --git a/SOURCES/0022-feat-LargeBlock-backport-of-largeblocksr-51-55.patch b/SOURCES/0022-feat-LargeBlock-backport-of-largeblocksr-51-55.patch index 5951274..325761b 100644 --- a/SOURCES/0022-feat-LargeBlock-backport-of-largeblocksr-51-55.patch +++ b/SOURCES/0022-feat-LargeBlock-backport-of-largeblocksr-51-55.patch @@ -1,7 +1,8 @@ From 05e5ce03d6fd4523a942345cf49cc41eebaa78f7 Mon Sep 17 00:00:00 2001 From: Damien Thenot Date: Tue, 7 May 2024 15:20:22 +0200 -Subject: [PATCH 22/23] feat(LargeBlock): backport of largeblocksr (#51) (#55) +Subject: [PATCH 022/177] feat(LargeBlock): backport of largeblocksr (#51) + (#55) A SR inheriting from a EXTSR allowing to use a 4KiB blocksize device as SR. diff --git a/SOURCES/0023-feat-LVHDSR-add-a-way-to-modify-config-of-LVMs-56.patch b/SOURCES/0023-feat-LVHDSR-add-a-way-to-modify-config-of-LVMs-56.patch index 7b2a623..0b71988 100644 --- a/SOURCES/0023-feat-LVHDSR-add-a-way-to-modify-config-of-LVMs-56.patch +++ b/SOURCES/0023-feat-LVHDSR-add-a-way-to-modify-config-of-LVMs-56.patch @@ -1,7 +1,8 @@ From bc8ee2e18aa6ba1a8923c09ff0d765f931ddc348 Mon Sep 17 00:00:00 2001 From: Ronan Abhamon Date: Tue, 28 May 2024 15:17:21 +0200 -Subject: [PATCH 23/23] feat(LVHDSR): add a way to modify config of LVMs (#56) +Subject: [PATCH 023/177] feat(LVHDSR): add a way to modify config of LVMs + (#56) With this change the driver supports a "lvm-conf" param on "other-config". For now The configuration is only used by "remove" calls from LVMCache. diff --git a/SOURCES/0024-Fix-timeout_call-alarm-must-be-reset-in-case-of-succ.patch b/SOURCES/0024-Fix-timeout_call-alarm-must-be-reset-in-case-of-succ.patch new file mode 100644 index 0000000..fed5680 --- /dev/null +++ b/SOURCES/0024-Fix-timeout_call-alarm-must-be-reset-in-case-of-succ.patch @@ -0,0 +1,29 @@ +From f76ccb66f769ca352bf1dea9abdaaea5ec1b7942 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 16 Feb 2022 18:24:56 +0100 +Subject: [PATCH 024/177] Fix timeout_call: alarm must be reset in case of + success + +Otherwise the SIGALRM signal can be emitted after the execution +of the given user function. + +Signed-off-by: Ronan Abhamon +--- + drivers/util.py | 3 +-- + 1 file changed, 1 insertion(+), 2 deletions(-) + +diff --git a/drivers/util.py b/drivers/util.py +index 8bd36351..77f3f190 100755 +--- a/drivers/util.py ++++ b/drivers/util.py +@@ -852,9 +852,8 @@ def timeout_call(timeoutseconds, function, *arguments): + signal.alarm(timeoutseconds) + try: + function(*arguments) +- except: ++ finally: + signal.alarm(0) +- raise + + + def _incr_iscsiSR_refcount(targetIQN, uuid): diff --git a/SOURCES/0025-timeout_call-returns-the-result-of-user-function-now.patch b/SOURCES/0025-timeout_call-returns-the-result-of-user-function-now.patch new file mode 100644 index 0000000..d984c17 --- /dev/null +++ b/SOURCES/0025-timeout_call-returns-the-result-of-user-function-now.patch @@ -0,0 +1,23 @@ +From 3db6e67815f3c85768cf253e3d78d01c416fc16f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 16 Feb 2022 18:28:06 +0100 +Subject: [PATCH 025/177] timeout_call returns the result of user function now + +Signed-off-by: Ronan Abhamon +--- + drivers/util.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/util.py b/drivers/util.py +index 77f3f190..54fda469 100755 +--- a/drivers/util.py ++++ b/drivers/util.py +@@ -851,7 +851,7 @@ def timeout_call(timeoutseconds, function, *arguments): + signal.signal(signal.SIGALRM, handler) + signal.alarm(timeoutseconds) + try: +- function(*arguments) ++ return function(*arguments) + finally: + signal.alarm(0) + diff --git a/SOURCES/0026-Always-remove-the-pause-tag-from-VDIs-in-case-of-fai.patch b/SOURCES/0026-Always-remove-the-pause-tag-from-VDIs-in-case-of-fai.patch new file mode 100644 index 0000000..c729d4f --- /dev/null +++ b/SOURCES/0026-Always-remove-the-pause-tag-from-VDIs-in-case-of-fai.patch @@ -0,0 +1,55 @@ +From 7ebdef745b90fe8479555bc44d8d7ce594fc209f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 5 Feb 2024 23:16:16 +0100 +Subject: [PATCH 026/177] Always remove the pause tag from VDIs in case of + failure + +During VDI activation in the blktap module and in case of failure +in "sm.VDI.from_uuid" call, the pause tag is never removed. +As a result a VDI can no longer be used correctly: an assert is +triggered each time we try to re-activate this volume because +the tag is still present. + +Signed-off-by: Ronan Abhamon +--- + drivers/blktap2.py | 22 +++++++++++++--------- + 1 file changed, 13 insertions(+), 9 deletions(-) + +diff --git a/drivers/blktap2.py b/drivers/blktap2.py +index e1f75e9f..e9305ce9 100755 +--- a/drivers/blktap2.py ++++ b/drivers/blktap2.py +@@ -1599,20 +1599,24 @@ class VDI(object): + import VDI as sm + + #util.SMlog("VDI.activate %s" % vdi_uuid) ++ refresh = False + if self.tap_wanted(): + if not self._add_tag(vdi_uuid, not options["rdonly"]): + return False +- # it is possible that while the VDI was paused some of its +- # attributes have changed (e.g. its size if it was inflated; or its +- # path if it was leaf-coalesced onto a raw LV), so refresh the +- # object completely +- params = self.target.vdi.sr.srcmd.params +- target = sm.VDI.from_uuid(self.target.vdi.session, vdi_uuid) +- target.sr.srcmd.params = params +- driver_info = target.sr.srcmd.driver_info +- self.target = self.TargetDriver(target, driver_info) ++ refresh = True + + try: ++ if refresh: ++ # it is possible that while the VDI was paused some of its ++ # attributes have changed (e.g. its size if it was inflated; or its ++ # path if it was leaf-coalesced onto a raw LV), so refresh the ++ # object completely ++ params = self.target.vdi.sr.srcmd.params ++ target = sm.VDI.from_uuid(self.target.vdi.session, vdi_uuid) ++ target.sr.srcmd.params = params ++ driver_info = target.sr.srcmd.driver_info ++ self.target = self.TargetDriver(target, driver_info) ++ + util.fistpoint.activate_custom_fn( + "blktap_activate_inject_failure", + lambda: util.inject_failure()) diff --git a/SOURCES/0027-fix-LinstorSR-repair-volumes-only-if-an-exclusive-co.patch b/SOURCES/0027-fix-LinstorSR-repair-volumes-only-if-an-exclusive-co.patch new file mode 100644 index 0000000..41fa7ed --- /dev/null +++ b/SOURCES/0027-fix-LinstorSR-repair-volumes-only-if-an-exclusive-co.patch @@ -0,0 +1,33 @@ +From e3f11afdfdc43aa871a7d4f8c6d25564150fd0e4 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 20 Nov 2020 16:42:52 +0100 +Subject: [PATCH 027/177] fix(LinstorSR): repair volumes only if an exclusive + command is executed + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 8 +++++++- + 1 file changed, 7 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 8be18367..a5bf5abd 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -388,10 +388,16 @@ class LinstorSR(SR.SR): + + try: + # Try to open SR if exists. ++ # We can repair only if we are on the master AND if ++ # we are trying to execute an exclusive operation. ++ # Otherwise we could try to delete a VDI being created or ++ # during a snapshot. An exclusive op is the guarantee that the ++ # SR is locked. + self._linstor = LinstorVolumeManager( + self._master_uri, + self._group_name, +- repair=self._is_master, ++ repair=self._is_master and ++ self.srcmd.cmd in self.ops_exclusive, + logger=util.SMlog + ) + self._vhdutil = LinstorVhdUtil(self.session, self._linstor) diff --git a/SOURCES/0028-feat-LinstorSR-Improve-LINSTOR-performance.patch b/SOURCES/0028-feat-LinstorSR-Improve-LINSTOR-performance.patch new file mode 100644 index 0000000..41d83d6 --- /dev/null +++ b/SOURCES/0028-feat-LinstorSR-Improve-LINSTOR-performance.patch @@ -0,0 +1,1418 @@ +From fa743060d859c6bcdff75293bfee749b3eaf734c Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 10 Dec 2020 17:56:15 +0100 +Subject: [PATCH 028/177] feat(LinstorSR): Improve LINSTOR performance + +Details: +- vdi_attach and vdi_detach are now exclusive +- lock volumes on slaves (when vdi_xxx command is used) and avoid release if a timeout is reached +- load all VDIs only when necessary, so only if it exists at least a journal entry or if sr_scan/sr_attach is executed +- use a __slots__ attr in LinstorVolumeManager to increase performance +- use a cache directly in LinstorVolumeManager to reduce network request count with LINSTOR +- try to always use the same LINSTOR KV object to limit netwok usage +- use a cache to avoid a new JSON parsing when all VDIs are loaded in LinstorSR +- limit request count when LINSTOR storage pool info is fetched using a fetch interval +- avoid race condition in cleanup: check if a volume is locked in a slave or not before modify it +- ... + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 150 ++++++++--- + drivers/cleanup.py | 52 ++-- + drivers/linstor-manager | 9 +- + drivers/linstorvhdutil.py | 58 ++++- + drivers/linstorvolumemanager.py | 432 +++++++++++++++++++------------- + 5 files changed, 463 insertions(+), 238 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index a5bf5abd..548f4b17 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -92,7 +92,8 @@ DRIVER_CONFIG = {'ATTACH_FROM_CONFIG_WITH_TAPDISK': False} + + OPS_EXCLUSIVE = [ + 'sr_create', 'sr_delete', 'sr_attach', 'sr_detach', 'sr_scan', +- 'sr_update', 'vdi_create', 'vdi_delete', 'vdi_clone', 'vdi_snapshot' ++ 'sr_update', 'sr_probe', 'vdi_init', 'vdi_create', 'vdi_delete', ++ 'vdi_attach', 'vdi_detach', 'vdi_clone', 'vdi_snapshot', + ] + + # ============================================================================== +@@ -185,7 +186,9 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + + volume_info = linstor.get_volume_info(vdi_uuid) + old_volume_size = volume_info.virtual_size +- deflate(vdi_uuid, device_path, new_volume_size, old_volume_size) ++ deflate( ++ linstor, vdi_uuid, device_path, new_volume_size, old_volume_size ++ ) + finally: + lock.release() + +@@ -215,11 +218,11 @@ def inflate(journaler, linstor, vdi_uuid, vdi_path, new_size, old_size): + opterr='Failed to zero out VHD footer {}'.format(vdi_path) + ) + +- vhdutil.setSizePhys(vdi_path, new_size, False) ++ LinstorVhdUtil(None, linstor).set_size_phys(vdi_path, new_size, False) + journaler.remove(LinstorJournaler.INFLATE, vdi_uuid) + + +-def deflate(vdi_uuid, vdi_path, new_size, old_size): ++def deflate(linstor, vdi_uuid, vdi_path, new_size, old_size): + new_size = LinstorVolumeManager.round_up_volume_size(new_size) + if new_size >= old_size: + return +@@ -229,7 +232,7 @@ def deflate(vdi_uuid, vdi_path, new_size, old_size): + .format(vdi_uuid, new_size, old_size) + ) + +- vhdutil.setSizePhys(vdi_path, new_size) ++ LinstorVhdUtil(None, linstor).set_size_phys(vdi_path, new_size) + # TODO: Change the LINSTOR volume size using linstor.resize_volume. + + +@@ -318,10 +321,13 @@ class LinstorSR(SR.SR): + self._group_name = self.dconf['group-name'] + + self._master_uri = None +- self._vdi_shared_locked = False ++ self._vdi_shared_time = 0 + + self._initialized = False + ++ self._all_volume_info_cache = None ++ self._all_volume_metadata_cache = None ++ + def _locked_load(method): + @functools.wraps(method) + def wrap(self, *args, **kwargs): +@@ -374,7 +380,7 @@ class LinstorSR(SR.SR): + # behaviors if the GC is executed during an action on a slave. + if self.cmd.startswith('vdi_'): + self._shared_lock_vdi(self.srcmd.params['vdi_uuid']) +- self._vdi_shared_locked = True ++ self._vdi_shared_time = time.time() + + self._journaler = LinstorJournaler( + self._master_uri, self._group_name, logger=util.SMlog +@@ -396,8 +402,10 @@ class LinstorSR(SR.SR): + self._linstor = LinstorVolumeManager( + self._master_uri, + self._group_name, +- repair=self._is_master and +- self.srcmd.cmd in self.ops_exclusive, ++ repair=( ++ self._is_master and ++ self.srcmd.cmd in self.ops_exclusive ++ ), + logger=util.SMlog + ) + self._vhdutil = LinstorVhdUtil(self.session, self._linstor) +@@ -422,22 +430,55 @@ class LinstorSR(SR.SR): + if hosts: + util.SMlog('Failed to join node(s): {}'.format(hosts)) + ++ # Ensure we use a non-locked volume when vhdutil is called. ++ if ( ++ self._is_master and self.cmd.startswith('vdi_') and ++ self.cmd != 'vdi_create' ++ ): ++ self._linstor.ensure_volume_is_not_locked( ++ self.srcmd.params['vdi_uuid'] ++ ) ++ + try: +- # If the command is a SR command on the master, we must +- # load all VDIs and clean journal transactions. +- # We must load the VDIs in the snapshot case too. ++ # If the command is a SR scan command on the master, ++ # we must load all VDIs and clean journal transactions. ++ # We must load the VDIs in the snapshot case too only if ++ # there is at least one entry in the journal. ++ # ++ # If the command is a SR command we want at least to remove ++ # resourceless volumes. + if self._is_master and self.cmd not in [ + 'vdi_attach', 'vdi_detach', + 'vdi_activate', 'vdi_deactivate', + 'vdi_epoch_begin', 'vdi_epoch_end', + 'vdi_update', 'vdi_destroy' + ]: +- self._load_vdis() +- self._undo_all_journal_transactions() ++ load_vdis = ( ++ self.cmd == 'sr_scan' or ++ self.cmd == 'sr_attach' ++ ) or len( ++ self._journaler.get_all(LinstorJournaler.INFLATE) ++ ) or len( ++ self._journaler.get_all(LinstorJournaler.CLONE) ++ ) ++ ++ if load_vdis: ++ # We use a cache to avoid repeated JSON parsing. ++ # The performance gain is not big but we can still ++ # enjoy it with a few lines. ++ self._create_linstor_cache() ++ self._load_vdis() ++ self._destroy_linstor_cache() ++ ++ self._undo_all_journal_transactions() + self._linstor.remove_resourceless_volumes() + + self._synchronize_metadata() + except Exception as e: ++ if self.cmd == 'sr_scan': ++ # Always raise, we don't want to remove VDIs ++ # from the XAPI database otherwise. ++ raise e + util.SMlog( + 'Ignoring exception in LinstorSR.load: {}'.format(e) + ) +@@ -449,7 +490,7 @@ class LinstorSR(SR.SR): + + @_locked_load + def cleanup(self): +- if self._vdi_shared_locked: ++ if self._vdi_shared_time: + self._shared_lock_vdi(self.srcmd.params['vdi_uuid'], locked=False) + + @_locked_load +@@ -605,6 +646,23 @@ class LinstorSR(SR.SR): + 'locked': str(locked) + } + ++ # Note: We must avoid to unlock the volume if the timeout is reached ++ # because during volume unlock, the SR lock is not used. Otherwise ++ # we could destroy a valid lock acquired from another host... ++ # ++ # This code is not very clean, the ideal solution would be to acquire ++ # the SR lock during volume unlock (like lock) but it's not easy ++ # to implement without impacting performance. ++ if not locked: ++ elapsed_time = time.time() - self._vdi_shared_time ++ timeout = LinstorVolumeManager.LOCKED_EXPIRATION_DELAY * 0.7 ++ if elapsed_time >= timeout: ++ util.SMlog( ++ 'Avoid unlock call of {} because timeout has been reached' ++ .format(vdi_uuid) ++ ) ++ return ++ + ret = self.session.xenapi.host.call_plugin( + master, self.MANAGER_PLUGIN, method, args + ) +@@ -659,7 +717,7 @@ class LinstorSR(SR.SR): + + # Now update the VDI information in the metadata if required. + xenapi = self.session.xenapi +- volumes_metadata = self._linstor.volumes_with_metadata ++ volumes_metadata = self._linstor.get_volumes_with_metadata() + for vdi_uuid, volume_metadata in volumes_metadata.items(): + try: + vdi_ref = xenapi.VDI.get_by_uuid(vdi_uuid) +@@ -751,8 +809,8 @@ class LinstorSR(SR.SR): + xapi_vdi_uuids.add(xenapi.VDI.get_uuid(vdi)) + + # 2. Get volumes info. +- all_volume_info = self._linstor.volumes_with_info +- volumes_metadata = self._linstor.volumes_with_metadata ++ all_volume_info = self._all_volume_info_cache ++ volumes_metadata = self._all_volume_metadata_cache + + # 3. Get CBT vdis. + # See: https://support.citrix.com/article/CTX230619 +@@ -1020,13 +1078,13 @@ class LinstorSR(SR.SR): + util.SMlog('Cannot deflate missing VDI {}'.format(vdi_uuid)) + return + +- current_size = self._linstor.get_volume_info(self.uuid).virtual_size ++ current_size = self._all_volume_info_cache.get(self.uuid).virtual_size + util.zeroOut( + vdi.path, + current_size - vhdutil.VHD_FOOTER_SIZE, + vhdutil.VHD_FOOTER_SIZE + ) +- deflate(vdi_uuid, vdi.path, old_size, current_size) ++ deflate(self._linstor, vdi_uuid, vdi.path, old_size, current_size) + + def _handle_interrupted_clone( + self, vdi_uuid, clone_info, force_undo=False +@@ -1039,7 +1097,7 @@ class LinstorSR(SR.SR): + base_uuid, snap_uuid = clone_info.split('_') + + # Use LINSTOR data because new VDIs may not be in the XAPI. +- volume_names = self._linstor.volumes_with_name ++ volume_names = self._linstor.get_volumes_with_name() + + # Check if we don't have a base VDI. (If clone failed at startup.) + if base_uuid not in volume_names: +@@ -1095,7 +1153,7 @@ class LinstorSR(SR.SR): + if base_type == vhdutil.VDI_TYPE_VHD: + vhd_info = self._vhdutil.get_vhd_info(base_uuid, False) + if vhd_info.hidden: +- vhdutil.setHidden(base_path, False) ++ self._vhdutil.set_hidden(base_path, False) + elif base_type == vhdutil.VDI_TYPE_RAW and \ + base_metadata.get(HIDDEN_TAG): + self._linstor.update_volume_metadata( +@@ -1156,6 +1214,19 @@ class LinstorSR(SR.SR): + + util.SMlog('*** INTERRUPTED CLONE OP: rollback success') + ++ # -------------------------------------------------------------------------- ++ # Cache. ++ # -------------------------------------------------------------------------- ++ ++ def _create_linstor_cache(self): ++ self._all_volume_metadata_cache = \ ++ self._linstor.get_volumes_with_metadata() ++ self._all_volume_info_cache = self._linstor.get_volumes_with_info() ++ ++ def _destroy_linstor_cache(self): ++ self._all_volume_info_cache = None ++ self._all_volume_metadata_cache = None ++ + # -------------------------------------------------------------------------- + # Misc. + # -------------------------------------------------------------------------- +@@ -1326,16 +1397,16 @@ class LinstorVDI(VDI.VDI): + if self.vdi_type == vhdutil.VDI_TYPE_RAW: + self.size = volume_info.virtual_size + else: +- vhdutil.create( ++ self.sr._vhdutil.create( + self.path, size, False, self.MAX_METADATA_VIRT_SIZE + ) + self.size = self.sr._vhdutil.get_size_virt(self.uuid) + + if self._key_hash: +- vhdutil.setKey(self.path, self._key_hash) ++ self.sr._vhdutil.set_key(self.path, self._key_hash) + + # Because vhdutil commands modify the volume data, +- # we must retrieve a new time the utilisation size. ++ # we must retrieve a new time the utilization size. + volume_info = self._linstor.get_volume_info(self.uuid) + + volume_metadata = { +@@ -1548,7 +1619,7 @@ class LinstorVDI(VDI.VDI): + self.sr._journaler, self._linstor, self.uuid, self.path, + new_volume_size, old_volume_size + ) +- vhdutil.setSizeVirtFast(self.path, size) ++ self.sr._vhdutil.set_size_virt_fast(self.path, size) + + # Reload size attributes. + self._load_this() +@@ -1580,8 +1651,8 @@ class LinstorVDI(VDI.VDI): + if not blktap2.VDI.tap_pause(self.session, self.sr.uuid, self.uuid): + raise util.SMException('Failed to pause VDI {}'.format(self.uuid)) + try: +- vhdutil.setParent(self.path, parent_path, False) +- vhdutil.setHidden(parent_path) ++ self.sr._vhdutil.set_parent(self.path, parent_path, False) ++ self.sr._vhdutil.set_hidden(parent_path) + self.sr.session.xenapi.VDI.set_managed( + self.sr.srcmd.params['args'][0], False + ) +@@ -1658,11 +1729,20 @@ class LinstorVDI(VDI.VDI): + .format(self.uuid) + ) + +- vhdutil.killData(self.path) ++ self.sr._vhdutil.kill_data(self.path) + + def _load_this(self): +- volume_metadata = self._linstor.get_volume_metadata(self.uuid) +- volume_info = self._linstor.get_volume_info(self.uuid) ++ volume_metadata = None ++ if self.sr._all_volume_metadata_cache: ++ volume_metadata = self.sr._all_volume_metadata_cache.get(self.uuid) ++ if volume_metadata is None: ++ volume_metadata = self._linstor.get_volume_metadata(self.uuid) ++ ++ volume_info = None ++ if self.sr._all_volume_info_cache: ++ volume_info = self.sr._all_volume_info_cache.get(self.uuid) ++ if volume_info is None: ++ volume_info = self._linstor.get_volume_info(self.uuid) + + # Contains the physical size used on all disks. + # When LINSTOR LVM driver is used, the size should be similar to +@@ -1697,7 +1777,7 @@ class LinstorVDI(VDI.VDI): + return + + if self.vdi_type == vhdutil.VDI_TYPE_VHD: +- vhdutil.setHidden(self.path, hidden) ++ self.sr._vhdutil.set_hidden(self.path, hidden) + else: + self._linstor.update_volume_metadata(self.uuid, { + HIDDEN_TAG: hidden +@@ -1813,9 +1893,7 @@ class LinstorVDI(VDI.VDI): + 'VDIUnavailable', + opterr='failed to get vdi_type in metadata' + ) +- self._update_device_name( +- self._linstor.get_volume_name(self.uuid) +- ) ++ self._update_device_name(self._linstor.get_volume_name(self.uuid)) + + def _update_device_name(self, device_name): + self._device_name = device_name +@@ -1838,7 +1916,7 @@ class LinstorVDI(VDI.VDI): + + # 2. Write the snapshot content. + is_raw = (self.vdi_type == vhdutil.VDI_TYPE_RAW) +- vhdutil.snapshot( ++ self.sr._vhdutil.snapshot( + snap_path, self.path, is_raw, self.MAX_METADATA_VIRT_SIZE + ) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 926525f1..895f36e8 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -449,7 +449,7 @@ class XAPI: + # + # VDI + # +-class VDI: ++class VDI(object): + """Object representing a VDI of a VHD-based SR""" + + POLL_INTERVAL = 1 +@@ -1425,17 +1425,15 @@ class LinstorVDI(VDI): + self.sr.unlock() + VDI.delete(self) + +- def pauseVDIs(self, vdiList): +- self.sr._linstor.ensure_volume_list_is_not_locked( +- vdiList, timeout=self.VOLUME_LOCK_TIMEOUT +- ) +- return super(VDI).pauseVDIs(vdiList) ++ def validate(self, fast=False): ++ if not self.sr._vhdutil.check(self.uuid, fast=fast): ++ raise util.SMException('VHD {} corrupted'.format(self)) + +- def _liveLeafCoalesce(self, vdi): ++ def pause(self, failfast=False): + self.sr._linstor.ensure_volume_is_not_locked( +- vdi.uuid, timeout=self.VOLUME_LOCK_TIMEOUT ++ self.uuid, timeout=self.VOLUME_LOCK_TIMEOUT + ) +- return super(VDI)._liveLeafCoalesce(vdi) ++ return super(LinstorVDI, self).pause(failfast) + + def _relinkSkip(self): + abortFlag = IPCFlag(self.sr.uuid) +@@ -1479,7 +1477,7 @@ class LinstorVDI(VDI): + # + # SR + # +-class SR: ++class SR(object): + class LogFilter: + def __init__(self, sr): + self.sr = sr +@@ -2902,6 +2900,12 @@ class LinstorSR(SR): + self.logFilter.logState() + self._handleInterruptedCoalesceLeaf() + ++ def pauseVDIs(self, vdiList): ++ self._linstor.ensure_volume_list_is_not_locked( ++ vdiList, timeout=LinstorVDI.VOLUME_LOCK_TIMEOUT ++ ) ++ return super(LinstorSR, self).pauseVDIs(vdiList) ++ + def _reloadLinstor(self): + session = self.xapi.session + host_ref = util.get_this_host_ref(session) +@@ -2952,8 +2956,8 @@ class LinstorSR(SR): + + # TODO: Ensure metadata contains the right info. + +- all_volume_info = self._linstor.volumes_with_info +- volumes_metadata = self._linstor.volumes_with_metadata ++ all_volume_info = self._linstor.get_volumes_with_info() ++ volumes_metadata = self._linstor.get_volumes_with_metadata() + for vdi_uuid, volume_info in all_volume_info.items(): + try: + if not volume_info.name and \ +@@ -2984,8 +2988,10 @@ class LinstorSR(SR): + virtual_size = LinstorVolumeManager.round_up_volume_size( + parent.sizeVirt + meta_overhead + bitmap_overhead + ) +- # TODO: Check result. +- return virtual_size - self._linstor.get_volume_size(parent.uuid) ++ volume_size = self._linstor.get_volume_size(parent.uuid) ++ ++ assert virtual_size >= volume_size ++ return virtual_size - volume_size + + def _hasValidDevicePath(self, uuid): + try: +@@ -2995,6 +3001,16 @@ class LinstorSR(SR): + return False + return True + ++ def _liveLeafCoalesce(self, vdi): ++ self.lock() ++ try: ++ self._linstor.ensure_volume_is_not_locked( ++ vdi.uuid, timeout=LinstorVDI.VOLUME_LOCK_TIMEOUT ++ ) ++ return super(LinstorSR, self)._liveLeafCoalesce(vdi) ++ finally: ++ self.unlock() ++ + def _handleInterruptedCoalesceLeaf(self): + entries = self.journaler.get_all(VDI.JRN_LEAF) + for uuid, parentUuid in entries.iteritems(): +@@ -3021,7 +3037,6 @@ class LinstorSR(SR): + 'Renaming parent back: {} -> {}'.format(childUuid, parentUuid) + ) + parent.rename(parentUuid) +- util.fistpoint.activate('LVHDRT_coaleaf_undo_after_rename', self.uuid) + + child = self.getVDI(childUuid) + if not child: +@@ -3037,9 +3052,6 @@ class LinstorSR(SR): + Util.log('Updating the VDI record') + child.setConfig(VDI.DB_VHD_PARENT, parentUuid) + child.setConfig(VDI.DB_VDI_TYPE, vhdutil.VDI_TYPE_VHD) +- util.fistpoint.activate( +- 'LVHDRT_coaleaf_undo_after_rename2', self.uuid +- ) + + # TODO: Maybe deflate here. + +@@ -3048,10 +3060,7 @@ class LinstorSR(SR): + if not parent.hidden: + parent._setHidden(True) + self._updateSlavesOnUndoLeafCoalesce(parent, child) +- util.fistpoint.activate('LVHDRT_coaleaf_undo_end', self.uuid) + Util.log('*** leaf-coalesce undo successful') +- if util.fistpoint.is_active('LVHDRT_coaleaf_stop_after_recovery'): +- child.setConfig(VDI.DB_LEAFCLSC, VDI.LEAFCLSC_DISABLED) + + def _finishInterruptedCoalesceLeaf(self, childUuid, parentUuid): + Util.log('*** FINISH LEAF-COALESCE') +@@ -3064,7 +3073,6 @@ class LinstorSR(SR): + except XenAPI.Failure: + pass + self._updateSlavesOnResize(vdi) +- util.fistpoint.activate('LVHDRT_coaleaf_finish_end', self.uuid) + Util.log('*** finished leaf-coalesce successfully') + + def _checkSlaves(self, vdi): +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index f7ce1809..e7e58fd8 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -118,7 +118,9 @@ def detach(session, args): + def check(session, args): + try: + device_path = args['devicePath'] +- return str(vhdutil.check(device_path)) ++ ignore_missing_footer = args['ignoreMissingFooter'] ++ fast = args['fast'] ++ return str(vhdutil.check(device_path, ignore_missing_footer, fast)) + except Exception as e: + util.SMlog('linstor-manager:check error: {}'.format(e)) + raise +@@ -236,7 +238,10 @@ def lock_vdi(session, args): + group_name = args['groupName'] + locked = distutils.util.strtobool(args['locked']) + ++ # We must lock to mark the VDI. + lock = Lock(vhdutil.LOCK_TYPE_SR, sr_uuid) ++ if locked: ++ lock.acquire() + + linstor = LinstorVolumeManager( + get_linstor_uri(session), +@@ -249,7 +254,7 @@ def lock_vdi(session, args): + except Exception as e: + util.SMlog('linstor-manager:lock_vdi error: {}'.format(e)) + finally: +- if lock: ++ if locked and lock: + lock.release() + return str(False) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index f31c7525..ac858371 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -89,13 +89,33 @@ def linstorhostcall(local_method, remote_method): + return decorated + + ++def linstormodifier(): ++ def decorated(func): ++ def wrapper(*args, **kwargs): ++ self = args[0] ++ ++ ret = func(*args, **kwargs) ++ self._linstor.invalidate_resource_cache() ++ return ret ++ return wrapper ++ return decorated ++ ++ + class LinstorVhdUtil: + def __init__(self, session, linstor): + self._session = session + self._linstor = linstor + ++ # -------------------------------------------------------------------------- ++ # Getters. ++ # -------------------------------------------------------------------------- ++ ++ def check(self, vdi_uuid, ignore_missing_footer=False, fast=False): ++ kwargs = {'ignoreMissingFooter': ignore_missing_footer, 'fast': fast} ++ return self._check(vdi_uuid, **kwargs) ++ + @linstorhostcall(vhdutil.check, 'check') +- def check(self, vdi_uuid, **kwargs): ++ def _check(self, vdi_uuid, **kwargs): + return distutils.util.strtobool(kwargs['response']) + + def get_vhd_info(self, vdi_uuid, include_parent=True): +@@ -148,6 +168,42 @@ class LinstorVhdUtil: + def get_block_bitmap(self, vdi_uuid, **kwargs): + return base64.b64decode(kwargs['response']) + ++ # -------------------------------------------------------------------------- ++ # Setters. ++ # -------------------------------------------------------------------------- ++ ++ @linstormodifier() ++ def create(self, path, size, static, msize=0): ++ return vhdutil.create(path, size, static, msize) ++ ++ @linstormodifier() ++ def set_size_virt_fast(self, path, size): ++ return vhdutil.setSizeVirtFast(path, size) ++ ++ @linstormodifier() ++ def set_size_phys(self, path, size, debug=True): ++ return vhdutil.setSizePhys(path, size, debug) ++ ++ @linstormodifier() ++ def set_parent(self, path, parentPath, parentRaw): ++ return vhdutil.setParent(path, parentPath, parentRaw) ++ ++ @linstormodifier() ++ def set_hidden(self, path, hidden=True): ++ return vhdutil.setHidden(path, hidden) ++ ++ @linstormodifier() ++ def set_key(self, path, key_hash): ++ return vhdutil.setKey(path, key_hash) ++ ++ @linstormodifier() ++ def kill_data(self, path): ++ return vhdutil.killData(path) ++ ++ @linstormodifier() ++ def snapshot(self, path, parent, parentRaw, msize=0, checkEmpty=True): ++ return vhdutil.snapshot(path, parent, parentRaw, msize, checkEmpty) ++ + # -------------------------------------------------------------------------- + # Helpers. + # -------------------------------------------------------------------------- +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index d4004217..d617655f 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -63,6 +63,16 @@ class LinstorVolumeManager(object): + A volume in this context is a physical part of the storage layer. + """ + ++ __slots__ = ( ++ '_linstor', '_logger', ++ '_uri', '_base_group_name', ++ '_redundancy', '_group_name', ++ '_volumes', '_storage_pools', ++ '_storage_pools_time', ++ '_kv_cache', '_resource_cache', '_volume_info_cache', ++ '_kv_cache_dirty', '_resource_cache_dirty', '_volume_info_cache_dirty' ++ ) ++ + DEV_ROOT_PATH = '/dev/drbd/by-res/' + + # Default LVM extent size. +@@ -106,6 +116,10 @@ class LinstorVolumeManager(object): + PREFIX_SR = 'xcp-sr-' + PREFIX_VOLUME = 'xcp-volume-' + ++ # Limit request number when storage pool info is asked, we fetch ++ # the current pool status after N elapsed seconds. ++ STORAGE_POOLS_FETCH_INTERVAL = 15 ++ + @staticmethod + def default_logger(*args): + print(args) +@@ -164,6 +178,16 @@ class LinstorVolumeManager(object): + self._logger = logger + self._redundancy = groups[0].select_filter.place_count + self._group_name = group_name ++ self._volumes = set() ++ self._storage_pools_time = 0 ++ ++ # To increate performance and limit request count to LINSTOR services, ++ # we use caches. ++ self._kv_cache = self._create_kv_cache() ++ self._resource_cache = None ++ self._resource_cache_dirty = True ++ self._volume_info_cache = None ++ self._volume_info_cache_dirty = True + self._build_volumes(repair=repair) + + @property +@@ -184,66 +208,6 @@ class LinstorVolumeManager(object): + """ + return self._volumes + +- @property +- def volumes_with_name(self): +- """ +- Give a volume dictionnary that contains names actually owned. +- :return: A volume/name dict. +- :rtype: dict(str, str) +- """ +- return self._get_volumes_by_property(self.REG_VOLUME_NAME) +- +- @property +- def volumes_with_info(self): +- """ +- Give a volume dictionnary that contains VolumeInfos. +- :return: A volume/VolumeInfo dict. +- :rtype: dict(str, VolumeInfo) +- """ +- +- volumes = {} +- +- all_volume_info = self._get_volumes_info() +- volume_names = self.volumes_with_name +- for volume_uuid, volume_name in volume_names.items(): +- if volume_name: +- volume_info = all_volume_info.get(volume_name) +- if volume_info: +- volumes[volume_uuid] = volume_info +- continue +- +- # Well I suppose if this volume is not available, +- # LINSTOR has been used directly without using this API. +- volumes[volume_uuid] = self.VolumeInfo('') +- +- return volumes +- +- @property +- def volumes_with_metadata(self): +- """ +- Give a volume dictionnary that contains metadata. +- :return: A volume/metadata dict. +- :rtype: dict(str, dict) +- """ +- +- volumes = {} +- +- metadata = self._get_volumes_by_property(self.REG_METADATA) +- for volume_uuid, volume_metadata in metadata.items(): +- if volume_metadata: +- volume_metadata = json.loads(volume_metadata) +- if isinstance(volume_metadata, dict): +- volumes[volume_uuid] = volume_metadata +- continue +- raise LinstorVolumeManagerError( +- 'Expected dictionary in volume metadata: {}' +- .format(volume_uuid) +- ) +- +- volumes[volume_uuid] = {} +- +- return volumes +- + @property + def max_volume_size_allowed(self): + """ +@@ -292,7 +256,7 @@ class LinstorVolumeManager(object): + """ + + size = 0 +- for resource in self._linstor.resource_list_raise().resources: ++ for resource in self._get_resource_cache().resources: + for volume in resource.volumes: + # We ignore diskless pools of the form "DfltDisklessStorPool". + if volume.storage_pool_name == self._group_name: +@@ -346,12 +310,8 @@ class LinstorVolumeManager(object): + :rtype: set(str) + """ + +- pools = self._linstor.storage_pool_list_raise( +- filter_by_stor_pools=[self._group_name] +- ).storage_pools +- + disconnected_hosts = set() +- for pool in pools: ++ for pool in self._get_storage_pools(): + for report in pool.reports: + if report.ret_code & linstor.consts.WARN_NOT_CONNECTED == \ + linstor.consts.WARN_NOT_CONNECTED: +@@ -397,7 +357,7 @@ class LinstorVolumeManager(object): + ) + return device_path + except Exception: +- self._force_destroy_volume(volume_uuid, volume_properties) ++ self._force_destroy_volume(volume_uuid) + raise + + def mark_volume_as_persistent(self, volume_uuid): +@@ -426,7 +386,7 @@ class LinstorVolumeManager(object): + volume_properties[self.PROP_NOT_EXISTS] = self.STATE_NOT_EXISTS + + self._volumes.remove(volume_uuid) +- self._destroy_volume(volume_uuid, volume_properties) ++ self._destroy_volume(volume_uuid) + + def lock_volume(self, volume_uuid, locked=True): + """ +@@ -476,12 +436,15 @@ class LinstorVolumeManager(object): + + waiting = False + ++ volume_properties = self._get_kv_cache() ++ + start = time.time() + while True: + # Can't delete in for loop, use a copy of the list. + remaining = checked.copy() + for volume_uuid in checked: +- volume_properties = self._get_volume_properties(volume_uuid) ++ volume_properties.namespace = \ ++ self._build_volume_namespace(volume_uuid) + timestamp = volume_properties.get( + self.PROP_IS_READONLY_TIMESTAMP + ) +@@ -519,6 +482,7 @@ class LinstorVolumeManager(object): + # We must wait to use the volume. After that we can modify it + # ONLY if the SR is locked to avoid bad reads on the slaves. + time.sleep(1) ++ volume_properties = self._create_kv_cache() + + if waiting: + self._logger('No volume locked now!') +@@ -542,6 +506,9 @@ class LinstorVolumeManager(object): + volume_nr=0, + size=new_size / 1024 + ) ++ ++ self._mark_resource_cache_as_dirty() ++ + error_str = self._get_error_str(result) + if error_str: + raise LinstorVolumeManagerError( +@@ -596,7 +563,7 @@ class LinstorVolumeManager(object): + """ + + volume_name = self.get_volume_name(volume_uuid) +- return self._get_volumes_info(filter=[volume_name])[volume_name] ++ return self._get_volumes_info()[volume_name] + + def get_device_path(self, volume_uuid): + """ +@@ -620,7 +587,7 @@ class LinstorVolumeManager(object): + expected_volume_name = \ + self.get_volume_name_from_device_path(device_path) + +- volume_names = self.volumes_with_name ++ volume_names = self.get_volumes_with_name() + for volume_uuid, volume_name in volume_names.items(): + if volume_name == expected_volume_name: + return volume_uuid +@@ -638,9 +605,11 @@ class LinstorVolumeManager(object): + """ + + node_name = socket.gethostname() +- resources = self._linstor.resource_list_raise( +- filter_by_nodes=[node_name] +- ).resources ++ ++ resources = filter( ++ lambda resource: resource.node_name == node_name, ++ self._get_resource_cache().resources ++ ) + + real_device_path = os.path.realpath(device_path) + for resource in resources: +@@ -664,6 +633,8 @@ class LinstorVolumeManager(object): + deleted VDI. + """ + ++ assert volume_uuid != new_volume_uuid ++ + self._logger( + 'Trying to update volume UUID {} to {}...' + .format(volume_uuid, new_volume_uuid) +@@ -685,36 +656,41 @@ class LinstorVolumeManager(object): + .format(volume_uuid) + ) + +- new_volume_properties = self._get_volume_properties( ++ # 1. Copy in temp variables metadata and volume_name. ++ metadata = volume_properties.get(self.PROP_METADATA) ++ volume_name = volume_properties.get(self.PROP_VOLUME_NAME) ++ ++ # 2. Switch to new volume namespace. ++ volume_properties.namespace = self._build_volume_namespace( + new_volume_uuid + ) +- if list(new_volume_properties.items()): ++ ++ if list(volume_properties.items()): + raise LinstorVolumeManagerError( + 'Cannot update volume uuid {} to {}: ' + .format(volume_uuid, new_volume_uuid) + + 'this last one is not empty' + ) + +- assert volume_properties.namespace != \ +- new_volume_properties.namespace +- + try: +- # 1. Mark new volume properties with PROP_UPDATING_UUID_SRC. ++ # 3. Mark new volume properties with PROP_UPDATING_UUID_SRC. + # If we crash after that, the new properties can be removed + # properly. +- new_volume_properties[self.PROP_NOT_EXISTS] = self.STATE_NOT_EXISTS +- new_volume_properties[self.PROP_UPDATING_UUID_SRC] = volume_uuid ++ volume_properties[self.PROP_NOT_EXISTS] = self.STATE_NOT_EXISTS ++ volume_properties[self.PROP_UPDATING_UUID_SRC] = volume_uuid + +- # 2. Copy the properties. +- for property in [self.PROP_METADATA, self.PROP_VOLUME_NAME]: +- new_volume_properties[property] = \ +- volume_properties.get(property) ++ # 4. Copy the properties. ++ volume_properties[self.PROP_METADATA] = metadata ++ volume_properties[self.PROP_VOLUME_NAME] = volume_name + +- # 3. Ok! +- new_volume_properties[self.PROP_NOT_EXISTS] = self.STATE_EXISTS ++ # 5. Ok! ++ volume_properties[self.PROP_NOT_EXISTS] = self.STATE_EXISTS + except Exception as e: + try: +- new_volume_properties.clear() ++ # Clear the new volume properties in case of failure. ++ assert volume_properties.namespace == \ ++ self._build_volume_namespace(new_volume_uuid) ++ volume_properties.clear() + except Exception as e: + self._logger( + 'Failed to clear new volume properties: {} (ignoring...)' +@@ -725,11 +701,21 @@ class LinstorVolumeManager(object): + ) + + try: +- # 4. After this point, it's ok we can remove the ++ # 6. After this point, it's ok we can remove the + # PROP_UPDATING_UUID_SRC property and clear the src properties + # without problems. ++ ++ # 7. Switch to old volume namespace. ++ volume_properties.namespace = self._build_volume_namespace( ++ volume_uuid ++ ) + volume_properties.clear() +- new_volume_properties.pop(self.PROP_UPDATING_UUID_SRC) ++ ++ # 8. Switch a last time to new volume namespace. ++ volume_properties.namespace = self._build_volume_namespace( ++ new_volume_uuid ++ ) ++ volume_properties.pop(self.PROP_UPDATING_UUID_SRC) + except Exception as e: + raise LinstorVolumeManagerError( + 'Failed to clear volume properties ' +@@ -743,7 +729,7 @@ class LinstorVolumeManager(object): + 'UUID update succeeded of {} to {}! (properties={})' + .format( + volume_uuid, new_volume_uuid, +- self._get_filtered_properties(new_volume_properties) ++ self._get_filtered_properties(volume_properties) + ) + ) + +@@ -788,6 +774,63 @@ class LinstorVolumeManager(object): + + return states + ++ def get_volumes_with_name(self): ++ """ ++ Give a volume dictionnary that contains names actually owned. ++ :return: A volume/name dict. ++ :rtype: dict(str, str) ++ """ ++ return self._get_volumes_by_property(self.REG_VOLUME_NAME) ++ ++ def get_volumes_with_info(self): ++ """ ++ Give a volume dictionnary that contains VolumeInfos. ++ :return: A volume/VolumeInfo dict. ++ :rtype: dict(str, VolumeInfo) ++ """ ++ ++ volumes = {} ++ ++ all_volume_info = self._get_volumes_info() ++ volume_names = self.get_volumes_with_name() ++ for volume_uuid, volume_name in volume_names.items(): ++ if volume_name: ++ volume_info = all_volume_info.get(volume_name) ++ if volume_info: ++ volumes[volume_uuid] = volume_info ++ continue ++ ++ # Well I suppose if this volume is not available, ++ # LINSTOR has been used directly without using this API. ++ volumes[volume_uuid] = self.VolumeInfo('') ++ ++ return volumes ++ ++ def get_volumes_with_metadata(self): ++ """ ++ Give a volume dictionnary that contains metadata. ++ :return: A volume/metadata dict. ++ :rtype: dict(str, dict) ++ """ ++ ++ volumes = {} ++ ++ metadata = self._get_volumes_by_property(self.REG_METADATA) ++ for volume_uuid, volume_metadata in metadata.items(): ++ if volume_metadata: ++ volume_metadata = json.loads(volume_metadata) ++ if isinstance(volume_metadata, dict): ++ volumes[volume_uuid] = volume_metadata ++ continue ++ raise LinstorVolumeManagerError( ++ 'Expected dictionary in volume metadata: {}' ++ .format(volume_uuid) ++ ) ++ ++ volumes[volume_uuid] = {} ++ ++ return volumes ++ + def get_volume_metadata(self, volume_uuid): + """ + Get the metadata of a volume. +@@ -918,9 +961,9 @@ class LinstorVolumeManager(object): + )) + + # 5. Create resources! +- def clean(properties): ++ def clean(): + try: +- self._destroy_volume(clone_uuid, properties) ++ self._destroy_volume(clone_uuid) + except Exception as e: + self._logger( + 'Unable to destroy volume {} after shallow clone fail: {}' +@@ -946,7 +989,7 @@ class LinstorVolumeManager(object): + ) + return volume_properties + except Exception: +- clean(volume_properties) ++ clean() + raise + + # Retry because we can get errors like this: +@@ -962,7 +1005,7 @@ class LinstorVolumeManager(object): + self._volumes.add(clone_uuid) + return device_path + except Exception as e: +- clean(volume_properties) ++ clean() + raise + + def remove_resourceless_volumes(self): +@@ -974,7 +1017,7 @@ class LinstorVolumeManager(object): + """ + + resource_names = self._fetch_resource_names() +- for volume_uuid, volume_name in self.volumes_with_name.items(): ++ for volume_uuid, volume_name in self.get_volumes_with_name().items(): + if not volume_name or volume_name not in resource_names: + self.destroy_volume(volume_uuid) + +@@ -992,11 +1035,7 @@ class LinstorVolumeManager(object): + # TODO: What's the required action if it exists remaining volumes? + + self._destroy_resource_group(self._linstor, self._group_name) +- +- pools = self._linstor.storage_pool_list_raise( +- filter_by_stor_pools=[self._group_name] +- ).storage_pools +- for pool in pools: ++ for pool in self._get_storage_pools(force=True): + self._destroy_storage_pool( + self._linstor, pool.name, pool.node_name + ) +@@ -1014,10 +1053,13 @@ class LinstorVolumeManager(object): + + in_use = False + node_names = set() +- resource_list = self._linstor.resource_list_raise( +- filter_by_resources=[volume_name] ++ ++ resource_states = filter( ++ lambda resource_state: resource_state.name == volume_name, ++ self._get_resource_cache().resource_states + ) +- for resource_state in resource_list.resource_states: ++ ++ for resource_state in resource_states: + volume_state = resource_state.volume_states[0] + if volume_state.disk_state == 'UpToDate': + node_names.add(resource_state.node_name) +@@ -1026,6 +1068,14 @@ class LinstorVolumeManager(object): + + return (node_names, in_use) + ++ def invalidate_resource_cache(self): ++ """ ++ If resources are impacted by external commands like vhdutil, ++ it's necessary to call this function to invalidate current resource ++ cache. ++ """ ++ self._mark_resource_cache_as_dirty() ++ + @classmethod + def create_sr( + cls, uri, group_name, node_names, redundancy, +@@ -1149,6 +1199,12 @@ class LinstorVolumeManager(object): + instance._redundancy = redundancy + instance._group_name = group_name + instance._volumes = set() ++ instance._storage_pools_time = 0 ++ instance._kv_cache = instance._create_kv_cache() ++ instance._resource_cache = None ++ instance._resource_cache_dirty = True ++ instance._volume_info_cache = None ++ instance._volume_info_cache_dirty = True + return instance + + @classmethod +@@ -1196,6 +1252,32 @@ class LinstorVolumeManager(object): + # Private helpers. + # -------------------------------------------------------------------------- + ++ def _create_kv_cache(self): ++ self._kv_cache = self._create_linstor_kv('/') ++ self._kv_cache_dirty = False ++ return self._kv_cache ++ ++ def _get_kv_cache(self): ++ if self._kv_cache_dirty: ++ self._kv_cache = self._create_kv_cache() ++ return self._kv_cache ++ ++ def _create_resource_cache(self): ++ self._resource_cache = self._linstor.resource_list_raise() ++ self._resource_cache_dirty = False ++ return self._resource_cache ++ ++ def _get_resource_cache(self): ++ if self._resource_cache_dirty: ++ self._resource_cache = self._create_resource_cache() ++ return self._resource_cache ++ ++ def _mark_resource_cache_as_dirty(self): ++ self._resource_cache_dirty = True ++ self._volume_info_cache_dirty = True ++ ++ # -------------------------------------------------------------------------- ++ + def _ensure_volume_exists(self, volume_uuid): + if volume_uuid not in self._volumes: + raise LinstorVolumeManagerError( +@@ -1224,12 +1306,13 @@ class LinstorVolumeManager(object): + resource_names.add(dfn.name) + return resource_names + +- def _get_volumes_info(self, filter=None): ++ def _get_volumes_info(self, volume_name=None): + all_volume_info = {} +- resources = self._linstor.resource_list_raise( +- filter_by_resources=filter +- ) +- for resource in resources.resources: ++ ++ if not self._volume_info_cache_dirty: ++ return self._volume_info_cache ++ ++ for resource in self._get_resource_cache().resources: + if resource.name not in all_volume_info: + current = all_volume_info[resource.name] = self.VolumeInfo( + resource.name +@@ -1261,6 +1344,9 @@ class LinstorVolumeManager(object): + current.physical_size *= 1024 + current.virtual_size *= 1024 + ++ self._volume_info_cache_dirty = False ++ self._volume_info_cache = all_volume_info ++ + return all_volume_info + + def _get_volume_node_names_and_size(self, volume_name): +@@ -1289,12 +1375,8 @@ class LinstorVolumeManager(object): + return (node_names, size * 1024) + + def _compute_size(self, attr): +- pools = self._linstor.storage_pool_list_raise( +- filter_by_stor_pools=[self._group_name] +- ).storage_pools +- + capacity = 0 +- for pool in pools: ++ for pool in self._get_storage_pools(force=True): + space = pool.free_space + if space: + size = getattr(space, attr) +@@ -1308,13 +1390,22 @@ class LinstorVolumeManager(object): + + def _get_node_names(self): + node_names = set() +- pools = self._linstor.storage_pool_list_raise( +- filter_by_stor_pools=[self._group_name] +- ).storage_pools +- for pool in pools: ++ for pool in self._get_storage_pools(): + node_names.add(pool.node_name) + return node_names + ++ def _get_storage_pools(self, force=False): ++ cur_time = time.time() ++ elsaped_time = cur_time - self._storage_pools_time ++ ++ if force or elsaped_time >= self.STORAGE_POOLS_FETCH_INTERVAL: ++ self._storage_pools = self._linstor.storage_pool_list_raise( ++ filter_by_stor_pools=[self._group_name] ++ ).storage_pools ++ self._storage_pools_time = time.time() ++ ++ return self._storage_pools ++ + def _check_volume_creation_errors(self, result, volume_uuid): + errors = self._filter_errors(result) + if self._check_errors(errors, [ +@@ -1338,6 +1429,7 @@ class LinstorVolumeManager(object): + def _create_volume(self, volume_uuid, volume_name, size, place_resources): + size = self.round_up_volume_size(size) + ++ self._mark_resource_cache_as_dirty() + self._check_volume_creation_errors(self._linstor.resource_group_spawn( + rsc_grp_name=self._group_name, + rsc_dfn_name=volume_name, +@@ -1378,6 +1470,8 @@ class LinstorVolumeManager(object): + volume_uuid, volume_name, size, place_resources + ) + ++ assert volume_properties.namespace == \ ++ self._build_volume_namespace(volume_uuid) + return volume_properties + except LinstorVolumeManagerError as e: + # Do not destroy existing resource! +@@ -1387,10 +1481,10 @@ class LinstorVolumeManager(object): + # call in another host. + if e.code == LinstorVolumeManagerError.ERR_VOLUME_EXISTS: + raise +- self._force_destroy_volume(volume_uuid, volume_properties) ++ self._force_destroy_volume(volume_uuid) + raise + except Exception: +- self._force_destroy_volume(volume_uuid, volume_properties) ++ self._force_destroy_volume(volume_uuid) + raise + + def _find_device_path(self, volume_uuid, volume_name): +@@ -1417,34 +1511,26 @@ class LinstorVolumeManager(object): + + def _request_device_path(self, volume_uuid, volume_name, activate=False): + node_name = socket.gethostname() +- resources = self._linstor.resource_list( +- filter_by_nodes=[node_name], +- filter_by_resources=[volume_name] ++ ++ resources = filter( ++ lambda resource: resource.node_name == node_name and ++ resource.name == volume_name, ++ self._get_resource_cache().resources + ) + +- if not resources or not resources[0]: ++ if not resources: ++ if activate: ++ self._activate_device_path(node_name, volume_name) ++ return self._request_device_path(volume_uuid, volume_name) + raise LinstorVolumeManagerError( +- 'No response list for dev path of `{}`'.format(volume_uuid) +- ) +- if isinstance(resources[0], linstor.responses.ResourceResponse): +- if not resources[0].resources: +- if activate: +- self._activate_device_path(node_name, volume_name) +- return self._request_device_path(volume_uuid, volume_name) +- raise LinstorVolumeManagerError( +- 'Empty dev path for `{}`, but definition "seems" to exist' +- .format(volume_uuid) +- ) +- # Contains a path of the /dev/drbd form. +- return resources[0].resources[0].volumes[0].device_path +- +- raise LinstorVolumeManagerError( +- 'Unable to get volume dev path `{}`: {}'.format( +- volume_uuid, str(resources[0]) ++ 'Empty dev path for `{}`, but definition "seems" to exist' ++ .format(volume_uuid) + ) +- ) ++ # Contains a path of the /dev/drbd form. ++ return resources[0].volumes[0].device_path + + def _activate_device_path(self, node_name, volume_name): ++ self._mark_resource_cache_as_dirty() + result = self._linstor.resource_create([ + linstor.ResourceData(node_name, volume_name, diskless=True) + ]) +@@ -1463,6 +1549,7 @@ class LinstorVolumeManager(object): + ) + + def _destroy_resource(self, resource_name): ++ self._mark_resource_cache_as_dirty() + result = self._linstor.resource_dfn_delete(resource_name) + error_str = self._get_error_str(result) + if error_str: +@@ -1471,10 +1558,8 @@ class LinstorVolumeManager(object): + .format(resource_name, self._group_name, error_str) + ) + +- def _destroy_volume(self, volume_uuid, volume_properties): +- assert volume_properties.namespace == \ +- self._build_volume_namespace(volume_uuid) +- ++ def _destroy_volume(self, volume_uuid): ++ volume_properties = self._get_volume_properties(volume_uuid) + try: + volume_name = volume_properties.get(self.PROP_VOLUME_NAME) + if volume_name in self._fetch_resource_names(): +@@ -1487,19 +1572,14 @@ class LinstorVolumeManager(object): + 'Cannot destroy volume `{}`: {}'.format(volume_uuid, e) + ) + +- def _force_destroy_volume(self, volume_uuid, volume_properties): ++ def _force_destroy_volume(self, volume_uuid): + try: +- self._destroy_volume(volume_uuid, volume_properties) ++ self._destroy_volume(volume_uuid) + except Exception as e: + self._logger('Ignore fail: {}'.format(e)) + + def _build_volumes(self, repair): +- properties = linstor.KV( +- self._get_store_name(), +- uri=self._uri, +- namespace=self._build_volume_namespace() +- ) +- ++ properties = self._kv_cache + resource_names = self._fetch_resource_names() + + self._volumes = set() +@@ -1517,9 +1597,7 @@ class LinstorVolumeManager(object): + self.REG_NOT_EXISTS, ignore_inexisting_volumes=False + ) + for volume_uuid, not_exists in existing_volumes.items(): +- properties.namespace = self._build_volume_namespace( +- volume_uuid +- ) ++ properties.namespace = self._build_volume_namespace(volume_uuid) + + src_uuid = properties.get(self.PROP_UPDATING_UUID_SRC) + if src_uuid: +@@ -1580,36 +1658,31 @@ class LinstorVolumeManager(object): + ) + + for dest_uuid, src_uuid in updating_uuid_volumes.items(): +- dest_properties = self._get_volume_properties(dest_uuid) +- if int(dest_properties.get(self.PROP_NOT_EXISTS) or +- self.STATE_EXISTS): +- dest_properties.clear() ++ dest_namespace = self._build_volume_namespace(dest_uuid) ++ ++ properties.namespace = dest_namespace ++ if int(properties.get(self.PROP_NOT_EXISTS)): ++ properties.clear() + continue + +- src_properties = self._get_volume_properties(src_uuid) +- src_properties.clear() ++ properties.namespace = self._build_volume_namespace(src_uuid) ++ properties.clear() + +- dest_properties.pop(self.PROP_UPDATING_UUID_SRC) ++ properties.namespace = dest_namespace ++ properties.pop(self.PROP_UPDATING_UUID_SRC) + + if src_uuid in self._volumes: + self._volumes.remove(src_uuid) + self._volumes.add(dest_uuid) + + def _get_sr_properties(self): +- return linstor.KV( +- self._get_store_name(), +- uri=self._uri, +- namespace=self._build_sr_namespace() +- ) ++ return self._create_linstor_kv(self._build_sr_namespace()) + + def _get_volumes_by_property( + self, reg_prop, ignore_inexisting_volumes=True + ): +- base_properties = linstor.KV( +- self._get_store_name(), +- uri=self._uri, +- namespace=self._build_volume_namespace() +- ) ++ base_properties = self._get_kv_cache() ++ base_properties.namespace = self._build_volume_namespace() + + volume_properties = {} + for volume_uuid in self._volumes: +@@ -1625,13 +1698,18 @@ class LinstorVolumeManager(object): + + return volume_properties + +- def _get_volume_properties(self, volume_uuid): ++ def _create_linstor_kv(self, namespace): + return linstor.KV( + self._get_store_name(), + uri=self._uri, +- namespace=self._build_volume_namespace(volume_uuid) ++ namespace=namespace + ) + ++ def _get_volume_properties(self, volume_uuid): ++ properties = self._get_kv_cache() ++ properties.namespace = self._build_volume_namespace(volume_uuid) ++ return properties ++ + def _get_store_name(self): + return 'xcp-sr-{}'.format(self._group_name) + diff --git a/SOURCES/0029-feat-LinstorSR-robustify-scan-to-avoid-losing-VDIs-i.patch b/SOURCES/0029-feat-LinstorSR-robustify-scan-to-avoid-losing-VDIs-i.patch new file mode 100644 index 0000000..5aa1fef --- /dev/null +++ b/SOURCES/0029-feat-LinstorSR-robustify-scan-to-avoid-losing-VDIs-i.patch @@ -0,0 +1,88 @@ +From 045d23321e99b135556cb3d9872c1f3c8ae4b9af Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 7 Jan 2021 11:17:08 +0100 +Subject: [PATCH 029/177] feat(LinstorSR): robustify scan to avoid losing VDIs + if function is called outside module + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 30 +++++++++++++++++++++--------- + 1 file changed, 21 insertions(+), 9 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 548f4b17..52131a59 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -325,6 +325,7 @@ class LinstorSR(SR.SR): + + self._initialized = False + ++ self._vdis_loaded = False + self._all_volume_info_cache = None + self._all_volume_metadata_cache = None + +@@ -463,19 +464,13 @@ class LinstorSR(SR.SR): + ) + + if load_vdis: +- # We use a cache to avoid repeated JSON parsing. +- # The performance gain is not big but we can still +- # enjoy it with a few lines. +- self._create_linstor_cache() + self._load_vdis() +- self._destroy_linstor_cache() + +- self._undo_all_journal_transactions() + self._linstor.remove_resourceless_volumes() + + self._synchronize_metadata() + except Exception as e: +- if self.cmd == 'sr_scan': ++ if self.cmd == 'sr_scan' or self.cmd == 'sr_attach': + # Always raise, we don't want to remove VDIs + # from the XAPI database otherwise. + raise e +@@ -612,6 +607,9 @@ class LinstorSR(SR.SR): + opterr='no such volume group: {}'.format(self._group_name) + ) + ++ # Note: `scan` can be called outside this module, so ensure the VDIs ++ # are loaded. ++ self._load_vdis() + self._update_physical_size() + + for vdi_uuid in self.vdis.keys(): +@@ -799,9 +797,22 @@ class LinstorSR(SR.SR): + # -------------------------------------------------------------------------- + + def _load_vdis(self): +- if self.vdis: ++ if self._vdis_loaded: + return ++ self._vdis_loaded = True ++ ++ assert self._is_master ++ ++ # We use a cache to avoid repeated JSON parsing. ++ # The performance gain is not big but we can still ++ # enjoy it with a few lines. ++ self._create_linstor_cache() ++ self._load_vdis_ex() ++ self._destroy_linstor_cache() ++ ++ self._undo_all_journal_transactions() + ++ def _load_vdis_ex(self): + # 1. Get existing VDIs in XAPI. + xenapi = self.session.xenapi + xapi_vdi_uuids = set() +@@ -822,7 +833,8 @@ class LinstorSR(SR.SR): + + introduce = False + +- if self.cmd == 'sr_scan': ++ # Try to introduce VDIs only during scan/attach. ++ if self.cmd == 'sr_scan' or self.cmd == 'sr_attach': + has_clone_entries = list(self._journaler.get_all( + LinstorJournaler.CLONE + ).items()) diff --git a/SOURCES/0030-feat-LinstorSR-display-a-correctly-readable-size-for.patch b/SOURCES/0030-feat-LinstorSR-display-a-correctly-readable-size-for.patch new file mode 100644 index 0000000..5819a4c --- /dev/null +++ b/SOURCES/0030-feat-LinstorSR-display-a-correctly-readable-size-for.patch @@ -0,0 +1,278 @@ +From 6594b0cfd56df7d8beed5442b7932e6d46623f5f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 8 Jan 2021 16:12:15 +0100 +Subject: [PATCH 030/177] feat(LinstorSR): display a correctly readable size + for the user + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 46 +++++++++------------ + drivers/linstorvolumemanager.py | 72 +++++++++++++++++++++++++++++---- + 2 files changed, 83 insertions(+), 35 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 52131a59..16cb0d62 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -770,27 +770,19 @@ class LinstorSR(SR.SR): + # Update size attributes of the SR parent class. + self.virtual_allocation = valloc + virt_alloc_delta + +- # Physical size contains the total physical size. +- # i.e. the sum of the sizes of all devices on all hosts, not the AVG. + self._update_physical_size() + + # Notify SR parent class. + self._db_update() + + def _update_physical_size(self): +- # Physical size contains the total physical size. +- # i.e. the sum of the sizes of all devices on all hosts, not the AVG. +- self.physical_size = self._linstor.physical_size ++ # We use the size of the smallest disk, this is an approximation that ++ # ensures the displayed physical size is reachable by the user. ++ self.physical_size = \ ++ self._linstor.min_physical_size * len(self._hosts) / \ ++ self._redundancy + +- # `self._linstor.physical_free_size` contains the total physical free +- # memory. If Thin provisioning is used we can't use it, we must use +- # LINSTOR volume size to gives a good idea of the required +- # usable memory to the users. +- self.physical_utilisation = self._linstor.total_allocated_volume_size +- +- # If Thick provisioning is used, we can use this line instead: +- # self.physical_utilisation = \ +- # self.physical_size - self._linstor.physical_free_size ++ self.physical_utilisation = self._linstor.allocated_volume_size + + # -------------------------------------------------------------------------- + # VDIs. +@@ -912,10 +904,10 @@ class LinstorSR(SR.SR): + + util.SMlog( + 'Introducing VDI {} '.format(vdi_uuid) + +- ' (name={}, virtual_size={}, physical_size={})'.format( ++ ' (name={}, virtual_size={}, allocated_size={})'.format( + name_label, + volume_info.virtual_size, +- volume_info.physical_size ++ volume_info.allocated_size + ) + ) + +@@ -933,7 +925,7 @@ class LinstorSR(SR.SR): + sm_config, + managed, + str(volume_info.virtual_size), +- str(volume_info.physical_size) ++ str(volume_info.allocated_size) + ) + + is_a_snapshot = volume_metadata.get(IS_A_SNAPSHOT_TAG) +@@ -1016,7 +1008,7 @@ class LinstorSR(SR.SR): + else: + geneology[vdi.parent] = [vdi_uuid] + if not vdi.hidden: +- self.virtual_allocation += vdi.utilisation ++ self.virtual_allocation += vdi.size + + # 9. Remove all hidden leaf nodes to avoid introducing records that + # will be GC'ed. +@@ -1453,11 +1445,11 @@ class LinstorVDI(VDI.VDI): + '{}'.format(e) + ) + +- self.utilisation = volume_info.physical_size ++ self.utilisation = volume_info.allocated_size + self.sm_config['vdi_type'] = self.vdi_type + + self.ref = self._db_introduce() +- self.sr._update_stats(volume_info.virtual_size) ++ self.sr._update_stats(self.size) + + return VDI.VDI.get_params(self) + +@@ -1496,7 +1488,7 @@ class LinstorVDI(VDI.VDI): + del self.sr.vdis[self.uuid] + + # TODO: Check size after delete. +- self.sr._update_stats(-self.capacity) ++ self.sr._update_stats(-self.size) + self.sr._kick_gc() + return super(LinstorVDI, self).delete(sr_uuid, vdi_uuid, data_only) + +@@ -1622,7 +1614,7 @@ class LinstorVDI(VDI.VDI): + space_needed = new_volume_size - old_volume_size + self.sr._ensure_space_available(space_needed) + +- old_capacity = self.capacity ++ old_size = self.size + if self.vdi_type == vhdutil.VDI_TYPE_RAW: + self._linstor.resize(self.uuid, new_volume_size) + else: +@@ -1641,7 +1633,7 @@ class LinstorVDI(VDI.VDI): + self.session.xenapi.VDI.set_physical_utilisation( + vdi_ref, str(self.utilisation) + ) +- self.sr._update_stats(self.capacity - old_capacity) ++ self.sr._update_stats(self.size - old_size) + return VDI.VDI.get_params(self) + + def clone(self, sr_uuid, vdi_uuid): +@@ -1756,13 +1748,13 @@ class LinstorVDI(VDI.VDI): + if volume_info is None: + volume_info = self._linstor.get_volume_info(self.uuid) + +- # Contains the physical size used on all disks. ++ # Contains the max physical size used on a disk. + # When LINSTOR LVM driver is used, the size should be similar to + # virtual size (i.e. the LINSTOR max volume size). + # When LINSTOR Thin LVM driver is used, the used physical size should + # be lower than virtual size at creation. + # The physical size increases after each write in a new block. +- self.utilisation = volume_info.physical_size ++ self.utilisation = volume_info.allocated_size + self.capacity = volume_info.virtual_size + + if self.vdi_type == vhdutil.VDI_TYPE_RAW: +@@ -1958,7 +1950,7 @@ class LinstorVDI(VDI.VDI): + volume_info = self._linstor.get_volume_info(snap_uuid) + + snap_vdi.size = self.sr._vhdutil.get_size_virt(snap_uuid) +- snap_vdi.utilisation = volume_info.physical_size ++ snap_vdi.utilisation = volume_info.allocated_size + + # 6. Update sm config. + snap_vdi.sm_config = {} +@@ -2156,7 +2148,7 @@ class LinstorVDI(VDI.VDI): + raise + + if snap_type != VDI.SNAPSHOT_INTERNAL: +- self.sr._update_stats(self.capacity) ++ self.sr._update_stats(self.size) + + # 10. Return info on the new user-visible leaf VDI. + ret_vdi = snap_vdi +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index d617655f..a6f67d8d 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -131,20 +131,19 @@ class LinstorVolumeManager(object): + class VolumeInfo(object): + __slots__ = ( + 'name', +- 'physical_size', # Total physical size used by this volume on +- # all disks. ++ 'allocated_size', # Allocated size, place count is not used. + 'virtual_size' # Total virtual available size of this volume + # (i.e. the user size at creation). + ) + + def __init__(self, name): + self.name = name +- self.physical_size = 0 ++ self.allocated_size = 0 + self.virtual_size = 0 + + def __repr__(self): + return 'VolumeInfo("{}", {}, {})'.format( +- self.name, self.physical_size, self.virtual_size ++ self.name, self.allocated_size, self.virtual_size + ) + + # -------------------------------------------------------------------------- +@@ -248,9 +247,31 @@ class LinstorVolumeManager(object): + return self._compute_size('free_capacity') + + @property +- def total_allocated_volume_size(self): ++ def min_physical_size(self): + """ +- Give the sum of all created volumes. ++ Give the minimum physical size of the SR. ++ I.e. the size of the smallest disk. ++ :return: The physical min size. ++ :rtype: int ++ """ ++ size = None ++ for pool in self._get_storage_pools(force=True): ++ space = pool.free_space ++ if space: ++ current_size = space.total_capacity ++ if current_size < 0: ++ raise LinstorVolumeManagerError( ++ 'Failed to get pool total_capacity attr of `{}`' ++ .format(pool.node_name) ++ ) ++ if size is None or current_size < size: ++ size = current_size ++ return size * 1024 ++ ++ @property ++ def total_volume_size(self): ++ """ ++ Give the sum of all created volumes. The place count is used. + :return: The physical required size to use the volumes. + :rtype: int + """ +@@ -269,6 +290,37 @@ class LinstorVolumeManager(object): + size += current_size + return size * 1024 + ++ @property ++ def allocated_volume_size(self): ++ """ ++ Give the allocated size for all volumes. The place count is not ++ used here. When thick lvm is used, the size for one volume should ++ be equal to the virtual volume size. With thin lvm, the size is equal ++ or lower to the volume size. ++ :return: The allocated size of all volumes. ++ :rtype: int ++ """ ++ ++ size = 0 ++ for resource in self._get_resource_cache().resources: ++ volume_size = None ++ for volume in resource.volumes: ++ # We ignore diskless pools of the form "DfltDisklessStorPool". ++ if volume.storage_pool_name == self._group_name: ++ current_size = volume.allocated_size ++ if current_size < 0: ++ raise LinstorVolumeManagerError( ++ 'Failed to get allocated size of `{}` on `{}`' ++ .format(resource.name, volume.storage_pool_name) ++ ) ++ ++ if volume_size is None or current_size > volume_size: ++ volume_size = current_size ++ if volume_size is not None: ++ size += volume_size ++ ++ return size * 1024 ++ + @property + def metadata(self): + """ +@@ -1328,7 +1380,11 @@ class LinstorVolumeManager(object): + 'Failed to get allocated size of `{}` on `{}`' + .format(resource.name, volume.storage_pool_name) + ) +- current.physical_size += volume.allocated_size ++ allocated_size = volume.allocated_size ++ ++ current.allocated_size = current.allocated_size and \ ++ max(current.allocated_size, allocated_size) or \ ++ allocated_size + + if volume.usable_size < 0: + raise LinstorVolumeManagerError( +@@ -1341,7 +1397,7 @@ class LinstorVolumeManager(object): + min(current.virtual_size, virtual_size) or virtual_size + + for current in all_volume_info.values(): +- current.physical_size *= 1024 ++ current.allocated_size *= 1024 + current.virtual_size *= 1024 + + self._volume_info_cache_dirty = False diff --git a/SOURCES/0031-feat-linstor-monitord-scan-all-LINSTOR-SRs-every-12-.patch b/SOURCES/0031-feat-linstor-monitord-scan-all-LINSTOR-SRs-every-12-.patch new file mode 100644 index 0000000..4e9cdd8 --- /dev/null +++ b/SOURCES/0031-feat-linstor-monitord-scan-all-LINSTOR-SRs-every-12-.patch @@ -0,0 +1,332 @@ +From bf497ab757571bdb722323cfb98aaa500ec68c44 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 12 Jan 2021 14:06:34 +0100 +Subject: [PATCH 031/177] feat(linstor-monitord): scan all LINSTOR SRs every 12 + minutes to update allocated size stats + +Signed-off-by: Ronan Abhamon +--- + linstor/linstor-monitord.c | 194 ++++++++++++++++++++++++++++++++----- + 1 file changed, 171 insertions(+), 23 deletions(-) + +diff --git a/linstor/linstor-monitord.c b/linstor/linstor-monitord.c +index 8161813d..a1592fda 100644 +--- a/linstor/linstor-monitord.c ++++ b/linstor/linstor-monitord.c +@@ -14,8 +14,10 @@ + * along with this program. If not, see . + */ + ++#include + #include + #include ++#include + #include + #include + #include +@@ -39,7 +41,8 @@ + #define POOL_CONF_ABS_FILE POOL_CONF_DIR "/" POOL_CONF_FILE + + // In milliseconds. +-#define POLL_TIMEOUT 2000 ++#define UPDATE_LINSTOR_NODE_TIMEOUT 2000 ++#define SR_SCAN_TIMEOUT 720000 + + // ----------------------------------------------------------------------------- + +@@ -130,24 +133,120 @@ static inline int isMasterHost (int *error) { + + typedef struct { + int inotifyFd; ++ struct timespec lastScanTime; ++ int isMaster; + // TODO: Should be completed with at least a hostname field. + } State; + + // ----------------------------------------------------------------------------- + +-static inline int execCommand (char *argv[]) { ++typedef struct { ++ char *data; ++ size_t size; ++ size_t capacity; ++} Buffer; ++ ++#define max(a, b) ({ \ ++ __typeof__(a) _a = (a); \ ++ __typeof__(b) _b = (b); \ ++ _a > _b ? _a : _b; \ ++}) ++ ++static inline ssize_t readAll (int fd, Buffer *buffer) { ++ assert(buffer->capacity >= buffer->size); ++ ++ ssize_t ret = 0; ++ do { ++ size_t byteCount = buffer->capacity - buffer->size; ++ if (byteCount < 16) { ++ const size_t newCapacity = max(buffer->capacity << 1, 64); ++ char *p = realloc(buffer->data, newCapacity); ++ if (!p) ++ return -errno; ++ ++ buffer->data = p; ++ buffer->capacity = newCapacity; ++ ++ byteCount = buffer->capacity - buffer->size; ++ } ++ ++ ret = read(fd, buffer->data + buffer->size, byteCount); ++ if (ret > 0) ++ buffer->size += ret; ++ else if (ret < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) ++ ret = 0; ++ } while (ret > 0); ++ ++ return ret; ++} ++ ++// ----------------------------------------------------------------------------- ++ ++static inline int execCommand (char *argv[], Buffer *buffer) { ++ int pipefd[2]; ++ if (buffer) { ++ if (pipe(pipefd) < 0) { ++ syslog(LOG_ERR, "Failed to exec pipe: `%s`.", strerror(errno)); ++ return -errno; ++ } ++ ++ if (fcntl(pipefd[0], F_SETFL, O_NONBLOCK) < 0) { ++ syslog(LOG_ERR, "Failed to exec fcntl on pipe in: `%s`.", strerror(errno)); ++ close(pipefd[0]); ++ close(pipefd[1]); ++ return -errno; ++ } ++ } ++ + const pid_t pid = fork(); +- if (pid < 0) ++ if (pid < 0) { ++ syslog(LOG_ERR, "Failed to fork: `%s`.", strerror(errno)); ++ if (buffer) { ++ close(pipefd[0]); ++ close(pipefd[1]); ++ } + return -errno; ++ } + + // Child process. + if (pid == 0) { ++ if (buffer) { ++ close(STDOUT_FILENO); ++ dup(pipefd[1]); ++ ++ close(pipefd[0]); ++ close(pipefd[1]); ++ } ++ + if (execvp(*argv, argv) < 0) + syslog(LOG_ERR, "Failed to exec `%s` command.", *argv); + exit(EXIT_FAILURE); + } + + // Main process. ++ int ret = 0; ++ if (buffer) { ++ close(pipefd[1]); ++ ++ do { ++ struct pollfd fds = { pipefd[0], POLLIN | POLLHUP, 0 }; ++ const int res = poll(&fds, 1, 0); ++ if (res < 0) { ++ if (errno == EAGAIN) ++ continue; ++ syslog(LOG_ERR, "Failed to poll from command: `%s`.", strerror(errno)); ++ ret = -errno; ++ } else if (res > 0) { ++ if (fds.revents & POLLIN) ++ ret = readAll(pipefd[0], buffer); ++ if (fds.revents & POLLHUP) ++ break; // Input has been closed. ++ } ++ } while (ret >= 0); ++ ++ close(pipefd[0]); ++ } ++ + int status; + if (waitpid(pid, &status, 0) < 0) { + syslog(LOG_ERR, "Failed to wait command: `%s`.", *argv); +@@ -163,7 +262,7 @@ static inline int execCommand (char *argv[]) { + } else if (WIFSIGNALED(status)) + syslog(LOG_ERR, "`%s` terminated by signal %d.", *argv, WTERMSIG(status)); + +- return 0; ++ return ret; + } + + // ----------------------------------------------------------------------------- +@@ -188,12 +287,7 @@ static inline int addInotifyWatch (int inotifyFd, const char *filepath, uint32_t + + // ----------------------------------------------------------------------------- + +-static inline int updateLinstorServices () { +- int error; +- const int isMaster = isMasterHost(&error); +- if (error) +- return error; +- ++static inline int updateLinstorController (int isMaster) { + syslog(LOG_INFO, "%s linstor-controller...", isMaster ? "Enabling" : "Disabling"); + char *argv[] = { + "systemctl", +@@ -202,7 +296,7 @@ static inline int updateLinstorServices () { + "linstor-controller", + NULL + }; +- return execCommand(argv); ++ return execCommand(argv, NULL); + } + + static inline int updateLinstorNode (State *state) { +@@ -219,14 +313,53 @@ static inline int updateLinstorNode (State *state) { + + // ----------------------------------------------------------------------------- + ++#define UUID_PARAM "uuid=" ++#define UUID_PARAM_LEN (sizeof(UUID_PARAM) - 1) ++#define UUID_LENGTH 36 ++ ++static inline void scanLinstorSr (const char *uuid) { ++ char uuidBuf[UUID_LENGTH + UUID_PARAM_LEN + 1] = UUID_PARAM; ++ strncpy(uuidBuf + UUID_PARAM_LEN, uuid, UUID_LENGTH); ++ uuidBuf[UUID_LENGTH + UUID_PARAM_LEN] = '\0'; ++ execCommand((char *[]){ "xe", "sr-scan", uuidBuf, NULL }, NULL); ++} ++ ++// Called to update the physical/virtual size used by LINSTOR SRs in XAPI DB. ++static inline int scanLinstorSrs () { ++ Buffer srs = {}; ++ const int ret = execCommand((char *[]){ "xe", "sr-list", "type=linstor", "--minimal", NULL }, &srs); ++ if (ret) { ++ free(srs.data); ++ return ret; ++ } ++ ++ const char *end = srs.data + srs.size; ++ char *pos = srs.data; ++ for (char *off; (off = memchr(pos, ',', end - pos)); pos = off + 1) ++ if (off - pos == UUID_LENGTH) ++ scanLinstorSr(pos); ++ ++ if (end - pos >= UUID_LENGTH) { ++ for (--end; end - pos >= UUID_LENGTH && isspace(*end); --end) {} ++ if (isalnum(*end)) ++ scanLinstorSr(pos); ++ } ++ ++ free(srs.data); ++ ++ return 0; ++} ++ ++// ----------------------------------------------------------------------------- ++ + #define PROCESS_MODE_DEFAULT 0 + #define PROCESS_MODE_WAIT_FILE_CREATION 1 + + static inline int waitForPoolConfCreation (State *state, int *wdFile); + +-static inline int processPoolConfEvents (int inotifyFd, int wd, char **buffer, size_t *bufferSize, int mode, int *process) { ++static inline int processPoolConfEvents (State *state, int wd, char **buffer, size_t *bufferSize, int mode, int *process) { + size_t size = 0; +- if (ioctl(inotifyFd, FIONREAD, (char *)&size) == -1) { ++ if (ioctl(state->inotifyFd, FIONREAD, (char *)&size) == -1) { + syslog(LOG_ERR, "Failed to get buffer size from inotify descriptor: `%s`.", strerror(errno)); + return -errno; + } +@@ -241,7 +374,7 @@ static inline int processPoolConfEvents (int inotifyFd, int wd, char **buffer, s + *bufferSize = size; + } + +- if ((size = (size_t)read(inotifyFd, *buffer, size)) == (size_t)-1) { ++ if ((size = (size_t)read(state->inotifyFd, *buffer, size)) == (size_t)-1) { + syslog(LOG_ERR, "Failed to read buffer from inotify descriptor: `%s`.", strerror(errno)); + return -errno; + } +@@ -280,10 +413,10 @@ static inline int processPoolConfEvents (int inotifyFd, int wd, char **buffer, s + syslog(LOG_INFO, "Updating linstor services... (Inotify mask=%" PRIu32 ")", mask); + if (mask & (IN_DELETE_SELF | IN_MOVE_SELF | IN_UNMOUNT)) { + syslog(LOG_ERR, "Watched `" POOL_CONF_ABS_FILE "` file has been removed!"); +- inotify_rm_watch(inotifyFd, wd); // Do not forget to remove watch to avoid leaks. ++ inotify_rm_watch(state->inotifyFd, wd); // Do not forget to remove watch to avoid leaks. + return -EIO; + } +- ret = updateLinstorServices(); ++ ret = updateLinstorController(state->isMaster); + } else { + if (mask & (IN_CREATE | IN_MOVED_TO)) { + syslog(LOG_ERR, "Watched `" POOL_CONF_ABS_FILE "` file has been recreated!"); +@@ -303,16 +436,24 @@ static inline int waitAndProcessEvents (State *state, int wd, int mode) { + + struct timespec previousTime = getCurrentTime(); + do { +- struct timespec currentTime = getCurrentTime(); ++ const struct timespec currentTime = getCurrentTime(); + const int64_t elapsedTime = convertToMilliseconds(getTimeDiff(¤tTime, &previousTime)); + + int timeout; +- if (elapsedTime >= POLL_TIMEOUT) { ++ if (elapsedTime >= UPDATE_LINSTOR_NODE_TIMEOUT) { + updateLinstorNode(state); +- timeout = POLL_TIMEOUT; ++ timeout = UPDATE_LINSTOR_NODE_TIMEOUT; + previousTime = getCurrentTime(); + } else { +- timeout = POLL_TIMEOUT - elapsedTime; ++ timeout = UPDATE_LINSTOR_NODE_TIMEOUT - elapsedTime; ++ } ++ ++ const int64_t elapsedScanTime = convertToMilliseconds(getTimeDiff(¤tTime, &state->lastScanTime)); ++ if (elapsedScanTime >= SR_SCAN_TIMEOUT) { ++ state->isMaster = isMasterHost(&ret); ++ if (state->isMaster) ++ scanLinstorSrs(); ++ state->lastScanTime = getCurrentTime(); + } + + struct pollfd fds = { state->inotifyFd, POLLIN, 0 }; +@@ -323,7 +464,9 @@ static inline int waitAndProcessEvents (State *state, int wd, int mode) { + syslog(LOG_ERR, "Failed to poll from inotify descriptor: `%s`.", strerror(errno)); + ret = -errno; + } else if (res > 0) { +- ret = processPoolConfEvents(state->inotifyFd, wd, &buffer, &bufferSize, mode, &process); ++ state->isMaster = isMasterHost(&ret); ++ if (!ret) ++ ret = processPoolConfEvents(state, wd, &buffer, &bufferSize, mode, &process); + } + } while (ret >= 0 && process); + +@@ -350,7 +493,10 @@ static inline int waitForPoolConfCreation (State *state, int *wdFile) { + do { + do { + // Update LINSTOR services... +- ret = updateLinstorServices(); ++ int ret; ++ state->isMaster = isMasterHost(&ret); ++ if (!ret) ++ ret = updateLinstorController(state->isMaster); + + // Ok we can't read the pool configuration file. + // Maybe the file doesn't exist. Waiting its creation... +@@ -378,7 +524,9 @@ int main (int argc, char *argv[]) { + setlogmask(LOG_UPTO(LOG_INFO)); + + State state = { +- .inotifyFd = -1 ++ .inotifyFd = -1, ++ .lastScanTime = getCurrentTime(), ++ .isMaster = 0 + }; + + const int inotifyFd = createInotifyInstance(); diff --git a/SOURCES/0032-fix-LinstorSR-call-correctly-method-in-_locked_load-.patch b/SOURCES/0032-fix-LinstorSR-call-correctly-method-in-_locked_load-.patch new file mode 100644 index 0000000..8ab48d0 --- /dev/null +++ b/SOURCES/0032-fix-LinstorSR-call-correctly-method-in-_locked_load-.patch @@ -0,0 +1,41 @@ +From 4d41a373da1b9e62d727553cc16f994362d7c5d2 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 15 Jan 2021 17:01:05 +0100 +Subject: [PATCH 032/177] fix(LinstorSR): call correctly method in _locked_load + when vdi_attach_from_config is executed + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 19 ++++++------------- + 1 file changed, 6 insertions(+), 13 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 16cb0d62..2df2d681 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -347,19 +347,12 @@ class LinstorSR(SR.SR): + self._master_uri, self._group_name, logger=util.SMlog + ) + +- try: +- self._linstor = LinstorVolumeManager( +- self._master_uri, +- self._group_name, +- logger=util.SMlog +- ) +- return +- except Exception as e: +- util.SMlog( +- 'Ignore exception. Failed to build LINSTOR ' +- 'instance without session: {}'.format(e) +- ) +- return ++ self._linstor = LinstorVolumeManager( ++ self._master_uri, ++ self._group_name, ++ logger=util.SMlog ++ ) ++ return method(self, *args, **kwargs) + + self._master_uri = 'linstor://{}'.format( + util.get_master_rec(self.session)['address'] diff --git a/SOURCES/0033-feat-LinstorSR-integrate-minidrbdcluster-daemon.patch b/SOURCES/0033-feat-LinstorSR-integrate-minidrbdcluster-daemon.patch new file mode 100644 index 0000000..39b492c --- /dev/null +++ b/SOURCES/0033-feat-LinstorSR-integrate-minidrbdcluster-daemon.patch @@ -0,0 +1,2066 @@ +From 2390152b1ccf081874356ded5ee9ae5bad0f00b6 Mon Sep 17 00:00:00 2001 +From: Wescoeur +Date: Wed, 20 Jan 2021 18:04:26 +0100 +Subject: [PATCH 033/177] feat(LinstorSR): integrate minidrbdcluster daemon + +Now, we can: +- Start a controller on any node +- Share the LINSTOR volume list using a specific volume "xcp-persistent-database" +- Use the HA with "xcp-persistent-ha-statefile" and "xcp-persistent-redo-log" volumes +- Create the nodes automatically during SR creation + +Signed-off-by: Ronan Abhamon +--- + Makefile | 14 + + drivers/LinstorSR.py | 269 ++++--- + drivers/cleanup.py | 12 +- + drivers/linstor-manager | 111 ++- + drivers/linstorjournaler.py | 40 +- + drivers/linstorvolumemanager.py | 691 +++++++++++++++--- + drivers/tapdisk-pause | 8 +- + drivers/util.py | 21 - + etc/minidrbdcluster.ini | 14 + + .../linstor-satellite.service.d/override.conf | 5 + + etc/systemd/system/var-lib-linstor.mount | 6 + + linstor/linstor-monitord.c | 15 - + scripts/minidrbdcluster | 171 +++++ + systemd/minidrbdcluster.service | 18 + + 14 files changed, 1124 insertions(+), 271 deletions(-) + create mode 100644 etc/minidrbdcluster.ini + create mode 100644 etc/systemd/system/linstor-satellite.service.d/override.conf + create mode 100644 etc/systemd/system/var-lib-linstor.mount + create mode 100755 scripts/minidrbdcluster + create mode 100644 systemd/minidrbdcluster.service + +diff --git a/Makefile b/Makefile +index 284b9a39..43dd5692 100755 +--- a/Makefile ++++ b/Makefile +@@ -92,6 +92,7 @@ PLUGIN_SCRIPT_DEST := /etc/xapi.d/plugins/ + LIBEXEC := /opt/xensource/libexec/ + UDEV_RULES_DIR := /etc/udev/rules.d/ + UDEV_SCRIPTS_DIR := /etc/udev/scripts/ ++SYSTEMD_CONF_DIR := /etc/systemd/system/ + SYSTEMD_SERVICE_DIR := /usr/lib/systemd/system/ + INIT_DIR := /etc/rc.d/init.d/ + MPATH_CONF_DIR := /etc/multipath.xenserver/ +@@ -99,6 +100,7 @@ MPATH_CUSTOM_CONF_DIR := /etc/multipath/conf.d/ + MODPROBE_DIR := /etc/modprobe.d/ + EXTENSION_SCRIPT_DEST := /etc/xapi.d/extensions/ + LOGROTATE_DIR := /etc/logrotate.d/ ++MINI_DRBD_CLUSTER_CONF_DIR := /etc/ + + SM_STAGING := $(DESTDIR) + SM_STAMP := $(MY_OBJ_DIR)/.staging_stamp +@@ -147,11 +149,14 @@ install: precheck + mkdir -p $(SM_STAGING)$(UDEV_RULES_DIR) + mkdir -p $(SM_STAGING)$(UDEV_SCRIPTS_DIR) + mkdir -p $(SM_STAGING)$(INIT_DIR) ++ mkdir -p $(SM_STAGING)$(SYSTEMD_CONF_DIR) ++ mkdir -p $(SM_STAGING)$(SYSTEMD_CONF_DIR)/linstor-satellite.service.d + mkdir -p $(SM_STAGING)$(SYSTEMD_SERVICE_DIR) + mkdir -p $(SM_STAGING)$(MPATH_CONF_DIR) + mkdir -p $(SM_STAGING)$(MPATH_CUSTOM_CONF_DIR) + mkdir -p $(SM_STAGING)$(MODPROBE_DIR) + mkdir -p $(SM_STAGING)$(LOGROTATE_DIR) ++ mkdir -p $(SM_STAGING)$(MINI_DRBD_CLUSTER_CONF_DIR) + mkdir -p $(SM_STAGING)$(DEBUG_DEST) + mkdir -p $(SM_STAGING)$(BIN_DEST) + mkdir -p $(SM_STAGING)$(MASTER_SCRIPT_DEST) +@@ -175,6 +180,12 @@ install: precheck + $(SM_STAGING)/$(SM_DEST) + install -m 644 etc/logrotate.d/$(SMLOG_CONF) \ + $(SM_STAGING)/$(LOGROTATE_DIR) ++ install -m 644 etc/systemd/system/linstor-satellite.service.d/override.conf \ ++ $(SM_STAGING)/$(SYSTEMD_CONF_DIR)/linstor-satellite.service.d/ ++ install -m 644 etc/systemd/system/var-lib-linstor.mount \ ++ $(SM_STAGING)/$(SYSTEMD_CONF_DIR) ++ install -m 644 etc/minidrbdcluster.ini \ ++ $(SM_STAGING)/$(MINI_DRBD_CLUSTER_CONF_DIR) + install -m 644 etc/make-dummy-sr.service \ + $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) + install -m 644 systemd/xs-sm.service \ +@@ -193,6 +204,8 @@ install: precheck + $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) + install -m 644 systemd/linstor-monitor.service \ + $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) ++ install -m 644 systemd/minidrbdcluster.service \ ++ $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) + for i in $(UDEV_RULES); do \ + install -m 644 udev/$$i.rules \ + $(SM_STAGING)$(UDEV_RULES_DIR); done +@@ -242,6 +255,7 @@ install: precheck + install -m 755 scripts/xe-getlunidentifier $(SM_STAGING)$(BIN_DEST) + install -m 755 scripts/make-dummy-sr $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/storage-init $(SM_STAGING)$(LIBEXEC) ++ install -m 755 scripts/minidrbdcluster $(SM_STAGING)$(LIBEXEC) + + .PHONY: clean + clean: +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 2df2d681..9650d712 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -19,8 +19,11 @@ from constants import CBTLOG_TAG + try: + from linstorjournaler import LinstorJournaler + from linstorvhdutil import LinstorVhdUtil +- from linstorvolumemanager \ +- import LinstorVolumeManager, LinstorVolumeManagerError ++ from linstorvolumemanager import get_controller_uri ++ from linstorvolumemanager import get_controller_node_name ++ from linstorvolumemanager import LinstorVolumeManager ++ from linstorvolumemanager import LinstorVolumeManagerError ++ + LINSTOR_AVAILABLE = True + except ImportError: + LINSTOR_AVAILABLE = False +@@ -310,7 +313,7 @@ class LinstorSR(SR.SR): + self.lock = Lock(vhdutil.LOCK_TYPE_SR, self.uuid) + self.sr_vditype = SR.DEFAULT_TAP + +- self._hosts = self.dconf['hosts'].split(',') ++ self._hosts = list(set(self.dconf['hosts'].split(','))) + self._redundancy = int(self.dconf['redundancy'] or 1) + self._linstor = None # Ensure that LINSTOR attribute exists. + self._journaler = None +@@ -320,7 +323,6 @@ class LinstorSR(SR.SR): + self._is_master = True + self._group_name = self.dconf['group-name'] + +- self._master_uri = None + self._vdi_shared_time = 0 + + self._initialized = False +@@ -340,24 +342,18 @@ class LinstorSR(SR.SR): + if self.srcmd.cmd == 'vdi_attach_from_config': + # We must have a valid LINSTOR instance here without using + # the XAPI. +- self._master_uri = 'linstor://{}'.format( +- util.get_master_address() +- ) ++ controller_uri = get_controller_uri() + self._journaler = LinstorJournaler( +- self._master_uri, self._group_name, logger=util.SMlog ++ controller_uri, self._group_name, logger=util.SMlog + ) + + self._linstor = LinstorVolumeManager( +- self._master_uri, ++ controller_uri, + self._group_name, + logger=util.SMlog + ) + return method(self, *args, **kwargs) + +- self._master_uri = 'linstor://{}'.format( +- util.get_master_rec(self.session)['address'] +- ) +- + if not self._is_master: + if self.cmd in [ + 'sr_create', 'sr_delete', 'sr_update', 'sr_probe', +@@ -376,43 +372,31 @@ class LinstorSR(SR.SR): + self._shared_lock_vdi(self.srcmd.params['vdi_uuid']) + self._vdi_shared_time = time.time() + +- self._journaler = LinstorJournaler( +- self._master_uri, self._group_name, logger=util.SMlog +- ) ++ if self.srcmd.cmd != 'sr_create' and self.srcmd.cmd != 'sr_detach': ++ try: ++ controller_uri = get_controller_uri() + +- # Ensure ports are opened and LINSTOR controller/satellite +- # are activated. +- if self.srcmd.cmd == 'sr_create': +- # TODO: Disable if necessary +- self._enable_linstor_on_all_hosts(status=True) ++ self._journaler = LinstorJournaler( ++ controller_uri, self._group_name, logger=util.SMlog ++ ) + +- try: +- # Try to open SR if exists. +- # We can repair only if we are on the master AND if +- # we are trying to execute an exclusive operation. +- # Otherwise we could try to delete a VDI being created or +- # during a snapshot. An exclusive op is the guarantee that the +- # SR is locked. +- self._linstor = LinstorVolumeManager( +- self._master_uri, +- self._group_name, +- repair=( +- self._is_master and +- self.srcmd.cmd in self.ops_exclusive +- ), +- logger=util.SMlog +- ) +- self._vhdutil = LinstorVhdUtil(self.session, self._linstor) +- except Exception as e: +- if self.srcmd.cmd == 'sr_create' or \ +- self.srcmd.cmd == 'sr_detach': +- # Ignore exception in this specific case: sr_create. +- # At this moment the LinstorVolumeManager cannot be +- # instantiated. Concerning the sr_detach command, we must +- # ignore LINSTOR exceptions (if the volume group doesn't +- # exist for example after a bad user action). +- pass +- else: ++ # Try to open SR if exists. ++ # We can repair only if we are on the master AND if ++ # we are trying to execute an exclusive operation. ++ # Otherwise we could try to delete a VDI being created or ++ # during a snapshot. An exclusive op is the guarantee that ++ # the SR is locked. ++ self._linstor = LinstorVolumeManager( ++ controller_uri, ++ self._group_name, ++ repair=( ++ self._is_master and ++ self.srcmd.cmd in self.ops_exclusive ++ ), ++ logger=util.SMlog ++ ) ++ self._vhdutil = LinstorVhdUtil(self.session, self._linstor) ++ except Exception as e: + raise xs_errors.XenError('SRUnavailable', opterr=str(e)) + + if self._linstor: +@@ -507,13 +491,44 @@ class LinstorSR(SR.SR): + opterr='group name must be unique' + ) + ++ if srs: ++ raise xs_errors.XenError( ++ 'LinstorSRCreate', ++ opterr='LINSTOR SR must be unique in a pool' ++ ) ++ ++ online_hosts = util.get_online_hosts(self.session) ++ if len(online_hosts) < len(self._hosts): ++ raise xs_errors.XenError( ++ 'LinstorSRCreate', ++ opterr='Not enough online hosts' ++ ) ++ ++ ips = {} ++ for host in online_hosts: ++ record = self.session.xenapi.host.get_record(host) ++ hostname = record['hostname'] ++ if hostname in self._hosts: ++ ips[hostname] = record['address'] ++ ++ if len(ips) != len(self._hosts): ++ raise xs_errors.XenError( ++ 'LinstorSRCreate', ++ opterr='Not enough online hosts' ++ ) ++ ++ # Ensure ports are opened and LINSTOR satellites ++ # are activated. In the same time the minidrbdcluster instances ++ # must be stopped. ++ self._prepare_sr_on_all_hosts(enabled=True) ++ + # Create SR. + # Throw if the SR already exists. + try: + self._linstor = LinstorVolumeManager.create_sr( +- self._master_uri, + self._group_name, + self._hosts, ++ ips, + self._redundancy, + thin_provisioning=self._provisioning == 'thin', + logger=util.SMlog +@@ -523,30 +538,79 @@ class LinstorSR(SR.SR): + util.SMlog('Failed to create LINSTOR SR: {}'.format(e)) + raise xs_errors.XenError('LinstorSRCreate', opterr=str(e)) + ++ try: ++ util.SMlog( ++ "Finishing SR creation, enable minidrbdcluster on all hosts..." ++ ) ++ self._update_minidrbdcluster_on_all_hosts(enabled=True) ++ except Exception as e: ++ try: ++ self._linstor.destroy() ++ except Exception as e2: ++ util.SMlog( ++ 'Failed to destroy LINSTOR SR after creation fail: {}' ++ .format(e2) ++ ) ++ raise e ++ + @_locked_load + def delete(self, uuid): + util.SMlog('LinstorSR.delete for {}'.format(self.uuid)) + cleanup.gc_force(self.session, self.uuid) + +- if self.vdis: ++ if self.vdis or self._linstor._volumes: + raise xs_errors.XenError('SRNotEmpty') + +- try: +- # TODO: Use specific exceptions. If the LINSTOR group doesn't +- # exist, we can remove it without problem. ++ node_name = get_controller_node_name() ++ if not node_name: ++ raise xs_errors.XenError( ++ 'LinstorSRDelete', ++ opterr='Cannot get controller node name' ++ ) + +- # TODO: Maybe remove all volumes unused by the SMAPI. +- # We must ensure it's a safe idea... ++ host = None ++ if node_name == 'localhost': ++ host = util.get_this_host_ref(self.session) ++ else: ++ for slave in util.get_all_slaves(self.session): ++ r_name = self.session.xenapi.host.get_record(slave)['hostname'] ++ if r_name == node_name: ++ host = slave ++ break + +- self._linstor.destroy() +- Lock.cleanupAll(self.uuid) ++ if not host: ++ raise xs_errors.XenError( ++ 'LinstorSRDelete', ++ opterr='Failed to find host with hostname: {}'.format( ++ node_name ++ ) ++ ) ++ ++ try: ++ self._update_minidrbdcluster_on_all_hosts(enabled=False) ++ ++ args = { ++ 'groupName': self._group_name, ++ } ++ self._exec_manager_command( ++ host, 'destroy', args, 'LinstorSRDelete' ++ ) + except Exception as e: ++ try: ++ self._update_minidrbdcluster_on_all_hosts(enabled=True) ++ except Exception as e2: ++ util.SMlog( ++ 'Failed to restart minidrbdcluster after destroy fail: {}' ++ .format(e2) ++ ) + util.SMlog('Failed to delete LINSTOR SR: {}'.format(e)) + raise xs_errors.XenError( + 'LinstorSRDelete', + opterr=str(e) + ) + ++ Lock.cleanupAll(self.uuid) ++ + @_locked_load + def update(self, uuid): + util.SMlog('LinstorSR.update for {}'.format(self.uuid)) +@@ -626,10 +690,9 @@ class LinstorSR(SR.SR): + # -------------------------------------------------------------------------- + + def _shared_lock_vdi(self, vdi_uuid, locked=True): +- pools = self.session.xenapi.pool.get_all() +- master = self.session.xenapi.pool.get_master(pools[0]) ++ master = util.get_master_ref(self.session) + +- method = 'lockVdi' ++ command = 'lockVdi' + args = { + 'groupName': self._group_name, + 'srUuid': self.uuid, +@@ -654,48 +717,56 @@ class LinstorSR(SR.SR): + ) + return + +- ret = self.session.xenapi.host.call_plugin( +- master, self.MANAGER_PLUGIN, method, args +- ) +- util.SMlog( +- 'call-plugin ({} with {}) returned: {}' +- .format(method, args, ret) +- ) +- if ret == 'False': +- raise xs_errors.XenError( +- 'VDIUnavailable', +- opterr='Plugin {} failed'.format(self.MANAGER_PLUGIN) +- ) ++ self._exec_manager_command(master, command, args, 'VDIUnavailable') + + # -------------------------------------------------------------------------- + # Network. + # -------------------------------------------------------------------------- + +- def _enable_linstor(self, host, status): +- method = 'enable' +- args = {'enabled': str(bool(status))} +- ++ def _exec_manager_command(self, host, command, args, error): + ret = self.session.xenapi.host.call_plugin( +- host, self.MANAGER_PLUGIN, method, args ++ host, self.MANAGER_PLUGIN, command, args + ) + util.SMlog( +- 'call-plugin ({} with {}) returned: {}'.format(method, args, ret) ++ 'call-plugin ({}:{} with {}) returned: {}'.format( ++ self.MANAGER_PLUGIN, command, args, ret ++ ) + ) + if ret == 'False': + raise xs_errors.XenError( +- 'SRUnavailable', ++ error, + opterr='Plugin {} failed'.format(self.MANAGER_PLUGIN) + ) + +- def _enable_linstor_on_master(self, status): +- pools = self.session.xenapi.pool.get_all() +- master = self.session.xenapi.pool.get_master(pools[0]) +- self._enable_linstor(master, status) ++ def _prepare_sr(self, host, enabled): ++ self._exec_manager_command( ++ host, ++ 'prepareSr' if enabled else 'releaseSr', ++ {}, ++ 'SRUnavailable' ++ ) ++ ++ def _prepare_sr_on_all_hosts(self, enabled): ++ master = util.get_master_ref(self.session) ++ self._prepare_sr(master, enabled) + +- def _enable_linstor_on_all_hosts(self, status): +- self._enable_linstor_on_master(status) + for slave in util.get_all_slaves(self.session): +- self._enable_linstor(slave, status) ++ self._prepare_sr(slave, enabled) ++ ++ def _update_minidrbdcluster(self, host, enabled): ++ self._exec_manager_command( ++ host, ++ 'updateMinidrbdcluster', ++ {'enabled': str(enabled)}, ++ 'SRUnavailable' ++ ) ++ ++ def _update_minidrbdcluster_on_all_hosts(self, enabled): ++ master = util.get_master_ref(self.session) ++ self._update_minidrbdcluster(master, enabled) ++ ++ for slave in util.get_all_slaves(self.session): ++ self._update_minidrbdcluster(slave, enabled) + + # -------------------------------------------------------------------------- + # Metadata. +@@ -1384,8 +1455,15 @@ class LinstorVDI(VDI.VDI): + # 4. Create! + failed = False + try: ++ volume_name = None ++ if self.ty == 'ha_statefile': ++ volume_name = 'xcp-persistent-ha-statefile' ++ elif self.ty == 'redo_log': ++ volume_name = 'xcp-persistent-redo-log' ++ + self._linstor.create_volume( +- self.uuid, volume_size, persistent=False ++ self.uuid, volume_size, persistent=False, ++ volume_name=volume_name + ) + volume_info = self._linstor.get_volume_info(self.uuid) + +@@ -1822,25 +1900,14 @@ class LinstorVDI(VDI.VDI): + else: + fn = 'attach' if attach else 'detach' + +- # We assume the first pool is always the one currently in use. +- pools = self.session.xenapi.pool.get_all() +- master = self.session.xenapi.pool.get_master(pools[0]) ++ master = util.get_master_ref(self.session) ++ + args = { + 'groupName': self.sr._group_name, + 'srUuid': self.sr.uuid, + 'vdiUuid': self.uuid + } +- ret = self.session.xenapi.host.call_plugin( +- master, self.sr.MANAGER_PLUGIN, fn, args +- ) +- util.SMlog( +- 'call-plugin ({} with {}) returned: {}'.format(fn, args, ret) +- ) +- if ret == 'False': +- raise xs_errors.XenError( +- 'VDIUnavailable', +- opterr='Plugin {} failed'.format(self.sr.MANAGER_PLUGIN) +- ) ++ self.sr._exec_manager_command(master, fn, args, 'VDIUnavailable') + + # Reload size attrs after inflate or deflate! + self._load_this() +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 895f36e8..9e3a5b07 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -52,8 +52,10 @@ from srmetadata import LVMMetadataHandler, VDI_TYPE_TAG + try: + from linstorjournaler import LinstorJournaler + from linstorvhdutil import LinstorVhdUtil +- from linstorvolumemanager \ +- import LinstorVolumeManager, LinstorVolumeManagerError ++ from linstorvolumemanager import get_controller_uri ++ from linstorvolumemanager import LinstorVolumeManager ++ from linstorvolumemanager import LinstorVolumeManagerError ++ + LINSTOR_AVAILABLE = True + except ImportError: + LINSTOR_AVAILABLE = False +@@ -2873,7 +2875,6 @@ class LinstorSR(SR): + ) + + SR.__init__(self, uuid, xapi, createLock, force) +- self._master_uri = 'linstor://localhost' + self.path = LinstorVolumeManager.DEV_ROOT_PATH + self._reloadLinstor() + +@@ -2918,12 +2919,13 @@ class LinstorSR(SR): + dconf = session.xenapi.PBD.get_device_config(pbd) + group_name = dconf['group-name'] + ++ controller_uri = get_controller_uri() + self.journaler = LinstorJournaler( +- self._master_uri, group_name, logger=util.SMlog ++ controller_uri, group_name, logger=util.SMlog + ) + + self._linstor = LinstorVolumeManager( +- self._master_uri, ++ controller_uri, + group_name, + repair=True, + logger=util.SMlog +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index e7e58fd8..f82b73f2 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -22,7 +22,7 @@ import XenAPIPlugin + + sys.path.append('/opt/xensource/sm/') + from linstorjournaler import LinstorJournaler +-from linstorvolumemanager import LinstorVolumeManager ++from linstorvolumemanager import get_controller_uri, LinstorVolumeManager + from lock import Lock + import json + import LinstorSR +@@ -34,10 +34,6 @@ FIREWALL_PORT_SCRIPT = '/etc/xapi.d/plugins/firewall-port' + LINSTOR_PORTS = [3366, 3370, 3376, 3377, '7000:8000'] + + +-def get_linstor_uri(session): +- return 'linstor://{}'.format(util.get_master_rec(session)['address']) +- +- + def update_port(port, open): + fn = 'open' if open else 'close' + args = ( +@@ -55,23 +51,72 @@ def update_all_ports(open): + update_port(port, open) + + +-def update_service(start): ++def enable_and_start_service(name, start): + fn = 'enable' if start else 'disable' +- args = ('systemctl', fn, '--now', 'linstor-satellite') ++ args = ('systemctl', fn, '--now', name) + (ret, out, err) = util.doexec(args) + if ret == 0: + return +- raise Exception('Failed to {} satellite: {} {}'.format(fn, out, err)) ++ raise Exception('Failed to {} {}: {} {}'.format(fn, name, out, err)) ++ ++ ++def restart_service(name): ++ args = ('systemctl', 'restart', name) ++ (ret, out, err) = util.doexec(args) ++ if ret == 0: ++ return ++ raise Exception('Failed to restart {}: {} {}'.format(name, out, err)) ++ ++ ++def stop_service(name): ++ args = ('systemctl', 'stop', name) ++ (ret, out, err) = util.doexec(args) ++ if ret == 0: ++ return ++ raise Exception('Failed to stop {}: {} {}'.format(name, out, err)) ++ ++ ++def update_linstor_satellite_service(start): ++ enable_and_start_service('linstor-satellite', start) ++ ++ ++def update_minidrbdcluster_service(start): ++ enable_and_start_service('minidrbdcluster', start) ++ ++ ++def prepare_sr(session, args): ++ try: ++ update_all_ports(open=True) ++ # We don't want to enable and start minidrbdcluster daemon during ++ # SR creation. ++ update_minidrbdcluster_service(start=False) ++ update_linstor_satellite_service(start=True) ++ return str(True) ++ except Exception as e: ++ util.SMlog('linstor-manager:prepare_sr error: {}'.format(e)) ++ return str(False) + + +-def enable(session, args): ++def release_sr(session, args): ++ try: ++ update_linstor_satellite_service(start=False) ++ update_minidrbdcluster_service(start=False) ++ update_all_ports(open=False) ++ return str(True) ++ except Exception as e: ++ util.SMlog('linstor-manager:release_sr error: {}'.format(e)) ++ return str(False) ++ ++ ++def update_minidrbdcluster(session, args): + try: + enabled = distutils.util.strtobool(args['enabled']) +- update_all_ports(open=enabled) +- update_service(start=enabled) ++ update_minidrbdcluster_service(start=enabled) + return str(True) + except Exception as e: +- util.SMlog('linstor-manager:disable error: {}'.format(e)) ++ util.SMlog( ++ 'linstor-manager:update_minidrbdcluster error: {}'.format(e) ++ ) + return str(False) + + +@@ -81,12 +126,12 @@ def attach(session, args): + vdi_uuid = args['vdiUuid'] + group_name = args['groupName'] + +- linstor_uri = get_linstor_uri(session) ++ controller_uri = get_controller_uri() + journaler = LinstorJournaler( +- linstor_uri, group_name, logger=util.SMlog ++ controller_uri, group_name, logger=util.SMlog + ) + linstor = LinstorVolumeManager( +- linstor_uri, ++ controller_uri, + group_name, + logger=util.SMlog + ) +@@ -104,7 +149,7 @@ def detach(session, args): + group_name = args['groupName'] + + linstor = LinstorVolumeManager( +- get_linstor_uri(session), ++ get_controller_uri(), + group_name, + logger=util.SMlog + ) +@@ -115,6 +160,29 @@ def detach(session, args): + return str(False) + + ++def destroy(session, args): ++ try: ++ group_name = args['groupName'] ++ ++ # When destroy is called, there are no running minidrbdcluster daemons. ++ # So the controllers are stopped too, we must start an instance. ++ restart_service('var-lib-linstor.mount') ++ restart_service('linstor-controller') ++ ++ linstor = LinstorVolumeManager( ++ 'linstor://localhost', ++ group_name, ++ logger=util.SMlog ++ ) ++ linstor.destroy() ++ return str(True) ++ except Exception as e: ++ stop_service('linstor-controller') ++ stop_service('var-lib-linstor.mount') ++ util.SMlog('linstor-manager:destroy error: {}'.format(e)) ++ return str(False) ++ ++ + def check(session, args): + try: + device_path = args['devicePath'] +@@ -133,7 +201,7 @@ def get_vhd_info(session, args): + include_parent = distutils.util.strtobool(args['includeParent']) + + linstor = LinstorVolumeManager( +- get_linstor_uri(session), ++ get_controller_uri(), + group_name, + logger=util.SMlog + ) +@@ -168,7 +236,7 @@ def get_parent(session, args): + group_name = args['groupName'] + + linstor = LinstorVolumeManager( +- get_linstor_uri(session), ++ get_controller_uri(), + group_name, + logger=util.SMlog + ) +@@ -244,7 +312,7 @@ def lock_vdi(session, args): + lock.acquire() + + linstor = LinstorVolumeManager( +- get_linstor_uri(session), ++ get_controller_uri(), + group_name, + logger=util.SMlog + ) +@@ -261,9 +329,12 @@ def lock_vdi(session, args): + + if __name__ == '__main__': + XenAPIPlugin.dispatch({ +- 'enable': enable, ++ 'prepareSr': prepare_sr, ++ 'releaseSr': release_sr, ++ 'updateMinidrbdcluster': update_minidrbdcluster, + 'attach': attach, + 'detach': detach, ++ 'destroy': destroy, + 'check': check, + 'getVHDInfo': get_vhd_info, + 'hasParent': has_parent, +diff --git a/drivers/linstorjournaler.py b/drivers/linstorjournaler.py +index 74953305..285012ca 100755 +--- a/drivers/linstorjournaler.py ++++ b/drivers/linstorjournaler.py +@@ -16,7 +16,7 @@ + # + + +-from linstorvolumemanager import LinstorVolumeManager ++from linstorvolumemanager import get_controller_uri, LinstorVolumeManager + import linstor + import re + import util +@@ -52,20 +52,10 @@ class LinstorJournaler: + self._namespace = '{}journal/'.format( + LinstorVolumeManager._build_sr_namespace() + ) +- +- def connect(): +- self._journal = linstor.KV( +- LinstorVolumeManager._build_group_name(group_name), +- uri=uri, +- namespace=self._namespace +- ) +- +- util.retry( +- connect, +- maxretry=60, +- exceptions=[linstor.errors.LinstorNetworkError] +- ) + self._logger = logger ++ self._journal = self._create_journal_instance( ++ uri, group_name, self._namespace ++ ) + + def create(self, type, identifier, value): + # TODO: Maybe rename to 'add' in the future (in Citrix code too). +@@ -150,6 +140,28 @@ class LinstorJournaler: + def _reset_namespace(self): + self._journal.namespace = self._namespace + ++ @classmethod ++ def _create_journal_instance(cls, uri, group_name, namespace): ++ def connect(uri): ++ if not uri: ++ uri = get_controller_uri() ++ return linstor.KV( ++ LinstorVolumeManager._build_group_name(group_name), ++ uri=uri, ++ namespace=namespace ++ ) ++ ++ try: ++ return connect(uri) ++ except linstor.errors.LinstorNetworkError: ++ pass ++ ++ return util.retry( ++ lambda: connect(None), ++ maxretry=10, ++ exceptions=[linstor.errors.LinstorNetworkError] ++ ) ++ + @staticmethod + def _get_key(type, identifier): + return '{}/{}'.format(type, identifier) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index a6f67d8d..a383e327 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -16,15 +16,30 @@ + # + + ++import glob + import json + import linstor + import os.path + import re ++import shutil + import socket + import time + import util ++import uuid + + ++# Contains the data of the "/var/lib/linstor" directory. ++DATABASE_VOLUME_NAME = 'xcp-persistent-database' ++DATABASE_SIZE = 1 << 30 # 1GB. ++DATABASE_PATH = '/var/lib/linstor' ++DATABASE_MKFS = 'mkfs.ext4' ++ ++REG_DRBDADM_PRIMARY = re.compile("([^\\s]+)\\s+role:Primary") ++REG_DRBDSETUP_IP = re.compile('[^\\s]+\\s+(.*):.*$') ++ ++ ++# ============================================================================== ++ + def round_up(value, divisor): + assert divisor + divisor = int(divisor) +@@ -37,6 +52,79 @@ def round_down(value, divisor): + return value - (value % int(divisor)) + + ++# ============================================================================== ++ ++def get_remote_host_ip(node_name): ++ (ret, stdout, stderr) = util.doexec([ ++ 'drbdsetup', 'show', DATABASE_VOLUME_NAME, '--json' ++ ]) ++ if ret != 0: ++ return ++ ++ try: ++ conf = json.loads(stdout) ++ if not conf: ++ return ++ ++ for connection in conf[0]['connections']: ++ if connection['net']['_name'] == node_name: ++ value = connection['path']['_remote_host'] ++ res = REG_DRBDSETUP_IP.match(value) ++ if res: ++ return res.groups()[0] ++ break ++ except Exception: ++ pass ++ ++ ++def _get_controller_uri(): ++ (ret, stdout, stderr) = util.doexec([ ++ 'drbdadm', 'status', DATABASE_VOLUME_NAME ++ ]) ++ if ret != 0: ++ return ++ ++ if stdout.startswith('{} role:Primary'.format(DATABASE_VOLUME_NAME)): ++ return 'linstor://localhost' ++ ++ res = REG_DRBDADM_PRIMARY.search(stdout) ++ if res: ++ node_name = res.groups()[0] ++ ip = get_remote_host_ip(node_name) ++ if ip: ++ return 'linstor://' + ip ++ ++ ++def get_controller_uri(): ++ retries = 0 ++ while True: ++ uri = _get_controller_uri() ++ if uri: ++ return uri ++ ++ retries += 1 ++ if retries >= 10: ++ break ++ time.sleep(1) ++ ++ ++def get_controller_node_name(): ++ (ret, stdout, stderr) = util.doexec([ ++ 'drbdadm', 'status', DATABASE_VOLUME_NAME ++ ]) ++ if ret != 0: ++ return ++ ++ if stdout.startswith('{} role:Primary'.format(DATABASE_VOLUME_NAME)): ++ return 'localhost' ++ ++ res = REG_DRBDADM_PRIMARY.search(stdout) ++ if res: ++ return res.groups()[0] ++ ++ ++# ============================================================================== ++ + class LinstorVolumeManagerError(Exception): + ERR_GENERIC = 0, + ERR_VOLUME_EXISTS = 1, +@@ -50,6 +138,7 @@ class LinstorVolumeManagerError(Exception): + def code(self): + return self._code + ++ + # ============================================================================== + + # Note: +@@ -152,7 +241,7 @@ class LinstorVolumeManager(object): + self, uri, group_name, repair=False, logger=default_logger.__func__ + ): + """ +- Create a new LinstorApi object. ++ Create a new LinstorVolumeManager object. + :param str uri: URI to communicate with the LINSTOR controller. + :param str group_name: The SR goup name to use. + :param bool repair: If true we try to remove bad volumes due to a crash +@@ -160,7 +249,6 @@ class LinstorVolumeManager(object): + :param function logger: Function to log messages. + """ + +- self._uri = uri + self._linstor = self._create_linstor_instance(uri) + self._base_group_name = group_name + +@@ -266,7 +354,7 @@ class LinstorVolumeManager(object): + ) + if size is None or current_size < size: + size = current_size +- return size * 1024 ++ return (size or 0) * 1024 + + @property + def total_volume_size(self): +@@ -379,19 +467,24 @@ class LinstorVolumeManager(object): + """ + return volume_uuid in self._volumes + +- def create_volume(self, volume_uuid, size, persistent=True): ++ def create_volume( ++ self, volume_uuid, size, persistent=True, volume_name=None ++ ): + """ + Create a new volume on the SR. + :param str volume_uuid: The volume uuid to use. + :param int size: volume size in B. + :param bool persistent: If false the volume will be unavailable + on the next constructor call LinstorSR(...). ++ :param str volume_name: If set, this name is used in the LINSTOR ++ database instead of a generated name. + :return: The current device path of the volume. + :rtype: str + """ + + self._logger('Creating LINSTOR volume {}...'.format(volume_uuid)) +- volume_name = self.build_volume_name(util.gen_uuid()) ++ if not volume_name: ++ volume_name = self.build_volume_name(util.gen_uuid()) + volume_properties = self._create_volume_with_properties( + volume_uuid, volume_name, size, place_resources=True + ) +@@ -1073,23 +1166,56 @@ class LinstorVolumeManager(object): + if not volume_name or volume_name not in resource_names: + self.destroy_volume(volume_uuid) + +- def destroy(self, force=False): ++ def destroy(self): + """ + Destroy this SR. Object should not be used after that. + :param bool force: Try to destroy volumes before if true. + """ + +- if (force): +- for volume_uuid in self._volumes: +- self.destroy_volume(volume_uuid) ++ if self._volumes: ++ raise LinstorVolumeManagerError( ++ 'Cannot destroy LINSTOR volume manager: ' ++ 'It exists remaining volumes' ++ ) + +- # TODO: Throw exceptions in the helpers below if necessary. +- # TODO: What's the required action if it exists remaining volumes? ++ uri = 'linstor://localhost' ++ try: ++ self._start_controller(start=False) + +- self._destroy_resource_group(self._linstor, self._group_name) +- for pool in self._get_storage_pools(force=True): +- self._destroy_storage_pool( +- self._linstor, pool.name, pool.node_name ++ # 1. Umount LINSTOR database. ++ self._mount_database_volume( ++ self.build_device_path(DATABASE_VOLUME_NAME), ++ mount=False, ++ force=True ++ ) ++ ++ # 2. Refresh instance. ++ self._start_controller(start=True) ++ self._linstor = self._create_linstor_instance( ++ uri, keep_uri_unmodified=True ++ ) ++ ++ # 3. Destroy database volume. ++ self._destroy_resource(DATABASE_VOLUME_NAME) ++ ++ # 4. Destroy group and storage pools. ++ self._destroy_resource_group(self._linstor, self._group_name) ++ for pool in self._get_storage_pools(force=True): ++ self._destroy_storage_pool( ++ self._linstor, pool.name, pool.node_name ++ ) ++ except Exception as e: ++ self._start_controller(start=True) ++ raise e ++ ++ try: ++ self._start_controller(start=False) ++ for file in glob.glob(DATABASE_PATH + '/'): ++ os.remove(file) ++ except Exception as e: ++ util.SMlog( ++ 'Ignoring failure after LINSTOR SR destruction: {}' ++ .format(e) + ) + + def find_up_to_date_diskfull_nodes(self, volume_uuid): +@@ -1130,29 +1256,75 @@ class LinstorVolumeManager(object): + + @classmethod + def create_sr( +- cls, uri, group_name, node_names, redundancy, ++ cls, group_name, node_names, ips, redundancy, + thin_provisioning=False, + logger=default_logger.__func__ + ): + """ + Create a new SR on the given nodes. +- :param str uri: URI to communicate with the LINSTOR controller. + :param str group_name: The SR group_name to use. + :param list[str] node_names: String list of nodes. + :param int redundancy: How many copy of volumes should we store? ++ :param set(str) ips: Node ips + :param function logger: Function to log messages. + :return: A new LinstorSr instance. + :rtype: LinstorSr + """ + ++ try: ++ cls._start_controller(start=True) ++ sr = cls._create_sr( ++ group_name, ++ node_names, ++ ips, ++ redundancy, ++ thin_provisioning, ++ logger ++ ) ++ finally: ++ # Controller must be stopped and volume unmounted because ++ # it is the role of the minidrbdcluster daemon to do the right ++ # actions. ++ cls._start_controller(start=False) ++ cls._mount_volume( ++ cls.build_device_path(DATABASE_VOLUME_NAME), ++ DATABASE_PATH, ++ mount=False ++ ) ++ return sr ++ ++ @classmethod ++ def _create_sr( ++ cls, group_name, node_names, ips, redundancy, ++ thin_provisioning=False, ++ logger=default_logger.__func__ ++ ): + # 1. Check if SR already exists. +- lin = cls._create_linstor_instance(uri) ++ uri = 'linstor://localhost' ++ ++ lin = cls._create_linstor_instance(uri, keep_uri_unmodified=True) ++ ++ for node_name in node_names: ++ ip = ips[node_name] ++ result = lin.node_create( ++ node_name, ++ linstor.consts.VAL_NODE_TYPE_CMBD, ++ ip ++ ) ++ errors = cls._filter_errors(result) ++ if cls._check_errors(errors, [linstor.consts.FAIL_EXISTS_NODE]): ++ continue ++ ++ if errors: ++ raise LinstorVolumeManagerError( ++ 'Failed to create node `{}` with ip `{}`: {}'.format( ++ node_name, ip, cls._get_error_str(errors) ++ ) ++ ) ++ + driver_pool_name = group_name + group_name = cls._build_group_name(group_name) + pools = lin.storage_pool_list_raise(filter_by_stor_pools=[group_name]) +- +- # TODO: Maybe if the SR already exists and if the nodes are the same, +- # we can try to use it directly. + pools = pools.storage_pools + if pools: + existing_node_names = map(lambda pool: pool.node_name, pools) +@@ -1227,25 +1399,64 @@ class LinstorVolumeManager(object): + ) + ) + +- # 3. Remove storage pools/resource/volume group in the case of errors. ++ # 3. Create the LINSTOR database volume and mount it. ++ try: ++ logger('Creating database volume...') ++ volume_path = cls._create_database_volume(lin, group_name) ++ except LinstorVolumeManagerError as e: ++ if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: ++ logger('Destroying database volume after creation fail...') ++ cls._force_destroy_database_volume(lin, group_name) ++ raise ++ ++ try: ++ logger('Mounting database volume...') ++ ++ # First we must disable the controller to move safely the ++ # LINSTOR config. ++ cls._start_controller(start=False) ++ ++ cls._mount_database_volume(volume_path) ++ except Exception as e: ++ # Ensure we are connected because controller has been ++ # restarted during mount call. ++ logger('Destroying database volume after mount fail...') ++ ++ try: ++ cls._start_controller(start=True) ++ except Exception: ++ pass ++ ++ lin = cls._create_linstor_instance( ++ uri, keep_uri_unmodified=True ++ ) ++ cls._force_destroy_database_volume(lin, group_name) ++ raise e ++ ++ cls._start_controller(start=True) ++ lin = cls._create_linstor_instance(uri, keep_uri_unmodified=True) ++ ++ # 4. Remove storage pools/resource/volume group in the case of errors. + except Exception as e: ++ logger('Destroying resource group and storage pools after fail...') + try: + cls._destroy_resource_group(lin, group_name) +- except Exception: ++ except Exception as e2: ++ logger('Failed to destroy resource group: {}'.format(e2)) + pass + j = 0 + i = min(i, len(node_names) - 1) + while j <= i: + try: + cls._destroy_storage_pool(lin, group_name, node_names[j]) +- except Exception: ++ except Exception as e2: ++ logger('Failed to destroy resource group: {}'.format(e2)) + pass + j += 1 + raise e + +- # 4. Return new instance. ++ # 5. Return new instance. + instance = cls.__new__(cls) +- instance._uri = uri + instance._linstor = lin + instance._logger = logger + instance._redundancy = redundancy +@@ -1462,26 +1673,6 @@ class LinstorVolumeManager(object): + + return self._storage_pools + +- def _check_volume_creation_errors(self, result, volume_uuid): +- errors = self._filter_errors(result) +- if self._check_errors(errors, [ +- linstor.consts.FAIL_EXISTS_RSC, linstor.consts.FAIL_EXISTS_RSC_DFN +- ]): +- raise LinstorVolumeManagerError( +- 'Failed to create volume `{}` from SR `{}`, it already exists' +- .format(volume_uuid, self._group_name), +- LinstorVolumeManagerError.ERR_VOLUME_EXISTS +- ) +- +- if errors: +- raise LinstorVolumeManagerError( +- 'Failed to create volume `{}` from SR `{}`: {}'.format( +- volume_uuid, +- self._group_name, +- self._get_error_str(errors) +- ) +- ) +- + def _create_volume(self, volume_uuid, volume_name, size, place_resources): + size = self.round_up_volume_size(size) + +@@ -1491,7 +1682,7 @@ class LinstorVolumeManager(object): + rsc_dfn_name=volume_name, + vlm_sizes=['{}B'.format(size)], + definitions_only=not place_resources +- ), volume_uuid) ++ ), volume_uuid, self._group_name) + + def _create_volume_with_properties( + self, volume_uuid, volume_name, size, place_resources +@@ -1535,12 +1726,8 @@ class LinstorVolumeManager(object): + # before the `self._create_volume` case. + # It can only happen if the same volume uuid is used in the same + # call in another host. +- if e.code == LinstorVolumeManagerError.ERR_VOLUME_EXISTS: +- raise +- self._force_destroy_volume(volume_uuid) +- raise +- except Exception: +- self._force_destroy_volume(volume_uuid) ++ if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: ++ self._force_destroy_volume(volume_uuid) + raise + + def _find_device_path(self, volume_uuid, volume_name): +@@ -1576,7 +1763,10 @@ class LinstorVolumeManager(object): + + if not resources: + if activate: +- self._activate_device_path(node_name, volume_name) ++ self._mark_resource_cache_as_dirty() ++ self._activate_device_path( ++ self._linstor, node_name, volume_name ++ ) + return self._request_device_path(volume_uuid, volume_name) + raise LinstorVolumeManagerError( + 'Empty dev path for `{}`, but definition "seems" to exist' +@@ -1585,25 +1775,6 @@ class LinstorVolumeManager(object): + # Contains a path of the /dev/drbd form. + return resources[0].volumes[0].device_path + +- def _activate_device_path(self, node_name, volume_name): +- self._mark_resource_cache_as_dirty() +- result = self._linstor.resource_create([ +- linstor.ResourceData(node_name, volume_name, diskless=True) +- ]) +- if linstor.Linstor.all_api_responses_no_error(result): +- return +- errors = linstor.Linstor.filter_api_call_response_errors(result) +- if len(errors) == 1 and errors[0].is_error( +- linstor.consts.FAIL_EXISTS_RSC +- ): +- return +- +- raise LinstorVolumeManagerError( +- 'Unable to activate device path of `{}` on node `{}`: {}' +- .format(volume_name, node_name, ', '.join( +- [str(x) for x in result])) +- ) +- + def _destroy_resource(self, resource_name): + self._mark_resource_cache_as_dirty() + result = self._linstor.resource_dfn_delete(resource_name) +@@ -1757,7 +1928,7 @@ class LinstorVolumeManager(object): + def _create_linstor_kv(self, namespace): + return linstor.KV( + self._get_store_name(), +- uri=self._uri, ++ uri=self._linstor.controller_host(), + namespace=namespace + ) + +@@ -1787,46 +1958,347 @@ class LinstorVolumeManager(object): + ]) + + @classmethod +- def _create_linstor_instance(cls, uri): +- def connect(): ++ def _create_linstor_instance(cls, uri, keep_uri_unmodified=False): ++ retry = False ++ ++ def connect(uri): ++ if not uri: ++ uri = get_controller_uri() ++ if not uri: ++ raise LinstorVolumeManagerError( ++ 'Unable to find controller uri...' ++ ) + instance = linstor.Linstor(uri, keep_alive=True) + instance.connect() + return instance + ++ try: ++ return connect(uri) ++ except (linstor.errors.LinstorNetworkError, LinstorVolumeManagerError): ++ pass ++ ++ if not keep_uri_unmodified: ++ uri = None ++ + return util.retry( +- connect, +- maxretry=60, +- exceptions=[linstor.errors.LinstorNetworkError] ++ lambda: connect(uri), ++ maxretry=10, ++ exceptions=[ ++ linstor.errors.LinstorNetworkError, ++ LinstorVolumeManagerError ++ ] + ) + + @classmethod +- def _destroy_storage_pool(cls, lin, group_name, node_name): +- result = lin.storage_pool_delete(node_name, group_name) ++ def _activate_device_path(cls, lin, node_name, volume_name): ++ result = lin.resource_create([ ++ linstor.ResourceData(node_name, volume_name, diskless=True) ++ ]) ++ if linstor.Linstor.all_api_responses_no_error(result): ++ return ++ errors = linstor.Linstor.filter_api_call_response_errors(result) ++ if len(errors) == 1 and errors[0].is_error( ++ linstor.consts.FAIL_EXISTS_RSC ++ ): ++ return ++ ++ raise LinstorVolumeManagerError( ++ 'Unable to activate device path of `{}` on node `{}`: {}' ++ .format(volume_name, node_name, ', '.join( ++ [str(x) for x in result])) ++ ) ++ ++ @classmethod ++ def _request_database_path(cls, lin, activate=False): ++ node_name = socket.gethostname() ++ ++ try: ++ resources = filter( ++ lambda resource: resource.node_name == node_name and ++ resource.name == DATABASE_VOLUME_NAME, ++ lin.resource_list_raise().resources ++ ) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Unable to get resources during database creation: {}' ++ .format(e) ++ ) ++ ++ if not resources: ++ if activate: ++ cls._activate_device_path( ++ lin, node_name, DATABASE_VOLUME_NAME ++ ) ++ return cls._request_database_path( ++ DATABASE_VOLUME_NAME, DATABASE_VOLUME_NAME ++ ) ++ raise LinstorVolumeManagerError( ++ 'Empty dev path for `{}`, but definition "seems" to exist' ++ .format(DATABASE_PATH) ++ ) ++ # Contains a path of the /dev/drbd form. ++ return resources[0].volumes[0].device_path ++ ++ @classmethod ++ def _create_database_volume(cls, lin, group_name): ++ try: ++ dfns = lin.resource_dfn_list_raise().resource_definitions ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Unable to get definitions during database creation: {}' ++ .format(e) ++ ) ++ ++ if dfns: ++ raise LinstorVolumeManagerError( ++ 'Could not create volume `{}` from SR `{}`, '.format( ++ DATABASE_VOLUME_NAME, group_name ++ ) + 'LINSTOR volume list must be empty.' ++ ) ++ ++ size = cls.round_up_volume_size(DATABASE_SIZE) ++ cls._check_volume_creation_errors(lin.resource_group_spawn( ++ rsc_grp_name=group_name, ++ rsc_dfn_name=DATABASE_VOLUME_NAME, ++ vlm_sizes=['{}B'.format(size)], ++ definitions_only=False ++ ), DATABASE_VOLUME_NAME, group_name) ++ ++ # We must modify the quorum. Otherwise we can't use correctly the ++ # minidrbdcluster daemon. ++ result = lin.resource_dfn_modify(DATABASE_VOLUME_NAME, { ++ 'DrbdOptions/auto-quorum': 'disabled', ++ 'DrbdOptions/Resource/quorum': 'majority' ++ }) + error_str = cls._get_error_str(result) + if error_str: + raise LinstorVolumeManagerError( +- 'Failed to destroy SP `{}` on node `{}`: {}'.format( +- group_name, +- node_name, +- error_str ++ 'Could not activate quorum on database volume: {}' ++ .format(error_str) ++ ) ++ ++ current_device_path = cls._request_database_path(lin, activate=True) ++ ++ # We use realpath here to get the /dev/drbd path instead of ++ # /dev/drbd/by-res/. ++ expected_device_path = cls.build_device_path(DATABASE_VOLUME_NAME) ++ util.wait_for_path(expected_device_path, 5) ++ ++ device_realpath = os.path.realpath(expected_device_path) ++ if current_device_path != device_realpath: ++ raise LinstorVolumeManagerError( ++ 'Invalid path, current={}, expected={} (realpath={})' ++ .format( ++ current_device_path, ++ expected_device_path, ++ device_realpath + ) + ) + ++ try: ++ util.pread2([DATABASE_MKFS, expected_device_path]) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to execute {} on database volume: {}' ++ .format(DATABASE_MKFS, e) ++ ) ++ ++ return expected_device_path ++ + @classmethod +- def _destroy_resource_group(cls, lin, group_name): +- result = lin.resource_group_delete(group_name) +- error_str = cls._get_error_str(result) ++ def _destroy_database_volume(cls, lin, group_name): ++ error_str = cls._get_error_str( ++ lin.resource_dfn_delete(DATABASE_VOLUME_NAME) ++ ) + if error_str: + raise LinstorVolumeManagerError( +- 'Failed to destroy RG `{}`: {}'.format(group_name, error_str) ++ 'Could not destroy resource `{}` from SR `{}`: {}' ++ .format(DATABASE_VOLUME_NAME, group_name, error_str) + ) + ++ @classmethod ++ def _mount_database_volume(cls, volume_path, mount=True, force=False): ++ backup_path = DATABASE_PATH + '-' + str(uuid.uuid4()) ++ ++ try: ++ # 1. Create a backup config folder. ++ database_not_empty = bool(os.listdir(DATABASE_PATH)) ++ if database_not_empty: ++ try: ++ os.mkdir(backup_path) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to create backup path {} of LINSTOR config: {}' ++ .format(backup_path, e) ++ ) ++ ++ # 2. Move the config in the mounted volume. ++ if database_not_empty: ++ cls._move_files(DATABASE_PATH, backup_path) ++ ++ cls._mount_volume(volume_path, DATABASE_PATH, mount) ++ ++ if database_not_empty: ++ cls._move_files(backup_path, DATABASE_PATH, force) ++ ++ # 3. Remove useless backup directory. ++ try: ++ os.rmdir(backup_path) ++ except Exception: ++ raise LinstorVolumeManagerError( ++ 'Failed to remove backup path {} of LINSTOR config {}' ++ .format(backup_path, e) ++ ) ++ except Exception as e: ++ def force_exec(fn): ++ try: ++ fn() ++ except Exception: ++ pass ++ ++ if mount == cls._is_mounted(DATABASE_PATH): ++ force_exec(lambda: cls._move_files( ++ DATABASE_PATH, backup_path ++ )) ++ force_exec(lambda: cls._mount_volume( ++ volume_path, DATABASE_PATH, not mount ++ )) ++ ++ if mount != cls._is_mounted(DATABASE_PATH): ++ force_exec(lambda: cls._move_files( ++ backup_path, DATABASE_PATH ++ )) ++ ++ force_exec(lambda: os.rmdir(backup_path)) ++ raise e ++ ++ @classmethod ++ def _force_destroy_database_volume(cls, lin, group_name): ++ try: ++ cls._destroy_database_volume(lin, group_name) ++ except Exception: ++ pass ++ ++ @classmethod ++ def _destroy_storage_pool(cls, lin, group_name, node_name): ++ def destroy(): ++ result = lin.storage_pool_delete(node_name, group_name) ++ errors = cls._filter_errors(result) ++ if cls._check_errors(errors, [ ++ linstor.consts.FAIL_NOT_FOUND_STOR_POOL, ++ linstor.consts.FAIL_NOT_FOUND_STOR_POOL_DFN ++ ]): ++ return ++ ++ if errors: ++ raise LinstorVolumeManagerError( ++ 'Failed to destroy SP `{}` on node `{}`: {}'.format( ++ group_name, ++ node_name, ++ cls._get_error_str(errors) ++ ) ++ ) ++ ++ # We must retry to avoid errors like: ++ # "can not be deleted as volumes / snapshot-volumes are still using it" ++ # after LINSTOR database volume destruction. ++ return util.retry(destroy, maxretry=10) ++ ++ @classmethod ++ def _destroy_resource_group(cls, lin, group_name): ++ def destroy(): ++ result = lin.resource_group_delete(group_name) ++ errors = cls._filter_errors(result) ++ if cls._check_errors(errors, [ ++ linstor.consts.FAIL_NOT_FOUND_RSC_GRP ++ ]): ++ return ++ ++ if errors: ++ raise LinstorVolumeManagerError( ++ 'Failed to destroy RG `{}`: {}' ++ .format(group_name, cls._get_error_str(errors)) ++ ) ++ ++ return util.retry(destroy, maxretry=10) ++ + @classmethod + def _build_group_name(cls, base_name): + # If thin provisioning is used we have a path like this: + # `VG/LV`. "/" is not accepted by LINSTOR. + return '{}{}'.format(cls.PREFIX_SR, base_name.replace('/', '_')) + ++ @classmethod ++ def _check_volume_creation_errors(cls, result, volume_uuid, group_name): ++ errors = cls._filter_errors(result) ++ if cls._check_errors(errors, [ ++ linstor.consts.FAIL_EXISTS_RSC, linstor.consts.FAIL_EXISTS_RSC_DFN ++ ]): ++ raise LinstorVolumeManagerError( ++ 'Failed to create volume `{}` from SR `{}`, it already exists' ++ .format(volume_uuid, group_name), ++ LinstorVolumeManagerError.ERR_VOLUME_EXISTS ++ ) ++ ++ if errors: ++ raise LinstorVolumeManagerError( ++ 'Failed to create volume `{}` from SR `{}`: {}'.format( ++ volume_uuid, ++ group_name, ++ cls._get_error_str(errors) ++ ) ++ ) ++ ++ @classmethod ++ def _move_files(cls, src_dir, dest_dir, force=False): ++ def listdir(dir): ++ ignored = ['lost+found'] ++ return filter(lambda file: file not in ignored, os.listdir(dir)) ++ ++ try: ++ if not force: ++ files = listdir(dest_dir) ++ if files: ++ raise LinstorVolumeManagerError( ++ 'Cannot move files from {} to {} because destination ' ++ 'contains: {}'.format(src_dir, dest_dir, files) ++ ) ++ except LinstorVolumeManagerError: ++ raise ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Cannot list dir {}: {}'.format(dest_dir, e) ++ ) ++ ++ try: ++ for file in listdir(src_dir): ++ try: ++ dest_file = os.path.join(dest_dir, file) ++ if not force and os.path.exists(dest_file): ++ raise LinstorVolumeManagerError( ++ 'Cannot move {} because it already exists in the ' ++ 'destination'.format(file) ++ ) ++ shutil.move(os.path.join(src_dir, file), dest_file) ++ except LinstorVolumeManagerError: ++ raise ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Cannot move {}: {}'.format(file, e) ++ ) ++ except Exception as e: ++ if not force: ++ try: ++ cls._move_files(dest_dir, src_dir, force=True) ++ except Exception: ++ pass ++ ++ raise LinstorVolumeManagerError( ++ 'Failed to move files from {} to {}: {}'.format( ++ src_dir, dest_dir, e ++ ) ++ ) ++ + @staticmethod + def _get_filtered_properties(properties): + return dict(properties.items()) +@@ -1845,3 +2317,44 @@ class LinstorVolumeManager(object): + if err.is_error(code): + return True + return False ++ ++ @classmethod ++ def _start_controller(cls, start=True): ++ return cls._start_service('linstor-controller', start) ++ ++ @staticmethod ++ def _start_service(name, start=True): ++ action = 'start' if start else 'stop' ++ (ret, out, err) = util.doexec([ ++ 'systemctl', action, name ++ ]) ++ if ret != 0: ++ raise LinstorVolumeManagerError( ++ 'Failed to {} {}: {} {}' ++ .format(action, name, out, err) ++ ) ++ ++ @staticmethod ++ def _is_mounted(mountpoint): ++ (ret, out, err) = util.doexec(['mountpoint', '-q', mountpoint]) ++ return ret == 0 ++ ++ @classmethod ++ def _mount_volume(cls, volume_path, mountpoint, mount=True): ++ if mount: ++ try: ++ util.pread(['mount', volume_path, mountpoint]) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to mount volume {} on {}: {}' ++ .format(volume_path, mountpoint, e) ++ ) ++ else: ++ try: ++ if cls._is_mounted(mountpoint): ++ util.pread(['umount', mountpoint]) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to umount volume {} on {}: {}' ++ .format(volume_path, mountpoint, e) ++ ) +diff --git a/drivers/tapdisk-pause b/drivers/tapdisk-pause +index ed6abede..e0bca7be 100755 +--- a/drivers/tapdisk-pause ++++ b/drivers/tapdisk-pause +@@ -30,7 +30,7 @@ import vhdutil + import lvmcache + + try: +- from linstorvolumemanager import LinstorVolumeManager ++ from linstorvolumemanager import get_controller_uri, LinstorVolumeManager + LINSTOR_AVAILABLE = True + except ImportError: + LINSTOR_AVAILABLE = False +@@ -152,10 +152,6 @@ class Tapdisk: + # "B" path. Note: "A", "B" and "OLD_A" are UUIDs. + session = self.session + +- linstor_uri = 'linstor://{}'.format( +- util.get_master_rec(session)['address'] +- ) +- + host_ref = util.get_this_host_ref(session) + sr_ref = session.xenapi.SR.get_by_uuid(self.sr_uuid) + +@@ -167,7 +163,7 @@ class Tapdisk: + group_name = dconf['group-name'] + + device_path = LinstorVolumeManager( +- linstor_uri, ++ get_controller_uri(), + group_name, + logger=util.SMlog + ).get_device_path(self.vdi_uuid) +diff --git a/drivers/util.py b/drivers/util.py +index 54fda469..7151f368 100755 +--- a/drivers/util.py ++++ b/drivers/util.py +@@ -659,31 +659,10 @@ def get_master_ref(session): + return session.xenapi.pool.get_master(pools[0]) + + +-def get_master_rec(session): +- return session.xenapi.host.get_record(get_master_ref(session)) +- +- + def is_master(session): + return get_this_host_ref(session) == get_master_ref(session) + + +-def get_master_address(): +- address = None +- try: +- fd = open('/etc/xensource/pool.conf', 'r') +- try: +- items = fd.readline().split(':') +- if items[0].strip() == 'master': +- address = 'localhost' +- else: +- address = items[1].strip() +- finally: +- fd.close() +- except Exception: +- pass +- return address +- +- + # XXX: this function doesn't do what it claims to do + def get_localhost_uuid(session): + filename = '/etc/xensource-inventory' +diff --git a/etc/minidrbdcluster.ini b/etc/minidrbdcluster.ini +new file mode 100644 +index 00000000..0126e862 +--- /dev/null ++++ b/etc/minidrbdcluster.ini +@@ -0,0 +1,14 @@ ++# minidrbdcluster keeps a service running on one of the nodes. ++# Quorum must be enabled in the DRBD resource! ++# ++# The section names are the names of DRBD resources. Within a ++# section name the systemd-units to activate on one of the nodes. ++ ++[xcp-persistent-database] ++systemd-units=var-lib-linstor.mount,linstor-controller.service ++ ++[xcp-persistent-ha-statefile] ++systemd-units= ++ ++[xcp-persistent-redo-log] ++systemd-units= +diff --git a/etc/systemd/system/linstor-satellite.service.d/override.conf b/etc/systemd/system/linstor-satellite.service.d/override.conf +new file mode 100644 +index 00000000..b1686b4f +--- /dev/null ++++ b/etc/systemd/system/linstor-satellite.service.d/override.conf +@@ -0,0 +1,5 @@ ++[Service] ++Environment=LS_KEEP_RES=^xcp-persistent* ++ ++[Unit] ++After=drbd.service +diff --git a/etc/systemd/system/var-lib-linstor.mount b/etc/systemd/system/var-lib-linstor.mount +new file mode 100644 +index 00000000..a05a7f74 +--- /dev/null ++++ b/etc/systemd/system/var-lib-linstor.mount +@@ -0,0 +1,6 @@ ++[Unit] ++Description=Filesystem for the LINSTOR controller ++ ++[Mount] ++What=/dev/drbd/by-res/xcp-persistent-database/0 ++Where=/var/lib/linstor +diff --git a/linstor/linstor-monitord.c b/linstor/linstor-monitord.c +index a1592fda..47740598 100644 +--- a/linstor/linstor-monitord.c ++++ b/linstor/linstor-monitord.c +@@ -287,18 +287,6 @@ static inline int addInotifyWatch (int inotifyFd, const char *filepath, uint32_t + + // ----------------------------------------------------------------------------- + +-static inline int updateLinstorController (int isMaster) { +- syslog(LOG_INFO, "%s linstor-controller...", isMaster ? "Enabling" : "Disabling"); +- char *argv[] = { +- "systemctl", +- isMaster ? "enable" : "disable", +- "--now", +- "linstor-controller", +- NULL +- }; +- return execCommand(argv, NULL); +-} +- + static inline int updateLinstorNode (State *state) { + char buffer[256]; + if (gethostname(buffer, sizeof buffer) == -1) { +@@ -416,7 +404,6 @@ static inline int processPoolConfEvents (State *state, int wd, char **buffer, si + inotify_rm_watch(state->inotifyFd, wd); // Do not forget to remove watch to avoid leaks. + return -EIO; + } +- ret = updateLinstorController(state->isMaster); + } else { + if (mask & (IN_CREATE | IN_MOVED_TO)) { + syslog(LOG_ERR, "Watched `" POOL_CONF_ABS_FILE "` file has been recreated!"); +@@ -495,8 +482,6 @@ static inline int waitForPoolConfCreation (State *state, int *wdFile) { + // Update LINSTOR services... + int ret; + state->isMaster = isMasterHost(&ret); +- if (!ret) +- ret = updateLinstorController(state->isMaster); + + // Ok we can't read the pool configuration file. + // Maybe the file doesn't exist. Waiting its creation... +diff --git a/scripts/minidrbdcluster b/scripts/minidrbdcluster +new file mode 100755 +index 00000000..a04b6c1c +--- /dev/null ++++ b/scripts/minidrbdcluster +@@ -0,0 +1,171 @@ ++#! /usr/bin/env python2 ++ ++import configparser ++import os ++import re ++import signal ++import subprocess ++ ++DRBDADM_OPEN_FAILED_RE = re.compile( ++ 'open\\((.*)\\) failed: No such file or directory' ++) ++MAY_PROMOT_RE = re.compile( ++ '(?:exists|change) resource name:((?:\\w|-)+) ' ++ '(?:\\w+\\:\\w+ )*may_promote:(yes|no) promotion_score:(\\d+)' ++) ++PEER_ROLE_RE = re.compile( ++ '(?:exists|change) connection name:((?:\\w|-)+) peer-node-id:(?:\\d+) ' ++ 'conn-name:(\\w+) (?:\\w+\\:\\w+ )*role:(Primary|Secondary|Unknown)' ++) ++HAVE_QUORUM_RE = re.compile( ++ '(?:exists|change) device name:((?:\\w|-)+) ' ++ '(?:\\w+\\:\\w+ )*quorum:(yes|no)' ++) ++ ++ ++class SigHupException(Exception): ++ pass ++ ++ ++def sig_handler(sig, frame): ++ raise SigHupException( ++ 'Received signal ' + str(sig) + ++ ' on line ' + str(frame.f_lineno) + ++ ' in ' + frame.f_code.co_filename ++ ) ++ ++ ++def call_systemd(operation, service): ++ verbose = operation in ('start', 'stop') ++ if verbose: ++ print('Trying to %s %s' % (operation, service)) ++ r = os.system('systemctl %s %s' % (operation, service)) ++ if verbose: ++ print('%s for %s %s' % ( ++ 'success' if r == 0 else 'failure', operation, service ++ )) ++ return r == 0 ++ ++ ++def ensure_systemd_started(service): ++ args = ['systemctl', 'is-active', '--quiet', service] ++ ++ proc = subprocess.Popen(args) ++ proc.wait() ++ if not proc.returncode: ++ return True # Already active. ++ ++ return call_systemd('start', service) ++ ++ ++def show_status(services, status): ++ print('status:') ++ for systemd_unit in services: ++ call_systemd('status', systemd_unit) ++ for res_name in status: ++ print('%s is %s' % (res_name, status[res_name])) ++ ++ ++def clean_up(services): ++ print('exiting:') ++ for systemd_unit in reversed(services): ++ call_systemd('stop', systemd_unit) ++ ++ ++def get_systemd_units(systemd_units_str): ++ systemd_units = [] ++ for systemd_unit in systemd_units_str.split(','): ++ systemd_unit = systemd_unit.strip() ++ if systemd_unit: ++ systemd_units.append(systemd_unit) ++ return systemd_units ++ ++ ++def process(events2, resources, services, status): ++ line = events2.stdout.readline() ++ m = MAY_PROMOT_RE.match(line) ++ if m: ++ res_name, may_promote, promotion_score = m.groups() ++ if res_name in resources and may_promote == 'yes': ++ systemd_units_str = resources[res_name]['systemd-units'] ++ for systemd_unit in get_systemd_units(systemd_units_str): ++ if not ensure_systemd_started(systemd_unit): ++ break ++ if systemd_unit not in services: ++ services.append(systemd_unit) ++ m = PEER_ROLE_RE.match(line) ++ if m: ++ res_name, conn_name, role = m.groups() ++ if res_name in status: ++ status[res_name][conn_name] = role ++ m = HAVE_QUORUM_RE.match(line) ++ if m: ++ res_name, have_quorum = m.groups() ++ if res_name in resources and have_quorum == 'no': ++ systemd_units_str = resources[res_name]['systemd-units'] ++ systemd_units = get_systemd_units(systemd_units_str) ++ to_stop = [x for x in systemd_units if x in services] ++ if to_stop: ++ print('Lost quorum on %s' % (res_name)) ++ for systemd_unit in reversed(to_stop): ++ r = call_systemd('stop', systemd_unit) ++ if r: ++ services.remove(systemd_unit) ++ ++ ++def active_drbd_volume(res_name): ++ retry = True ++ args = ['drbdadm', 'adjust', res_name] ++ while True: ++ proc = subprocess.Popen(args, stderr=subprocess.PIPE) ++ (stdout, stderr) = proc.communicate() ++ if not proc.returncode: ++ return # Success. \o/ ++ ++ if not retry: ++ break ++ ++ m = DRBDADM_OPEN_FAILED_RE.match(stderr) ++ if m and subprocess.call(['lvchange', '-ay', m.groups()[0]]) == 0: ++ retry = False ++ else: ++ break ++ ++ print('Failed to execute `{}`: {}'.format(args, stderr)) ++ ++ ++def main(): ++ services = [] ++ status = dict() ++ config = configparser.ConfigParser() ++ config.read('/etc/minidrbdcluster.ini') ++ resources = config._sections ++ if not resources: ++ raise Exception( ++ 'No resources to watch, maybe /etc/minidrbdcluster.ini missing' ++ ) ++ print('Managing DRBD resources: %s' % (' '.join(resources))) ++ for res_name in resources: ++ status[res_name] = dict() ++ active_drbd_volume(res_name) ++ ++ signal.signal(signal.SIGHUP, sig_handler) ++ ++ print('Starting process...') ++ events2 = subprocess.Popen( ++ ['drbdsetup', 'events2'], stdout=subprocess.PIPE ++ ) ++ run = True ++ while run: ++ try: ++ process(events2, resources, services, status) ++ except KeyboardInterrupt: ++ run = False ++ except SigHupException: ++ show_status(services, status) ++ ++ clean_up(services) ++ ++ ++if __name__ == '__main__': ++ main() +diff --git a/systemd/minidrbdcluster.service b/systemd/minidrbdcluster.service +new file mode 100644 +index 00000000..3de6ac4f +--- /dev/null ++++ b/systemd/minidrbdcluster.service +@@ -0,0 +1,18 @@ ++[Unit] ++Description=Minimalistic high-availability cluster resource manager ++Before=xs-sm.service ++Wants=network-online.target ++After=network-online.target ++ ++[Service] ++Type=simple ++Environment=PYTHONUNBUFFERED=1 ++ExecStart=/opt/xensource/libexec/minidrbdcluster ++KillMode=process ++KillSignal=SIGINT ++StandardOutput=journal ++StandardError=journal ++SyslogIdentifier=minidrbdcluster ++ ++[Install] ++WantedBy=multi-user.target diff --git a/SOURCES/0034-feat-LinstorSR-ensure-heartbeat-and-redo_log-VDIs-ar.patch b/SOURCES/0034-feat-LinstorSR-ensure-heartbeat-and-redo_log-VDIs-ar.patch new file mode 100644 index 0000000..f195b50 --- /dev/null +++ b/SOURCES/0034-feat-LinstorSR-ensure-heartbeat-and-redo_log-VDIs-ar.patch @@ -0,0 +1,165 @@ +From 392a7fc2cefde386c048c2daf1250d3b6fc70584 Mon Sep 17 00:00:00 2001 +From: Wescoeur +Date: Wed, 24 Feb 2021 11:17:23 +0100 +Subject: [PATCH 034/177] feat(LinstorSR): ensure heartbeat and redo_log VDIs + are not diskless + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 2 +- + drivers/linstorvolumemanager.py | 97 +++++++++++++++++++++++++++++---- + 2 files changed, 86 insertions(+), 13 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 9650d712..d943d49e 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1463,7 +1463,7 @@ class LinstorVDI(VDI.VDI): + + self._linstor.create_volume( + self.uuid, volume_size, persistent=False, +- volume_name=volume_name ++ volume_name=volume_name, no_diskless=(volume_name is not None) + ) + volume_info = self._linstor.get_volume_info(self.uuid) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index a383e327..d8d64b4a 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -468,7 +468,8 @@ class LinstorVolumeManager(object): + return volume_uuid in self._volumes + + def create_volume( +- self, volume_uuid, size, persistent=True, volume_name=None ++ self, volume_uuid, size, persistent=True, volume_name=None, ++ no_diskless=False + ): + """ + Create a new volume on the SR. +@@ -478,6 +479,8 @@ class LinstorVolumeManager(object): + on the next constructor call LinstorSR(...). + :param str volume_name: If set, this name is used in the LINSTOR + database instead of a generated name. ++ :param bool no_diskless: If set, the default group redundancy is not ++ used, instead the volume is created on all nodes. + :return: The current device path of the volume. + :rtype: str + """ +@@ -486,7 +489,8 @@ class LinstorVolumeManager(object): + if not volume_name: + volume_name = self.build_volume_name(util.gen_uuid()) + volume_properties = self._create_volume_with_properties( +- volume_uuid, volume_name, size, place_resources=True ++ volume_uuid, volume_name, size, place_resources=True, ++ no_diskless=no_diskless + ) + + try: +@@ -1673,19 +1677,88 @@ class LinstorVolumeManager(object): + + return self._storage_pools + +- def _create_volume(self, volume_uuid, volume_name, size, place_resources): ++ def _create_volume( ++ self, volume_uuid, volume_name, size, place_resources, ++ no_diskless=False ++ ): + size = self.round_up_volume_size(size) +- + self._mark_resource_cache_as_dirty() +- self._check_volume_creation_errors(self._linstor.resource_group_spawn( +- rsc_grp_name=self._group_name, +- rsc_dfn_name=volume_name, +- vlm_sizes=['{}B'.format(size)], +- definitions_only=not place_resources +- ), volume_uuid, self._group_name) ++ ++ # A. Basic case when we use the default redundancy of the group. ++ if not no_diskless: ++ self._check_volume_creation_errors( ++ self._linstor.resource_group_spawn( ++ rsc_grp_name=self._group_name, ++ rsc_dfn_name=volume_name, ++ vlm_sizes=['{}B'.format(size)], ++ definitions_only=not place_resources ++ ), ++ volume_uuid, ++ self._group_name ++ ) ++ return ++ ++ # B. Complex case. ++ if not place_resources: ++ raise LinstorVolumeManagerError( ++ 'Could not create volume `{}` from SR `{}`: it\'s impossible ' ++ .format(volume_uuid, self._group_name) + ++ 'to force no diskless without placing resources' ++ ) ++ ++ # B.1. Create resource list. ++ resources = [] ++ for node_name in self._get_node_names(): ++ resources.append(linstor.ResourceData( ++ node_name=node_name, ++ rsc_name=volume_name, ++ storage_pool=self._group_name ++ )) ++ ++ # B.2. Create volume! ++ def clean(): ++ try: ++ self._destroy_volume(volume_uuid) ++ except Exception as e: ++ self._logger( ++ 'Unable to destroy volume {} after creation fail: {}' ++ .format(volume_uuid, e) ++ ) ++ ++ def create(): ++ try: ++ self._check_volume_creation_errors( ++ self._linstor.resource_group_spawn( ++ rsc_grp_name=self._group_name, ++ rsc_dfn_name=volume_name, ++ vlm_sizes=['{}B'.format(size)], ++ definitions_only=True ++ ), ++ volume_uuid, ++ self._group_name ++ ) ++ ++ result = self._linstor.resource_create(resources) ++ error_str = self._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not create volume `{}` from SR `{}`: {}'.format( ++ volume_uuid, self._group_name, error_str ++ ) ++ ) ++ except LinstorVolumeManagerError as e: ++ if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: ++ clean() ++ raise ++ except Exception: ++ clean() ++ raise ++ ++ util.retry(create, maxretry=5) + + def _create_volume_with_properties( +- self, volume_uuid, volume_name, size, place_resources ++ self, volume_uuid, volume_name, size, place_resources, ++ no_diskless=False + ): + if self.check_volume_exists(volume_uuid): + raise LinstorVolumeManagerError( +@@ -1714,7 +1787,7 @@ class LinstorVolumeManager(object): + volume_properties[self.PROP_VOLUME_NAME] = volume_name + + self._create_volume( +- volume_uuid, volume_name, size, place_resources ++ volume_uuid, volume_name, size, place_resources, no_diskless + ) + + assert volume_properties.namespace == \ diff --git a/SOURCES/0035-feat-LinstorSR-protect-sr-commands-to-avoid-forgetti.patch b/SOURCES/0035-feat-LinstorSR-protect-sr-commands-to-avoid-forgetti.patch new file mode 100644 index 0000000..c00f3a2 --- /dev/null +++ b/SOURCES/0035-feat-LinstorSR-protect-sr-commands-to-avoid-forgetti.patch @@ -0,0 +1,163 @@ +From ad4076e7333d047a54a0fb6055d9b00d9030ae28 Mon Sep 17 00:00:00 2001 +From: Wescoeur +Date: Thu, 25 Feb 2021 17:52:57 +0100 +Subject: [PATCH 035/177] feat(LinstorSR): protect sr commands to avoid + forgetting LINSTOR volumes when master satellite is down + +Steps to reproduce: + +- Ensure the linstor satellite is not running on the master host, otherwise stop it +- Then restart the controller on the right host where the LINSTOR database is mounted +- Run st_attach command => All volumes will be forgotten + +To avoid this, it's possible to restart the satellite on the master before the sr_attach command. +Also it's funny to see you can start and stop the satellite juste before the sr_attach, and the volumes will not be removed. + +Explanations: + +In theory this bug is impossible because during the sr_attach execution, an exception is thrown +(so sr_scan should not be executed) BUT there is a piece of code that is executed +in SRCommand.py when sr_attach is called: + +```python +try: + return sr.attach(sr_uuid) +finally: + if is_master: + sr.after_master_attach(sr_uuid) +``` + +The exception is not immediately forwarded because a finally block must be executed before. +And what is the implementation of after_master_attach? + +```python +def after_master_attach(self, uuid): + """Perform actions required after attaching on the pool master + Return: + None + """ + self.scan(uuid) +``` + +Oh! Of course, a scan is always executed after a attach... What's the purpose of a scan if we can't +execute correctly an attach command before? I don't know, but it's probably error-prone like this context. +When scan is called, we suppose the SR is attached and we have all VDIs loaded but it's not the case +because an exception has been thrown. + +To solve this problem we forbid the execution of the scan if the attach failed. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 47 ++++++++++++++++++++++++++++++++++---------- + 1 file changed, 37 insertions(+), 10 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index d943d49e..092f5e8e 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -256,6 +256,11 @@ class LinstorSR(SR.SR): + + MANAGER_PLUGIN = 'linstor-manager' + ++ INIT_STATUS_NOT_SET = 0 ++ INIT_STATUS_IN_PROGRESS = 1 ++ INIT_STATUS_OK = 2 ++ INIT_STATUS_FAIL = 3 ++ + # -------------------------------------------------------------------------- + # SR methods. + # -------------------------------------------------------------------------- +@@ -325,19 +330,18 @@ class LinstorSR(SR.SR): + + self._vdi_shared_time = 0 + +- self._initialized = False ++ self._init_status = self.INIT_STATUS_NOT_SET + + self._vdis_loaded = False + self._all_volume_info_cache = None + self._all_volume_metadata_cache = None + + def _locked_load(method): +- @functools.wraps(method) +- def wrap(self, *args, **kwargs): +- if self._initialized: +- return method(self, *args, **kwargs) +- self._initialized = True ++ def wrapped_method(self, *args, **kwargs): ++ self._init_status = self.INIT_STATUS_OK ++ return method(self, *args, **kwargs) + ++ def load(self, *args, **kwargs): + if not self._has_session: + if self.srcmd.cmd == 'vdi_attach_from_config': + # We must have a valid LINSTOR instance here without using +@@ -352,7 +356,7 @@ class LinstorSR(SR.SR): + self._group_name, + logger=util.SMlog + ) +- return method(self, *args, **kwargs) ++ return wrapped_method(self, *args, **kwargs) + + if not self._is_master: + if self.cmd in [ +@@ -456,11 +460,29 @@ class LinstorSR(SR.SR): + ) + util.SMlog(traceback.format_exc()) + +- return method(self, *args, **kwargs) ++ return wrapped_method(self, *args, **kwargs) ++ ++ @functools.wraps(wrapped_method) ++ def wrap(self, *args, **kwargs): ++ if self._init_status in \ ++ (self.INIT_STATUS_OK, self.INIT_STATUS_IN_PROGRESS): ++ return wrapped_method(self, *args, **kwargs) ++ if self._init_status == self.INIT_STATUS_FAIL: ++ util.SMlog( ++ 'Can\'t call method {} because initialization failed' ++ .format(method) ++ ) ++ else: ++ try: ++ self._init_status = self.INIT_STATUS_IN_PROGRESS ++ return load(self, *args, **kwargs) ++ except Exception: ++ if self._init_status != self.INIT_STATUS_OK: ++ self._init_status = self.INIT_STATUS_FAIL ++ raise + + return wrap + +- @_locked_load + def cleanup(self): + if self._vdi_shared_time: + self._shared_lock_vdi(self.srcmd.params['vdi_uuid'], locked=False) +@@ -657,6 +679,9 @@ class LinstorSR(SR.SR): + + @_locked_load + def scan(self, uuid): ++ if self._init_status == self.INIT_STATUS_FAIL: ++ return ++ + util.SMlog('LinstorSR.scan for {}'.format(self.uuid)) + if not self._linstor: + raise xs_errors.XenError( +@@ -855,7 +880,6 @@ class LinstorSR(SR.SR): + def _load_vdis(self): + if self._vdis_loaded: + return +- self._vdis_loaded = True + + assert self._is_master + +@@ -866,6 +890,9 @@ class LinstorSR(SR.SR): + self._load_vdis_ex() + self._destroy_linstor_cache() + ++ # We must mark VDIs as loaded only if the load is a success. ++ self._vdis_loaded = True ++ + self._undo_all_journal_transactions() + + def _load_vdis_ex(self): diff --git a/SOURCES/0036-fix-LinstorJournaler-ensure-uri-is-not-None-during-l.patch b/SOURCES/0036-fix-LinstorJournaler-ensure-uri-is-not-None-during-l.patch new file mode 100644 index 0000000..541b7f3 --- /dev/null +++ b/SOURCES/0036-fix-LinstorJournaler-ensure-uri-is-not-None-during-l.patch @@ -0,0 +1,54 @@ +From dba2e9c951a47c4193a779a934c5a2d415675b10 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 8 Mar 2021 13:25:28 +0100 +Subject: [PATCH 036/177] fix(LinstorJournaler): ensure uri is not None during + linstor.KV creation + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorjournaler.py | 13 ++++++++++--- + 1 file changed, 10 insertions(+), 3 deletions(-) + +diff --git a/drivers/linstorjournaler.py b/drivers/linstorjournaler.py +index 285012ca..3993f601 100755 +--- a/drivers/linstorjournaler.py ++++ b/drivers/linstorjournaler.py +@@ -16,7 +16,8 @@ + # + + +-from linstorvolumemanager import get_controller_uri, LinstorVolumeManager ++from linstorvolumemanager import \ ++ get_controller_uri, LinstorVolumeManager, LinstorVolumeManagerError + import linstor + import re + import util +@@ -145,6 +146,10 @@ class LinstorJournaler: + def connect(uri): + if not uri: + uri = get_controller_uri() ++ if not uri: ++ raise LinstorVolumeManagerError( ++ 'Unable to find controller uri...' ++ ) + return linstor.KV( + LinstorVolumeManager._build_group_name(group_name), + uri=uri, +@@ -153,13 +158,15 @@ class LinstorJournaler: + + try: + return connect(uri) +- except linstor.errors.LinstorNetworkError: ++ except (linstor.errors.LinstorNetworkError, LinstorVolumeManagerError): + pass + + return util.retry( + lambda: connect(None), + maxretry=10, +- exceptions=[linstor.errors.LinstorNetworkError] ++ exceptions=[ ++ linstor.errors.LinstorNetworkError, LinstorVolumeManagerError ++ ] + ) + + @staticmethod diff --git a/SOURCES/0037-feat-LinstorSR-add-an-option-to-disable-auto-quorum-.patch b/SOURCES/0037-feat-LinstorSR-add-an-option-to-disable-auto-quorum-.patch new file mode 100644 index 0000000..6a27038 --- /dev/null +++ b/SOURCES/0037-feat-LinstorSR-add-an-option-to-disable-auto-quorum-.patch @@ -0,0 +1,142 @@ +From 70bb114f08a6957946ab653bef674d5cbe870998 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 22 Mar 2021 17:32:26 +0100 +Subject: [PATCH 037/177] feat(LinstorSR): add an option to disable auto-quorum + on volume DB + fix doc + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 9 ++++++++- + drivers/linstorvolumemanager.py | 36 +++++++++++++++++++-------------- + 2 files changed, 29 insertions(+), 16 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 092f5e8e..9f2be58c 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -31,6 +31,7 @@ except ImportError: + from lock import Lock + import blktap2 + import cleanup ++import distutils + import errno + import functools + import scsiutil +@@ -77,7 +78,8 @@ CONFIGURATION = [ + ['group-name', 'LVM group name'], + ['hosts', 'host names to use'], + ['redundancy', 'replication count'], +- ['provisioning', '"thin" or "thick" are accepted'] ++ ['provisioning', '"thin" or "thick" are accepted (optional, defaults to thin)'], ++ ['monitor-db-quorum', 'disable controller when only one host is online (optional, defaults to true)'] + ] + + DRIVER_INFO = { +@@ -300,6 +302,10 @@ class LinstorSR(SR.SR): + else: + self._provisioning = self.PROVISIONING_DEFAULT + ++ monitor_db_quorum = self.dconf.get('monitor-db-quorum') ++ self._monitor_db_quorum = (monitor_db_quorum is None) or \ ++ distutils.util.strtobool(monitor_db_quorum) ++ + # Note: We don't have access to the session field if the + # 'vdi_attach_from_config' command is executed. + self._has_session = self.sr_ref and self.session is not None +@@ -553,6 +559,7 @@ class LinstorSR(SR.SR): + ips, + self._redundancy, + thin_provisioning=self._provisioning == 'thin', ++ auto_quorum=self._monitor_db_quorum, + logger=util.SMlog + ) + self._vhdutil = LinstorVhdUtil(self.session, self._linstor) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index d8d64b4a..27c8df5d 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1261,15 +1261,17 @@ class LinstorVolumeManager(object): + @classmethod + def create_sr( + cls, group_name, node_names, ips, redundancy, +- thin_provisioning=False, ++ thin_provisioning, auto_quorum, + logger=default_logger.__func__ + ): + """ + Create a new SR on the given nodes. + :param str group_name: The SR group_name to use. + :param list[str] node_names: String list of nodes. ++ :param set(str) ips: Node ips. + :param int redundancy: How many copy of volumes should we store? +- :param set(str) ips: Node ips ++ :param bool thin_provisioning: Use thin or thick provisioning. ++ :param bool auto_quorum: DB quorum is monitored by LINSTOR. + :param function logger: Function to log messages. + :return: A new LinstorSr instance. + :rtype: LinstorSr +@@ -1283,6 +1285,7 @@ class LinstorVolumeManager(object): + ips, + redundancy, + thin_provisioning, ++ auto_quorum, + logger + ) + finally: +@@ -1300,7 +1303,7 @@ class LinstorVolumeManager(object): + @classmethod + def _create_sr( + cls, group_name, node_names, ips, redundancy, +- thin_provisioning=False, ++ thin_provisioning, auto_quorum, + logger=default_logger.__func__ + ): + # 1. Check if SR already exists. +@@ -1406,7 +1409,9 @@ class LinstorVolumeManager(object): + # 3. Create the LINSTOR database volume and mount it. + try: + logger('Creating database volume...') +- volume_path = cls._create_database_volume(lin, group_name) ++ volume_path = cls._create_database_volume( ++ lin, group_name, auto_quorum ++ ) + except LinstorVolumeManagerError as e: + if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: + logger('Destroying database volume after creation fail...') +@@ -2113,7 +2118,7 @@ class LinstorVolumeManager(object): + return resources[0].volumes[0].device_path + + @classmethod +- def _create_database_volume(cls, lin, group_name): ++ def _create_database_volume(cls, lin, group_name, auto_quorum): + try: + dfns = lin.resource_dfn_list_raise().resource_definitions + except Exception as e: +@@ -2139,16 +2144,17 @@ class LinstorVolumeManager(object): + + # We must modify the quorum. Otherwise we can't use correctly the + # minidrbdcluster daemon. +- result = lin.resource_dfn_modify(DATABASE_VOLUME_NAME, { +- 'DrbdOptions/auto-quorum': 'disabled', +- 'DrbdOptions/Resource/quorum': 'majority' +- }) +- error_str = cls._get_error_str(result) +- if error_str: +- raise LinstorVolumeManagerError( +- 'Could not activate quorum on database volume: {}' +- .format(error_str) +- ) ++ if auto_quorum: ++ result = lin.resource_dfn_modify(DATABASE_VOLUME_NAME, { ++ 'DrbdOptions/auto-quorum': 'disabled', ++ 'DrbdOptions/Resource/quorum': 'majority' ++ }) ++ error_str = cls._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not activate quorum on database volume: {}' ++ .format(error_str) ++ ) + + current_device_path = cls._request_database_path(lin, activate=True) + diff --git a/SOURCES/0038-fix-LinstorVolumeManager-add-a-workaround-to-create-.patch b/SOURCES/0038-fix-LinstorVolumeManager-add-a-workaround-to-create-.patch new file mode 100644 index 0000000..f391aed --- /dev/null +++ b/SOURCES/0038-fix-LinstorVolumeManager-add-a-workaround-to-create-.patch @@ -0,0 +1,33 @@ +From 5144ab80e4ec0af9e7c23c9ef508390b4900d1fd Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 23 Mar 2021 14:49:39 +0100 +Subject: [PATCH 038/177] fix(LinstorVolumeManager): add a workaround to create + properly SR with thin LVM + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 11 +++++++++++ + 1 file changed, 11 insertions(+) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 27c8df5d..3aaffdf4 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2134,6 +2134,17 @@ class LinstorVolumeManager(object): + ) + 'LINSTOR volume list must be empty.' + ) + ++ # Workaround to use thin lvm. Without this line an error is returned: ++ # "Not enough available nodes" ++ # I don't understand why but this command protect against this bug. ++ try: ++ lin.storage_pool_list_raise(filter_by_stor_pools=[group_name]) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to get storage pool list before database creation: {}' ++ .format(e) ++ ) ++ + size = cls.round_up_volume_size(DATABASE_SIZE) + cls._check_volume_creation_errors(lin.resource_group_spawn( + rsc_grp_name=group_name, diff --git a/SOURCES/0039-feat-LinstorSR-add-optional-ips-parameter.patch b/SOURCES/0039-feat-LinstorSR-add-optional-ips-parameter.patch new file mode 100644 index 0000000..f295b8e --- /dev/null +++ b/SOURCES/0039-feat-LinstorSR-add-optional-ips-parameter.patch @@ -0,0 +1,120 @@ +From c888113ede59c03474468767d28df14cb0e0bf73 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 24 Mar 2021 10:06:58 +0100 +Subject: [PATCH 039/177] feat(LinstorSR): add optional ips parameter + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 30 ++++++++++++++++++++----- + drivers/linstorvolumemanager.py | 40 ++++++++++++++++++++++----------- + 2 files changed, 52 insertions(+), 18 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 9f2be58c..4b761b56 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -77,6 +77,7 @@ CAPABILITIES = [ + CONFIGURATION = [ + ['group-name', 'LVM group name'], + ['hosts', 'host names to use'], ++ ['ips', 'ips to use (optional, defaults to management networks)'], + ['redundancy', 'replication count'], + ['provisioning', '"thin" or "thick" are accepted (optional, defaults to thin)'], + ['monitor-db-quorum', 'disable controller when only one host is online (optional, defaults to true)'] +@@ -325,6 +326,10 @@ class LinstorSR(SR.SR): + self.sr_vditype = SR.DEFAULT_TAP + + self._hosts = list(set(self.dconf['hosts'].split(','))) ++ if 'ips' not in self.dconf or not self.dconf['ips']: ++ self._ips = None ++ else: ++ self._ips = self.dconf['ips'].split(',') + self._redundancy = int(self.dconf['redundancy'] or 1) + self._linstor = None # Ensure that LINSTOR attribute exists. + self._journaler = None +@@ -533,11 +538,26 @@ class LinstorSR(SR.SR): + ) + + ips = {} +- for host in online_hosts: +- record = self.session.xenapi.host.get_record(host) +- hostname = record['hostname'] +- if hostname in self._hosts: +- ips[hostname] = record['address'] ++ if not self._ips: ++ for host in online_hosts: ++ record = self.session.xenapi.host.get_record(host) ++ hostname = record['hostname'] ++ if hostname in self._hosts: ++ ips[hostname] = record['address'] ++ elif len(self._ips) != len(self._hosts): ++ raise xs_errors.XenError( ++ 'LinstorSRCreate', ++ opterr='ips must be equal to host count' ++ ) ++ else: ++ for host in online_hosts: ++ record = self.session.xenapi.host.get_record(host) ++ hostname = record['hostname'] ++ try: ++ index = self._hosts.index(hostname) ++ ips[hostname] = self._ips[index] ++ except ValueError as e: ++ pass + + if len(ips) != len(self._hosts): + raise xs_errors.XenError( +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 3aaffdf4..5c04d028 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1313,22 +1313,36 @@ class LinstorVolumeManager(object): + + for node_name in node_names: + ip = ips[node_name] +- result = lin.node_create( +- node_name, +- linstor.consts.VAL_NODE_TYPE_CMBD, +- ip +- ) +- errors = cls._filter_errors(result) +- if cls._check_errors(errors, [linstor.consts.FAIL_EXISTS_NODE]): +- continue + +- if errors: +- raise LinstorVolumeManagerError( +- 'Failed to create node `{}` with ip `{}`: {}'.format( +- node_name, ip, cls._get_error_str(errors) +- ) ++ while True: ++ # Try to create node. ++ result = lin.node_create( ++ node_name, ++ linstor.consts.VAL_NODE_TYPE_CMBD, ++ ip + ) + ++ errors = cls._filter_errors(result) ++ if cls._check_errors( ++ errors, [linstor.consts.FAIL_EXISTS_NODE] ++ ): ++ # If it already exists, remove, then recreate. ++ result = lin.node_delete(node_name) ++ error_str = cls._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Failed to remove old node `{}`: {}' ++ .format(node_name, error_str) ++ ) ++ elif not errors: ++ break # Created! ++ else: ++ raise LinstorVolumeManagerError( ++ 'Failed to create node `{}` with ip `{}`: {}'.format( ++ node_name, ip, cls._get_error_str(errors) ++ ) ++ ) ++ + driver_pool_name = group_name + group_name = cls._build_group_name(group_name) + pools = lin.storage_pool_list_raise(filter_by_stor_pools=[group_name]) diff --git a/SOURCES/0040-feat-LinstorSR-add-a-helper-log_drbd_erofs-to-trace-.patch b/SOURCES/0040-feat-LinstorSR-add-a-helper-log_drbd_erofs-to-trace-.patch new file mode 100644 index 0000000..8cb5675 --- /dev/null +++ b/SOURCES/0040-feat-LinstorSR-add-a-helper-log_drbd_erofs-to-trace-.patch @@ -0,0 +1,214 @@ +From f6ed0e05e330f28d09819bf8085d171dfee84d71 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 26 Mar 2021 16:13:20 +0100 +Subject: [PATCH 040/177] feat(LinstorSR): add a helper `log_drbd_erofs` to + trace EROFS errno code with DRBD resources + check EROFS error + +Signed-off-by: Ronan Abhamon +--- + drivers/blktap2.py | 19 +++++++- + drivers/linstor-manager | 19 +++++++- + drivers/linstorvolumemanager.py | 83 ++++++++++++++++++++++++++++++++- + drivers/vhdutil.py | 2 +- + 4 files changed, 119 insertions(+), 4 deletions(-) + +diff --git a/drivers/blktap2.py b/drivers/blktap2.py +index e9305ce9..14c564e6 100755 +--- a/drivers/blktap2.py ++++ b/drivers/blktap2.py +@@ -36,6 +36,7 @@ import json + import xs_errors + import XenAPI + import scsiutil ++from linstorvolumemanager import log_lsof_drbd + from syslog import openlog, syslog + from stat import * # S_ISBLK(), ... + import nfs +@@ -817,7 +818,23 @@ class Tapdisk(object): + TapCtl.attach(pid, minor) + + try: +- TapCtl.open(pid, minor, _type, path, options) ++ retry_open = 0 ++ while True: ++ try: ++ TapCtl.open(pid, minor, _type, path, options) ++ break ++ except TapCtl.CommandFailure as e: ++ err = ( ++ 'status' in e.info and e.info['status'] ++ ) or None ++ if err in (errno.EIO, errno.EROFS, errno.EAGAIN): ++ if retry_open < 5: ++ retry_open += 1 ++ time.sleep(1) ++ continue ++ if err == errno.EROFS: ++ log_lsof_drbd(path) ++ raise + try: + tapdisk = cls.__from_blktap(blktap) + node = '/sys/dev/block/%d:%d' % (tapdisk.major(), tapdisk.minor) +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index f82b73f2..a06ed201 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -327,6 +327,22 @@ def lock_vdi(session, args): + return str(False) + + ++def lsof_resource(session, args): ++ try: ++ drbd_path = args['drbdPath'] ++ (ret, stdout, stderr) = util.doexec(['lsof', drbd_path]) ++ if ret == 0: ++ return 'DRBD resource `{}` is open: {}'.format( ++ drbd_path, stdout ++ ) ++ return '`lsof` on DRBD resource `{}` returned {}: {}'.format( ++ drbd_path, ret, stderr ++ ) ++ except Exception as e: ++ util.SMlog('linstor-manager:lsof_drbd error: {}'.format(e)) ++ raise ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -344,5 +360,6 @@ if __name__ == '__main__': + 'getDepth': get_depth, + 'getKeyHash': get_key_hash, + 'getBlockBitmap': get_block_bitmap, +- 'lockVdi': lock_vdi ++ 'lockVdi': lock_vdi, ++ 'lsofResource': lsof_resource + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 5c04d028..0357b92d 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -16,6 +16,7 @@ + # + + ++import errno + import glob + import json + import linstor +@@ -23,6 +24,7 @@ import os.path + import re + import shutil + import socket ++import stat + import time + import util + import uuid +@@ -37,6 +39,85 @@ DATABASE_MKFS = 'mkfs.ext4' + REG_DRBDADM_PRIMARY = re.compile("([^\\s]+)\\s+role:Primary") + REG_DRBDSETUP_IP = re.compile('[^\\s]+\\s+(.*):.*$') + ++DRBD_BY_RES_PATH = '/dev/drbd/by-res/' ++ ++ ++# Check if a path is a DRBD resource and log the process name/pid ++# that opened it. ++def log_lsof_drbd(path): ++ PLUGIN = 'linstor-manager' ++ PLUGIN_CMD = 'lsofResource' ++ ++ # Ignore if it's not a symlink to DRBD resource. ++ if not path.startswith(DRBD_BY_RES_PATH): ++ return ++ ++ # Compute resource name. ++ res_name_end = path.find('/', len(DRBD_BY_RES_PATH)) ++ if res_name_end == -1: ++ return ++ res_name = path[len(DRBD_BY_RES_PATH):res_name_end] ++ ++ try: ++ # Ensure path is a DRBD. ++ drbd_path = os.path.realpath(path) ++ stats = os.stat(drbd_path) ++ if not stat.S_ISBLK(stats.st_mode) or os.major(stats.st_rdev) != 147: ++ return ++ ++ # Find where the device is open. ++ (ret, stdout, stderr) = util.doexec(['drbdadm', 'status', res_name]) ++ if ret != 0: ++ util.SMlog('Failed to execute `drbdadm status` on `{}`: {}'.format( ++ res_name, stderr ++ )) ++ return ++ ++ # Is it a local device? ++ if stdout.startswith('{} role:Primary'.format(res_name)): ++ (ret, stdout, stderr) = util.doexec(['lsof', drbd_path]) ++ if ret == 0: ++ util.SMlog( ++ 'DRBD resource `{}` is open on local host: {}' ++ .format(path, stdout) ++ ) ++ else: ++ util.SMlog( ++ '`lsof` on local DRBD resource `{}` returned {}: {}' ++ .format(path, ret, stderr) ++ ) ++ return ++ ++ # Is it a remote device? ++ res = REG_DRBDADM_PRIMARY.search(stdout) ++ if not res: ++ util.SMlog( ++ 'Cannot find where is open DRBD resource `{}`' ++ .format(path) ++ ) ++ return ++ node_name = res.groups()[0] ++ ++ session = util.get_localAPI_session() ++ hosts = session.xenapi.host.get_all_records() ++ for host_ref, host_record in hosts.items(): ++ if node_name != host_record['hostname']: ++ continue ++ ++ ret = session.xenapi.host.call_plugin( ++ host_ref, PLUGIN, PLUGIN_CMD, {'drbdPath': drbd_path}, ++ ) ++ util.SMlog('DRBD resource `{}` status on host `{}`: {}'.format( ++ path, host_ref, ret ++ )) ++ return ++ util.SMlog('Cannot find primary host of DRBD resource {}'.format(path)) ++ except Exception as e: ++ util.SMlog( ++ 'Got exception while trying to determine where DRBD resource ' + ++ '`{}` is open: {}'.format(path, e) ++ ) ++ + + # ============================================================================== + +@@ -162,7 +243,7 @@ class LinstorVolumeManager(object): + '_kv_cache_dirty', '_resource_cache_dirty', '_volume_info_cache_dirty' + ) + +- DEV_ROOT_PATH = '/dev/drbd/by-res/' ++ DEV_ROOT_PATH = DRBD_BY_RES_PATH + + # Default LVM extent size. + BLOCK_SIZE = 4 * 1024 * 1024 +diff --git a/drivers/vhdutil.py b/drivers/vhdutil.py +index 422834eb..0a8fe918 100755 +--- a/drivers/vhdutil.py ++++ b/drivers/vhdutil.py +@@ -99,7 +99,7 @@ def fullSizeVHD(virtual_size): + + def ioretry(cmd): + return util.ioretry(lambda: util.pread2(cmd), +- errlist = [errno.EIO, errno.EAGAIN]) ++ errlist = [errno.EIO, errno.EROFS, errno.EAGAIN]) + + def getVHDInfo(path, extractUuidFunction, includeParent = True): + """Get the VHD info. The parent info may optionally be omitted: vhd-util diff --git a/SOURCES/0041-fix-LinstorSR-try-to-restart-the-services-again-if-t.patch b/SOURCES/0041-fix-LinstorSR-try-to-restart-the-services-again-if-t.patch new file mode 100644 index 0000000..f3009c1 --- /dev/null +++ b/SOURCES/0041-fix-LinstorSR-try-to-restart-the-services-again-if-t.patch @@ -0,0 +1,71 @@ +From 910405472a57989f5f3b9d6015e0c11970123b44 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 28 Apr 2021 15:15:58 +0200 +Subject: [PATCH 041/177] fix(LinstorSR): try to restart the services again if + there is a failure in linstor-manager + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 39 ++++++++++++++++++++++++++++----------- + 1 file changed, 28 insertions(+), 11 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index a06ed201..dcd4bc6f 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -16,6 +16,7 @@ + + import base64 + import distutils.util ++import time + import subprocess + import sys + import XenAPIPlugin +@@ -52,20 +53,36 @@ def update_all_ports(open): + + + def enable_and_start_service(name, start): +- fn = 'enable' if start else 'disable' +- args = ('systemctl', fn, '--now', name) +- (ret, out, err) = util.doexec(args) +- if ret == 0: +- return +- raise Exception('Failed to {} {}: {} {}'.format(fn, name, out, err)) ++ attempt = 0 ++ while True: ++ attempt += 1 ++ fn = 'enable' if start else 'disable' ++ args = ('systemctl', fn, '--now', name) ++ (ret, out, err) = util.doexec(args) ++ if ret == 0: ++ return ++ elif attempt >= 3: ++ raise Exception( ++ 'Failed to {} {}: {} {}'.format(fn, name, out, err) ++ ) ++ time.sleep(1) + + + def restart_service(name): +- args = ('systemctl', 'restart', name) +- (ret, out, err) = util.doexec(args) +- if ret == 0: +- return +- raise Exception('Failed to restart {}: {} {}'.format(name, out, err)) ++ attempt = 0 ++ while True: ++ attempt += 1 ++ util.SMlog('linstor-manager:restart service {} {}...'.format(name, attempt)) ++ args = ('systemctl', 'restart', name) ++ (ret, out, err) = util.doexec(args) ++ if ret == 0: ++ return ++ elif attempt >= 3: ++ util.SMlog('linstor-manager:restart service FAILED {} {}'.format(name, attempt)) ++ raise Exception( ++ 'Failed to restart {}: {} {}'.format(name, out, err) ++ ) ++ time.sleep(1) + + + def stop_service(name): diff --git a/SOURCES/0042-fix-LinstorSR-robustify-linstor-manager-to-never-inc.patch b/SOURCES/0042-fix-LinstorSR-robustify-linstor-manager-to-never-inc.patch new file mode 100644 index 0000000..a5905fa --- /dev/null +++ b/SOURCES/0042-fix-LinstorSR-robustify-linstor-manager-to-never-inc.patch @@ -0,0 +1,36 @@ +From 262c9328fe625aa44416b2795047584de7611f18 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 26 Aug 2021 15:26:11 +0200 +Subject: [PATCH 042/177] fix(LinstorSR): robustify linstor-manager to never + include from plugins path + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 8 ++++++-- + 1 file changed, 6 insertions(+), 2 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index dcd4bc6f..f12747f1 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -14,14 +14,18 @@ + # You should have received a copy of the GNU General Public License + # along with this program. If not, see . + ++# We must modify default import path, we don't want to import modules ++# installed in plugins folder and instead we must import from LINSTOR driver ++# folder. ++import sys ++sys.path[0] = '/opt/xensource/sm/' ++ + import base64 + import distutils.util + import time + import subprocess +-import sys + import XenAPIPlugin + +-sys.path.append('/opt/xensource/sm/') + from linstorjournaler import LinstorJournaler + from linstorvolumemanager import get_controller_uri, LinstorVolumeManager + from lock import Lock diff --git a/SOURCES/0043-fix-LinstorSR-prevent-starting-controller-during-fai.patch b/SOURCES/0043-fix-LinstorSR-prevent-starting-controller-during-fai.patch new file mode 100644 index 0000000..fc576e8 --- /dev/null +++ b/SOURCES/0043-fix-LinstorSR-prevent-starting-controller-during-fai.patch @@ -0,0 +1,62 @@ +From 855fb10643225cae3ec8e6b89c20972eefaee607 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 26 Aug 2021 16:52:01 +0200 +Subject: [PATCH 043/177] fix(LinstorSR): prevent starting controller during + fail in linstor manager destroy method + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 17 +++++++++++++++-- + 1 file changed, 15 insertions(+), 2 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 0357b92d..e9b7c2f3 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1263,9 +1263,11 @@ class LinstorVolumeManager(object): + 'It exists remaining volumes' + ) + ++ controller_is_running = self._controller_is_running() + uri = 'linstor://localhost' + try: +- self._start_controller(start=False) ++ if controller_is_running: ++ self._start_controller(start=False) + + # 1. Umount LINSTOR database. + self._mount_database_volume( +@@ -1290,7 +1292,7 @@ class LinstorVolumeManager(object): + self._linstor, pool.name, pool.node_name + ) + except Exception as e: +- self._start_controller(start=True) ++ self._start_controller(start=controller_is_running) + raise e + + try: +@@ -2503,6 +2505,10 @@ class LinstorVolumeManager(object): + return True + return False + ++ @classmethod ++ def _controller_is_running(cls): ++ return cls._service_is_running('linstor-controller') ++ + @classmethod + def _start_controller(cls, start=True): + return cls._start_service('linstor-controller', start) +@@ -2519,6 +2525,13 @@ class LinstorVolumeManager(object): + .format(action, name, out, err) + ) + ++ @staticmethod ++ def _service_is_running(name): ++ (ret, out, err) = util.doexec([ ++ 'systemctl', 'is-active', '--quiet', name ++ ]) ++ return not ret ++ + @staticmethod + def _is_mounted(mountpoint): + (ret, out, err) = util.doexec(['mountpoint', '-q', mountpoint]) diff --git a/SOURCES/0044-feat-LinstorVolumeManager-increase-peer-slots-limit-.patch b/SOURCES/0044-feat-LinstorVolumeManager-increase-peer-slots-limit-.patch new file mode 100644 index 0000000..e8e0f8d --- /dev/null +++ b/SOURCES/0044-feat-LinstorVolumeManager-increase-peer-slots-limit-.patch @@ -0,0 +1,167 @@ +From 02c4a4a0a092f9ae8d387a4773fbf0fc5b09cb11 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 19 Oct 2021 14:48:17 +0200 +Subject: [PATCH 044/177] feat(LinstorVolumeManager): increase peer slots limit + (support 31 connections to a DRBD) + +- Also, create diskless devices when db is created + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 84 ++++++++++++++++++++++++++------- + 1 file changed, 67 insertions(+), 17 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index e9b7c2f3..553e2f50 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1507,7 +1507,7 @@ class LinstorVolumeManager(object): + try: + logger('Creating database volume...') + volume_path = cls._create_database_volume( +- lin, group_name, auto_quorum ++ lin, group_name, node_names, redundancy, auto_quorum + ) + except LinstorVolumeManagerError as e: + if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: +@@ -1786,18 +1786,32 @@ class LinstorVolumeManager(object): + size = self.round_up_volume_size(size) + self._mark_resource_cache_as_dirty() + +- # A. Basic case when we use the default redundancy of the group. +- if not no_diskless: ++ def create_definition(): + self._check_volume_creation_errors( + self._linstor.resource_group_spawn( + rsc_grp_name=self._group_name, + rsc_dfn_name=volume_name, + vlm_sizes=['{}B'.format(size)], +- definitions_only=not place_resources ++ definitions_only=True + ), + volume_uuid, + self._group_name + ) ++ self._increase_volume_peer_slots(self._linstor, volume_name) ++ ++ # A. Basic case when we use the default redundancy of the group. ++ if not no_diskless: ++ create_definition() ++ if place_resources: ++ self._check_volume_creation_errors( ++ self._linstor.resource_auto_place( ++ rsc_name=volume_name, ++ place_count=self._redundancy, ++ diskless_on_remaining=not no_diskless ++ ), ++ volume_uuid, ++ self._group_name ++ ) + return + + # B. Complex case. +@@ -1829,17 +1843,7 @@ class LinstorVolumeManager(object): + + def create(): + try: +- self._check_volume_creation_errors( +- self._linstor.resource_group_spawn( +- rsc_grp_name=self._group_name, +- rsc_dfn_name=volume_name, +- vlm_sizes=['{}B'.format(size)], +- definitions_only=True +- ), +- volume_uuid, +- self._group_name +- ) +- ++ create_definition() + result = self._linstor.resource_create(resources) + error_str = self._get_error_str(result) + if error_str: +@@ -2164,6 +2168,16 @@ class LinstorVolumeManager(object): + ] + ) + ++ @classmethod ++ def _increase_volume_peer_slots(cls, lin, volume_name): ++ result = lin.resource_dfn_modify(volume_name, {}, peer_slots=31) ++ error_str = cls._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not increase volume peer slots of {}: {}' ++ .format(volume_name, error_str) ++ ) ++ + @classmethod + def _activate_device_path(cls, lin, node_name, volume_name): + result = lin.resource_create([ +@@ -2215,7 +2229,9 @@ class LinstorVolumeManager(object): + return resources[0].volumes[0].device_path + + @classmethod +- def _create_database_volume(cls, lin, group_name, auto_quorum): ++ def _create_database_volume( ++ cls, lin, group_name, node_names, redundancy, auto_quorum ++ ): + try: + dfns = lin.resource_dfn_list_raise().resource_definitions + except Exception as e: +@@ -2242,13 +2258,40 @@ class LinstorVolumeManager(object): + .format(e) + ) + ++ # Create the database definition. + size = cls.round_up_volume_size(DATABASE_SIZE) + cls._check_volume_creation_errors(lin.resource_group_spawn( + rsc_grp_name=group_name, + rsc_dfn_name=DATABASE_VOLUME_NAME, + vlm_sizes=['{}B'.format(size)], +- definitions_only=False ++ definitions_only=True + ), DATABASE_VOLUME_NAME, group_name) ++ cls._increase_volume_peer_slots(lin, DATABASE_VOLUME_NAME) ++ ++ # Create real resources on the first nodes. ++ resources = [] ++ for node_name in node_names[:redundancy]: ++ resources.append(linstor.ResourceData( ++ node_name=node_name, ++ rsc_name=DATABASE_VOLUME_NAME, ++ storage_pool=group_name ++ )) ++ # Create diskless resources on the remaining set. ++ for node_name in node_names[redundancy:]: ++ resources.append(linstor.ResourceData( ++ node_name=node_name, ++ rsc_name=DATABASE_VOLUME_NAME, ++ diskless=True ++ )) ++ ++ result = lin.resource_create(resources) ++ error_str = cls._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not create database volume from SR `{}`: {}'.format( ++ group_name, error_str ++ ) ++ ) + + # We must modify the quorum. Otherwise we can't use correctly the + # minidrbdcluster daemon. +@@ -2264,8 +2307,15 @@ class LinstorVolumeManager(object): + .format(error_str) + ) + ++ # Create database and ensure path exists locally and ++ # on replicated devices. + current_device_path = cls._request_database_path(lin, activate=True) + ++ # Ensure diskless paths exist on other hosts. Otherwise PBDs can't be ++ # plugged. ++ for node_name in node_names: ++ cls._activate_device_path(lin, node_name, DATABASE_VOLUME_NAME) ++ + # We use realpath here to get the /dev/drbd path instead of + # /dev/drbd/by-res/. + expected_device_path = cls.build_device_path(DATABASE_VOLUME_NAME) diff --git a/SOURCES/0045-feat-LinstorVolumeManager-add-a-fallback-to-find-con.patch b/SOURCES/0045-feat-LinstorVolumeManager-add-a-fallback-to-find-con.patch new file mode 100644 index 0000000..2d57b83 --- /dev/null +++ b/SOURCES/0045-feat-LinstorVolumeManager-add-a-fallback-to-find-con.patch @@ -0,0 +1,114 @@ +From 8ae786437072d4328825c97b0786d7caa1f4ca35 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 20 Oct 2021 14:33:04 +0200 +Subject: [PATCH 045/177] feat(LinstorVolumeManager): add a fallback to find + controller uri (when len(hosts) >= 4) + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 10 +++++++- + drivers/linstorvolumemanager.py | 42 ++++++++++++++++++++++++--------- + 2 files changed, 40 insertions(+), 12 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index f12747f1..afc4bfe5 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -364,6 +364,13 @@ def lsof_resource(session, args): + raise + + ++def has_controller_running(session, args): ++ (ret, stdout, stderr) = util.doexec([ ++ 'systemctl', 'is-active', '--quiet', 'linstor-controller' ++ ]) ++ return str(ret == 0) ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -382,5 +389,6 @@ if __name__ == '__main__': + 'getKeyHash': get_key_hash, + 'getBlockBitmap': get_block_bitmap, + 'lockVdi': lock_vdi, +- 'lsofResource': lsof_resource ++ 'lsofResource': lsof_resource, ++ 'hasControllerRunning': has_controller_running + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 553e2f50..821ef420 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -16,6 +16,7 @@ + # + + ++import distutils.util + import errno + import glob + import json +@@ -41,11 +42,12 @@ REG_DRBDSETUP_IP = re.compile('[^\\s]+\\s+(.*):.*$') + + DRBD_BY_RES_PATH = '/dev/drbd/by-res/' + ++PLUGIN = 'linstor-manager' ++ + + # Check if a path is a DRBD resource and log the process name/pid + # that opened it. + def log_lsof_drbd(path): +- PLUGIN = 'linstor-manager' + PLUGIN_CMD = 'lsofResource' + + # Ignore if it's not a symlink to DRBD resource. +@@ -159,21 +161,39 @@ def get_remote_host_ip(node_name): + + + def _get_controller_uri(): ++ PLUGIN_CMD = 'hasControllerRunning' ++ ++ # Try to find controller using drbdadm. + (ret, stdout, stderr) = util.doexec([ + 'drbdadm', 'status', DATABASE_VOLUME_NAME + ]) +- if ret != 0: +- return ++ if ret == 0: ++ # If we are here, the database device exists locally. + +- if stdout.startswith('{} role:Primary'.format(DATABASE_VOLUME_NAME)): +- return 'linstor://localhost' ++ if stdout.startswith('{} role:Primary'.format(DATABASE_VOLUME_NAME)): ++ # Nice case, we have the controller running on this local host. ++ return 'linstor://localhost' + +- res = REG_DRBDADM_PRIMARY.search(stdout) +- if res: +- node_name = res.groups()[0] +- ip = get_remote_host_ip(node_name) +- if ip: +- return 'linstor://' + ip ++ # Try to find the host using DRBD connections. ++ res = REG_DRBDADM_PRIMARY.search(stdout) ++ if res: ++ node_name = res.groups()[0] ++ ip = get_remote_host_ip(node_name) ++ if ip: ++ return 'linstor://' + ip ++ ++ # Worst case: we use many hosts in the pool (>= 4), so we can't find the ++ # primary using drbdadm because we don't have all connections to the ++ # replicated volume. `drbdadm status xcp-persistent-database` returns ++ # 3 connections by default. ++ session = util.get_localAPI_session() ++ for host_ref, host_record in session.xenapi.host.get_all_records().items(): ++ if distutils.util.strtobool( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) ++ ): ++ return 'linstor://' + host_record['hostname'] ++ ++ # Not found, maybe we are trying to create the SR... + + + def get_controller_uri(): diff --git a/SOURCES/0046-fix-var-lib-linstor.mount-ensure-we-always-mount-dat.patch b/SOURCES/0046-fix-var-lib-linstor.mount-ensure-we-always-mount-dat.patch new file mode 100644 index 0000000..cfc8f54 --- /dev/null +++ b/SOURCES/0046-fix-var-lib-linstor.mount-ensure-we-always-mount-dat.patch @@ -0,0 +1,107 @@ +From 7731198de9a684e000a7c70b19dca1cdd330e9c5 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 21 Oct 2021 11:13:07 +0200 +Subject: [PATCH 046/177] fix(var-lib-linstor.mount): ensure we always mount + database with RW flags + +Sometimes systemd fallback to read only FS if the volume can't be mounted, we must +forbid that. It's probably a DRBD error. + +Signed-off-by: Ronan Abhamon +--- + Makefile | 2 +- + drivers/linstor-manager | 4 ++-- + etc/minidrbdcluster.ini | 2 +- + etc/systemd/system/var-lib-linstor.mount | 6 ------ + etc/systemd/system/var-lib-linstor.service | 21 +++++++++++++++++++++ + 5 files changed, 25 insertions(+), 10 deletions(-) + delete mode 100644 etc/systemd/system/var-lib-linstor.mount + create mode 100644 etc/systemd/system/var-lib-linstor.service + +diff --git a/Makefile b/Makefile +index 43dd5692..2eb6a868 100755 +--- a/Makefile ++++ b/Makefile +@@ -182,7 +182,7 @@ install: precheck + $(SM_STAGING)/$(LOGROTATE_DIR) + install -m 644 etc/systemd/system/linstor-satellite.service.d/override.conf \ + $(SM_STAGING)/$(SYSTEMD_CONF_DIR)/linstor-satellite.service.d/ +- install -m 644 etc/systemd/system/var-lib-linstor.mount \ ++ install -m 644 etc/systemd/system/var-lib-linstor.service \ + $(SM_STAGING)/$(SYSTEMD_CONF_DIR) + install -m 644 etc/minidrbdcluster.ini \ + $(SM_STAGING)/$(MINI_DRBD_CLUSTER_CONF_DIR) +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index afc4bfe5..af8d2b9e 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -187,7 +187,7 @@ def destroy(session, args): + + # When destroy is called, there are no running minidrbdcluster daemons. + # So the controllers are stopped too, we must start an instance. +- restart_service('var-lib-linstor.mount') ++ restart_service('var-lib-linstor.service') + restart_service('linstor-controller') + + linstor = LinstorVolumeManager( +@@ -199,7 +199,7 @@ def destroy(session, args): + return str(True) + except Exception as e: + stop_service('linstor-controller') +- stop_service('var-lib-linstor.mount') ++ stop_service('var-lib-linstor.service') + util.SMlog('linstor-manager:destroy error: {}'.format(e)) + return str(False) + +diff --git a/etc/minidrbdcluster.ini b/etc/minidrbdcluster.ini +index 0126e862..9e523427 100644 +--- a/etc/minidrbdcluster.ini ++++ b/etc/minidrbdcluster.ini +@@ -5,7 +5,7 @@ + # section name the systemd-units to activate on one of the nodes. + + [xcp-persistent-database] +-systemd-units=var-lib-linstor.mount,linstor-controller.service ++systemd-units=var-lib-linstor.service,linstor-controller.service + + [xcp-persistent-ha-statefile] + systemd-units= +diff --git a/etc/systemd/system/var-lib-linstor.mount b/etc/systemd/system/var-lib-linstor.mount +deleted file mode 100644 +index a05a7f74..00000000 +--- a/etc/systemd/system/var-lib-linstor.mount ++++ /dev/null +@@ -1,6 +0,0 @@ +-[Unit] +-Description=Filesystem for the LINSTOR controller +- +-[Mount] +-What=/dev/drbd/by-res/xcp-persistent-database/0 +-Where=/var/lib/linstor +diff --git a/etc/systemd/system/var-lib-linstor.service b/etc/systemd/system/var-lib-linstor.service +new file mode 100644 +index 00000000..d230d048 +--- /dev/null ++++ b/etc/systemd/system/var-lib-linstor.service +@@ -0,0 +1,21 @@ ++# Regarding the current version of systemd (v.219) used in XCP-ng, we can't use ++# the ReadWriteOnly option (to apply the -w flag, it's not the same than -o rw). ++# This file is a workaround to avoid RO. It must be replaced with the code below ++# in a mount unit. Compatible with version >= 246. ++# ++# [Unit] ++# Description=Filesystem for the LINSTOR controller ++# ++# [Mount] ++# What=/dev/drbd/by-res/xcp-persistent-database/0 ++# Where=/var/lib/linstor ++# ReadWriteOnly=true ++ ++[Unit] ++Description=Mount filesystem for the LINSTOR controller ++ ++[Service] ++Type=oneshot ++ExecStart=/bin/mount -w /dev/drbd/by-res/xcp-persistent-database/0 /var/lib/linstor ++ExecStop=/bin/umount /var/lib/linstor ++RemainAfterExit=true diff --git a/SOURCES/0047-feat-LinstorVolumeManager-add-a-fallback-to-find-nod.patch b/SOURCES/0047-feat-LinstorVolumeManager-add-a-fallback-to-find-nod.patch new file mode 100644 index 0000000..c3ad307 --- /dev/null +++ b/SOURCES/0047-feat-LinstorVolumeManager-add-a-fallback-to-find-nod.patch @@ -0,0 +1,49 @@ +From 5273224cab21953acab55e07d2fa6941bfa19bba Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 21 Oct 2021 11:51:32 +0200 +Subject: [PATCH 047/177] feat(LinstorVolumeManager): add a fallback to find + node name (when len(hosts) >= 4) + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 21 ++++++++++++++------- + 1 file changed, 14 insertions(+), 7 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 821ef420..e497afa6 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -210,19 +210,26 @@ def get_controller_uri(): + + + def get_controller_node_name(): ++ PLUGIN_CMD = 'hasControllerRunning' ++ + (ret, stdout, stderr) = util.doexec([ + 'drbdadm', 'status', DATABASE_VOLUME_NAME + ]) +- if ret != 0: +- return + +- if stdout.startswith('{} role:Primary'.format(DATABASE_VOLUME_NAME)): +- return 'localhost' ++ if ret == 0: ++ if stdout.startswith('{} role:Primary'.format(DATABASE_VOLUME_NAME)): ++ return 'localhost' + +- res = REG_DRBDADM_PRIMARY.search(stdout) +- if res: +- return res.groups()[0] ++ res = REG_DRBDADM_PRIMARY.search(stdout) ++ if res: ++ return res.groups()[0] + ++ session = util.get_localAPI_session() ++ for host_ref, host_record in session.xenapi.host.get_all_records().items(): ++ if distutils.util.strtobool( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) ++ ): ++ return host_record['hostname'] + + # ============================================================================== + diff --git a/SOURCES/0048-feat-LinstorSR-explain-on-which-host-plugins-command.patch b/SOURCES/0048-feat-LinstorSR-explain-on-which-host-plugins-command.patch new file mode 100644 index 0000000..a6945ed --- /dev/null +++ b/SOURCES/0048-feat-LinstorSR-explain-on-which-host-plugins-command.patch @@ -0,0 +1,47 @@ +From c6328ccfe37e9ad5318dc6a7149ece8705f4a8b5 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 26 Oct 2021 10:44:00 +0200 +Subject: [PATCH 048/177] feat(LinstorSR): explain on which host, plugins + commands are executed + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 24 ++++++++++++++++++------ + 1 file changed, 18 insertions(+), 6 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 4b761b56..519afb29 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -775,13 +775,25 @@ class LinstorSR(SR.SR): + # Network. + # -------------------------------------------------------------------------- + +- def _exec_manager_command(self, host, command, args, error): +- ret = self.session.xenapi.host.call_plugin( +- host, self.MANAGER_PLUGIN, command, args +- ) ++ def _exec_manager_command(self, host_ref, command, args, error): ++ host_rec = self.session.xenapi.host.get_record(host_ref) ++ host_uuid = host_rec['uuid'] ++ ++ try: ++ ret = self.session.xenapi.host.call_plugin( ++ host_ref, self.MANAGER_PLUGIN, command, args ++ ) ++ except Exception as e: ++ util.SMlog( ++ 'call-plugin on {} ({}:{} with {}) raised'.format( ++ host_uuid, self.MANAGER_PLUGIN, command, args ++ ) ++ ) ++ raise e ++ + util.SMlog( +- 'call-plugin ({}:{} with {}) returned: {}'.format( +- self.MANAGER_PLUGIN, command, args, ret ++ 'call-plugin on {} ({}:{} with {}) returned: {}'.format( ++ host_uuid, self.MANAGER_PLUGIN, command, args, ret + ) + ) + if ret == 'False': diff --git a/SOURCES/0049-fix-LinstorSR-create-diskless-path-if-necessary-duri.patch b/SOURCES/0049-fix-LinstorSR-create-diskless-path-if-necessary-duri.patch new file mode 100644 index 0000000..e612b51 --- /dev/null +++ b/SOURCES/0049-fix-LinstorSR-create-diskless-path-if-necessary-duri.patch @@ -0,0 +1,33 @@ +From 0125298399ffa92e214ed3b059d3f7cb5443e08b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 3 Nov 2021 14:59:31 +0100 +Subject: [PATCH 049/177] fix(LinstorSR): create diskless path if necessary + during VDI loading + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 11 ++++++++++- + 1 file changed, 10 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 519afb29..15b9dda3 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1899,7 +1899,16 @@ class LinstorVDI(VDI.VDI): + self.size = volume_info.virtual_size + self.parent = '' + else: +- vhd_info = self.sr._vhdutil.get_vhd_info(self.uuid) ++ try: ++ vhd_info = self.sr._vhdutil.get_vhd_info(self.uuid) ++ except util.CommandException as e: ++ if e.code != errno.ENOENT: ++ raise ++ # Path doesn't exist. Probably a diskless without local path. ++ # Force creation and retry. ++ self._linstor.get_device_path(self.uuid) ++ vhd_info = self.sr._vhdutil.get_vhd_info(self.uuid) ++ + self.hidden = vhd_info.hidden + self.size = vhd_info.sizeVirt + self.parent = vhd_info.parentUuid diff --git a/SOURCES/0050-feat-LinstorSR-use-HTTP-NBD-instead-of-DRBD-directly.patch b/SOURCES/0050-feat-LinstorSR-use-HTTP-NBD-instead-of-DRBD-directly.patch new file mode 100644 index 0000000..3980e6c --- /dev/null +++ b/SOURCES/0050-feat-LinstorSR-use-HTTP-NBD-instead-of-DRBD-directly.patch @@ -0,0 +1,760 @@ +From 7983be637bf274036cd89552891cc406bf20ea0a Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 12 May 2022 17:52:35 +0200 +Subject: [PATCH 050/177] feat(LinstorSR): use HTTP/NBD instead of DRBD + directly with heartbeat VDI + +Signed-off-by: Ronan Abhamon +--- + Makefile | 1 + + drivers/LinstorSR.py | 380 ++++++++++++++++++++++++++++---- + drivers/linstor-manager | 43 +--- + drivers/linstorvhdutil.py | 6 +- + drivers/linstorvolumemanager.py | 17 +- + drivers/util.py | 49 ++++ + scripts/fork-log-daemon | 34 +++ + 7 files changed, 438 insertions(+), 92 deletions(-) + create mode 100755 scripts/fork-log-daemon + +diff --git a/Makefile b/Makefile +index 2eb6a868..af1011a1 100755 +--- a/Makefile ++++ b/Makefile +@@ -239,6 +239,7 @@ install: precheck + install -m 755 drivers/iscsilib.py $(SM_STAGING)$(SM_DEST) + install -m 755 drivers/fcoelib.py $(SM_STAGING)$(SM_DEST) + mkdir -p $(SM_STAGING)$(LIBEXEC) ++ install -m 755 scripts/fork-log-daemon $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/local-device-change $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/check-device-sharing $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/usb_change $(SM_STAGING)$(LIBEXEC) +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 15b9dda3..5bdf6769 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -34,9 +34,13 @@ import cleanup + import distutils + import errno + import functools ++import os ++import re + import scsiutil ++import signal + import SR + import SRCommand ++import subprocess + import time + import traceback + import util +@@ -52,6 +56,8 @@ from srmetadata import \ + + HIDDEN_TAG = 'hidden' + ++FORK_LOG_DAEMON = '/opt/xensource/libexec/fork-log-daemon' ++ + # ============================================================================== + + # TODO: Supports 'VDI_INTRODUCE', 'VDI_RESET_ON_BOOT/2', 'SR_TRIM', +@@ -354,7 +360,9 @@ class LinstorSR(SR.SR): + + def load(self, *args, **kwargs): + if not self._has_session: +- if self.srcmd.cmd == 'vdi_attach_from_config': ++ if self.srcmd.cmd in ( ++ 'vdi_attach_from_config', 'vdi_detach_from_config' ++ ): + # We must have a valid LINSTOR instance here without using + # the XAPI. + controller_uri = get_controller_uri() +@@ -1444,7 +1452,7 @@ class LinstorVDI(VDI.VDI): + if ( + self.sr.srcmd.cmd == 'vdi_attach_from_config' or + self.sr.srcmd.cmd == 'vdi_detach_from_config' +- ) and self.sr.srcmd.params['vdi_uuid'] == self.uuid: ++ ): + self.vdi_type = vhdutil.VDI_TYPE_RAW + self.path = self.sr.srcmd.params['vdi_path'] + else: +@@ -1529,7 +1537,7 @@ class LinstorVDI(VDI.VDI): + + self._linstor.create_volume( + self.uuid, volume_size, persistent=False, +- volume_name=volume_name, no_diskless=(volume_name is not None) ++ volume_name=volume_name + ) + volume_info = self._linstor.get_volume_info(self.uuid) + +@@ -1631,8 +1639,9 @@ class LinstorVDI(VDI.VDI): + + def attach(self, sr_uuid, vdi_uuid): + util.SMlog('LinstorVDI.attach for {}'.format(self.uuid)) ++ attach_from_config = self.sr.srcmd.cmd == 'vdi_attach_from_config' + if ( +- self.sr.srcmd.cmd != 'vdi_attach_from_config' or ++ not attach_from_config or + self.sr.srcmd.params['vdi_uuid'] != self.uuid + ) and self.sr._journaler.has_entries(self.uuid): + raise xs_errors.XenError( +@@ -1641,50 +1650,54 @@ class LinstorVDI(VDI.VDI): + 'scan SR first to trigger auto-repair' + ) + +- writable = 'args' not in self.sr.srcmd.params or \ +- self.sr.srcmd.params['args'][0] == 'true' ++ if not attach_from_config or self.sr._is_master: ++ writable = 'args' not in self.sr.srcmd.params or \ ++ self.sr.srcmd.params['args'][0] == 'true' + +- # We need to inflate the volume if we don't have enough place +- # to mount the VHD image. I.e. the volume capacity must be greater +- # than the VHD size + bitmap size. +- need_inflate = True +- if self.vdi_type == vhdutil.VDI_TYPE_RAW or not writable or \ +- self.capacity >= compute_volume_size(self.size, self.vdi_type): +- need_inflate = False +- +- if need_inflate: +- try: +- self._prepare_thin(True) +- except Exception as e: +- raise xs_errors.XenError( +- 'VDIUnavailable', +- opterr='Failed to attach VDI during "prepare thin": {}' +- .format(e) +- ) ++ # We need to inflate the volume if we don't have enough place ++ # to mount the VHD image. I.e. the volume capacity must be greater ++ # than the VHD size + bitmap size. ++ need_inflate = True ++ if ( ++ self.vdi_type == vhdutil.VDI_TYPE_RAW or ++ not writable or ++ self.capacity >= compute_volume_size(self.size, self.vdi_type) ++ ): ++ need_inflate = False + +- if not util.pathexists(self.path): +- raise xs_errors.XenError( +- 'VDIUnavailable', opterr='Could not find: {}'.format(self.path) +- ) ++ if need_inflate: ++ try: ++ self._prepare_thin(True) ++ except Exception as e: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Failed to attach VDI during "prepare thin": {}' ++ .format(e) ++ ) + + if not hasattr(self, 'xenstore_data'): + self.xenstore_data = {} ++ self.xenstore_data['storage-type'] = LinstorSR.DRIVER_TYPE + +- # TODO: Is it useful? +- self.xenstore_data.update(scsiutil.update_XS_SCSIdata( +- self.uuid, scsiutil.gen_synthetic_page_data(self.uuid) +- )) ++ if attach_from_config and self.path.startswith('/dev/http-nbd/'): ++ return self._attach_using_http_nbd() + +- self.xenstore_data['storage-type'] = LinstorSR.DRIVER_TYPE ++ if not util.pathexists(self.path): ++ raise xs_errors.XenError( ++ 'VDIUnavailable', opterr='Could not find: {}'.format(self.path) ++ ) + + self.attached = True +- + return VDI.VDI.attach(self, self.sr.uuid, self.uuid) + + def detach(self, sr_uuid, vdi_uuid): + util.SMlog('LinstorVDI.detach for {}'.format(self.uuid)) ++ detach_from_config = self.sr.srcmd.cmd == 'vdi_detach_from_config' + self.attached = False + ++ if detach_from_config and self.path.startswith('/dev/http-nbd/'): ++ return self._detach_using_http_nbd() ++ + if self.vdi_type == vhdutil.VDI_TYPE_RAW: + return + +@@ -1816,25 +1829,40 @@ class LinstorVDI(VDI.VDI): + + util.SMlog('LinstorVDI.generate_config for {}'.format(self.uuid)) + +- if not self.path or not util.pathexists(self.path): +- available = False +- # Try to refresh symlink path... +- try: +- self.path = self._linstor.get_device_path(vdi_uuid) +- available = util.pathexists(self.path) +- except Exception: +- pass +- if not available: +- raise xs_errors.XenError('VDIUnavailable') +- + resp = {} + resp['device_config'] = self.sr.dconf + resp['sr_uuid'] = sr_uuid + resp['vdi_uuid'] = self.uuid + resp['sr_sm_config'] = self.sr.sm_config +- resp['vdi_path'] = self.path + resp['command'] = 'vdi_attach_from_config' + ++ # By default, we generate a normal config. ++ # But if the disk is persistent, we must use a HTTP/NBD ++ # server to ensure we can always write or read data. ++ # Why? DRBD is unsafe when used with more than 4 hosts: ++ # We are limited to use 1 diskless and 3 full. ++ # We can't increase this limitation, so we use a NBD/HTTP device ++ # instead. ++ volume_name = self._linstor.get_volume_name(self.uuid) ++ if volume_name not in [ ++ 'xcp-persistent-ha-statefile', 'xcp-persistent-redo-log' ++ ]: ++ if not self.path or not util.pathexists(self.path): ++ available = False ++ # Try to refresh symlink path... ++ try: ++ self.path = self._linstor.get_device_path(vdi_uuid) ++ available = util.pathexists(self.path) ++ except Exception: ++ pass ++ if not available: ++ raise xs_errors.XenError('VDIUnavailable') ++ ++ resp['vdi_path'] = self.path ++ else: ++ # Axiom: DRBD device is present on at least one host. ++ resp['vdi_path'] = '/dev/http-nbd/' + volume_name ++ + config = xmlrpclib.dumps(tuple([resp]), 'vdi_attach_from_config') + return xmlrpclib.dumps((config,), "", True) + +@@ -2314,6 +2342,268 @@ class LinstorVDI(VDI.VDI): + + return ret_vdi.get_params() + ++ @staticmethod ++ def _start_persistent_http_server(volume_name): ++ null = None ++ pid_path = None ++ http_server = None ++ ++ try: ++ null = open(os.devnull, 'w') ++ ++ if volume_name == 'xcp-persistent-ha-statefile': ++ port = '8076' ++ else: ++ port = '8077' ++ ++ arguments = [ ++ 'http-disk-server', ++ '--disk', ++ '/dev/drbd/by-res/{}/0'.format(volume_name), ++ '--port', ++ port ++ ] ++ ++ util.SMlog('Starting {} on port {}...'.format(arguments[0], port)) ++ http_server = subprocess.Popen( ++ [FORK_LOG_DAEMON] + arguments, ++ stdout=null, ++ stderr=null, ++ # Ensure we use another group id to kill this process without ++ # touch the current one. ++ preexec_fn=os.setsid ++ ) ++ ++ pid_path = '/run/http-server-{}.pid'.format(volume_name) ++ with open(pid_path, 'w') as pid_file: ++ pid_file.write(str(http_server.pid)) ++ except Exception as e: ++ if pid_path: ++ try: ++ os.remove(pid_path) ++ except Exception: ++ pass ++ ++ if http_server: ++ # Kill process and children in this case... ++ os.killpg(os.getpgid(http_server.pid), signal.SIGTERM) ++ ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Failed to start http-server: {}'.format(e) ++ ) ++ finally: ++ if null: ++ null.close() ++ ++ def _start_persistent_nbd_server(self, volume_name): ++ null = None ++ pid_path = None ++ nbd_path = None ++ nbd_server = None ++ ++ try: ++ null = open(os.devnull, 'w') ++ ++ if volume_name == 'xcp-persistent-ha-statefile': ++ port = '8076' ++ else: ++ port = '8077' ++ ++ arguments = [ ++ 'nbd-http-server', ++ '--socket-path', ++ '/run/{}.socket'.format(volume_name), ++ '--nbd-name', ++ volume_name, ++ '--urls', ++ ','.join(map( ++ lambda host: "http://" + host + ':' + port, ++ self.sr._hosts ++ )) ++ ] ++ ++ util.SMlog('Starting {} using port {}...'.format(arguments[0], port)) ++ nbd_server = subprocess.Popen( ++ [FORK_LOG_DAEMON] + arguments, ++ stdout=subprocess.PIPE, ++ stderr=subprocess.STDOUT, ++ # Ensure we use another group id to kill this process without ++ # touch the current one. ++ preexec_fn=os.setsid ++ ) ++ ++ reg_nbd_path = re.compile("^NBD `(/dev/nbd[0-9]+)` is now attached.$") ++ def get_nbd_path(): ++ while nbd_server.poll() is None: ++ line = nbd_server.stdout.readline() ++ match = reg_nbd_path.match(line) ++ if match: ++ return match.group(1) ++ # Use a timeout to never block the smapi if there is a problem. ++ try: ++ nbd_path = util.timeout_call(10, get_nbd_path) ++ if nbd_path is None: ++ raise Exception('Empty NBD path (NBD server is probably dead)') ++ except util.TimeoutException: ++ raise Exception('Unable to read NBD path') ++ ++ pid_path = '/run/nbd-server-{}.pid'.format(volume_name) ++ with open(pid_path, 'w') as pid_file: ++ pid_file.write(str(nbd_server.pid)) ++ ++ util.SMlog('Create symlink: {} -> {}'.format(self.path, nbd_path)) ++ os.symlink(nbd_path, self.path) ++ except Exception as e: ++ if pid_path: ++ try: ++ os.remove(pid_path) ++ except Exception: ++ pass ++ ++ if nbd_path: ++ try: ++ os.remove(nbd_path) ++ except Exception: ++ pass ++ ++ if nbd_server: ++ # Kill process and children in this case... ++ os.killpg(os.getpgid(nbd_server.pid), signal.SIGTERM) ++ ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Failed to start nbd-server: {}'.format(e) ++ ) ++ finally: ++ if null: ++ null.close() ++ ++ @classmethod ++ def _kill_persistent_server(self, type, volume_name, sig): ++ try: ++ path = '/run/{}-server-{}.pid'.format(type, volume_name) ++ if not os.path.exists(path): ++ return ++ ++ pid = None ++ with open(path, 'r') as pid_file: ++ try: ++ pid = int(pid_file.read()) ++ except Exception: ++ pass ++ ++ if pid is not None and util.check_pid_exists(pid): ++ util.SMlog('Kill {} server {} (pid={})'.format(type, path, pid)) ++ try: ++ os.killpg(os.getpgid(pid), sig) ++ except Exception as e: ++ util.SMlog('Failed to kill {} server: {}'.format(type, e)) ++ ++ os.remove(path) ++ except: ++ pass ++ ++ @classmethod ++ def _kill_persistent_http_server(self, volume_name, sig=signal.SIGTERM): ++ return self._kill_persistent_server('nbd', volume_name, sig) ++ ++ @classmethod ++ def _kill_persistent_nbd_server(self, volume_name, sig=signal.SIGTERM): ++ return self._kill_persistent_server('http', volume_name, sig) ++ ++ def _check_http_nbd_volume_name(self): ++ volume_name = self.path[14:] ++ if volume_name not in [ ++ 'xcp-persistent-ha-statefile', 'xcp-persistent-redo-log' ++ ]: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Unsupported path: {}'.format(self.path) ++ ) ++ return volume_name ++ ++ def _attach_using_http_nbd(self): ++ volume_name = self._check_http_nbd_volume_name() ++ ++ # Ensure there is no NBD and HTTP server running. ++ self._kill_persistent_nbd_server(volume_name) ++ self._kill_persistent_http_server(volume_name) ++ ++ # 0. Fetch drbd path. ++ must_get_device_path = True ++ if not self.sr._is_master: ++ # We are on a slave, we must try to find a diskful locally. ++ try: ++ volume_info = self._linstor.get_volume_info(self.uuid) ++ except Exception as e: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Cannot get volume info of {}: {}' ++ .format(self.uuid, e) ++ ) ++ ++ must_get_device_path = volume_info.is_diskful ++ ++ drbd_path = None ++ if must_get_device_path or self.sr._is_master: ++ # If we are master, we must ensure we have a diskless ++ # or diskful available to init HA. ++ # It also avoid this error in xensource.log ++ # (/usr/libexec/xapi/cluster-stack/xhad/ha_set_pool_state): ++ # init exited with code 8 [stdout = ''; stderr = 'SF: failed to write in State-File \x10 (fd 4208696). (sys 28)\x0A'] ++ # init returned MTC_EXIT_CAN_NOT_ACCESS_STATEFILE (State-File is inaccessible) ++ available = False ++ try: ++ drbd_path = self._linstor.get_device_path(self.uuid) ++ available = util.pathexists(drbd_path) ++ except Exception: ++ pass ++ ++ if not available: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Cannot get device path of {}'.format(self.uuid) ++ ) ++ ++ # 1. Prepare http-nbd folder. ++ try: ++ if not os.path.exists('/dev/http-nbd/'): ++ os.makedirs('/dev/http-nbd/') ++ elif os.path.islink(self.path): ++ os.remove(self.path) ++ except OSError as e: ++ if e.errno != errno.EEXIST: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Cannot prepare http-nbd: {}'.format(e) ++ ) ++ ++ # 2. Start HTTP service if we have a diskful or if we are master. ++ http_service = None ++ if drbd_path: ++ assert(drbd_path in ( ++ '/dev/drbd/by-res/xcp-persistent-ha-statefile/0', ++ '/dev/drbd/by-res/xcp-persistent-redo-log/0' ++ )) ++ self._start_persistent_http_server(volume_name) ++ ++ # 3. Start NBD server in all cases. ++ try: ++ self._start_persistent_nbd_server(volume_name) ++ except Exception as e: ++ if drbd_path: ++ self._kill_persistent_http_server(volume_name) ++ raise ++ ++ self.attached = True ++ return VDI.VDI.attach(self, self.sr.uuid, self.uuid) ++ ++ def _detach_using_http_nbd(self): ++ volume_name = self._check_http_nbd_volume_name() ++ self._kill_persistent_nbd_server(volume_name) ++ self._kill_persistent_http_server(volume_name) ++ + # ------------------------------------------------------------------------------ + + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index af8d2b9e..30230adb 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -36,7 +36,7 @@ import vhdutil + + + FIREWALL_PORT_SCRIPT = '/etc/xapi.d/plugins/firewall-port' +-LINSTOR_PORTS = [3366, 3370, 3376, 3377, '7000:8000'] ++LINSTOR_PORTS = [3366, 3370, 3376, 3377, '7000:8000', 8076, 8077] + + + def update_port(port, open): +@@ -56,39 +56,6 @@ def update_all_ports(open): + update_port(port, open) + + +-def enable_and_start_service(name, start): +- attempt = 0 +- while True: +- attempt += 1 +- fn = 'enable' if start else 'disable' +- args = ('systemctl', fn, '--now', name) +- (ret, out, err) = util.doexec(args) +- if ret == 0: +- return +- elif attempt >= 3: +- raise Exception( +- 'Failed to {} {}: {} {}'.format(fn, name, out, err) +- ) +- time.sleep(1) +- +- +-def restart_service(name): +- attempt = 0 +- while True: +- attempt += 1 +- util.SMlog('linstor-manager:restart service {} {}...'.format(name, attempt)) +- args = ('systemctl', 'restart', name) +- (ret, out, err) = util.doexec(args) +- if ret == 0: +- return +- elif attempt >= 3: +- util.SMlog('linstor-manager:restart service FAILED {} {}'.format(name, attempt)) +- raise Exception( +- 'Failed to restart {}: {} {}'.format(name, out, err) +- ) +- time.sleep(1) +- +- + def stop_service(name): + args = ('systemctl', 'stop', name) + (ret, out, err) = util.doexec(args) +@@ -98,11 +65,11 @@ def stop_service(name): + + + def update_linstor_satellite_service(start): +- enable_and_start_service('linstor-satellite', start) ++ util.enable_and_start_service('linstor-satellite', start) + + + def update_minidrbdcluster_service(start): +- enable_and_start_service('minidrbdcluster', start) ++ util.enable_and_start_service('minidrbdcluster', start) + + + def prepare_sr(session, args): +@@ -187,8 +154,8 @@ def destroy(session, args): + + # When destroy is called, there are no running minidrbdcluster daemons. + # So the controllers are stopped too, we must start an instance. +- restart_service('var-lib-linstor.service') +- restart_service('linstor-controller') ++ util.restart_service('var-lib-linstor.service') ++ util.restart_service('linstor-controller') + + linstor = LinstorVolumeManager( + 'linstor://localhost', +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index ac858371..9ba0ac3b 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -42,7 +42,7 @@ def linstorhostcall(local_method, remote_method): + # Try to read locally if the device is not in use or if the device + # is up to date and not diskless. + (node_names, in_use) = \ +- self._linstor.find_up_to_date_diskfull_nodes(vdi_uuid) ++ self._linstor.find_up_to_date_diskful_nodes(vdi_uuid) + + try: + if not in_use or socket.gethostname() in node_names: +@@ -217,7 +217,7 @@ class LinstorVhdUtil: + def _get_readonly_host(self, vdi_uuid, device_path, node_names): + """ + When vhd-util is called to fetch VDI info we must find a +- diskfull DRBD disk to read the data. It's the goal of this function. ++ diskful DRBD disk to read the data. It's the goal of this function. + Why? Because when a VHD is open in RO mode, the LVM layer is used + directly to bypass DRBD verifications (we can have only one process + that reads/writes to disk with DRBD devices). +@@ -226,7 +226,7 @@ class LinstorVhdUtil: + if not node_names: + raise xs_errors.XenError( + 'VDIUnavailable', +- opterr='Unable to find diskfull node: {} (path={})' ++ opterr='Unable to find diskful node: {} (path={})' + .format(vdi_uuid, device_path) + ) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index e497afa6..da98e0b6 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -329,18 +329,21 @@ class LinstorVolumeManager(object): + __slots__ = ( + 'name', + 'allocated_size', # Allocated size, place count is not used. +- 'virtual_size' # Total virtual available size of this volume +- # (i.e. the user size at creation). ++ 'virtual_size', # Total virtual available size of this volume ++ # (i.e. the user size at creation). ++ 'is_diskful' + ) + + def __init__(self, name): + self.name = name + self.allocated_size = 0 + self.virtual_size = 0 ++ self.is_diskful = False + + def __repr__(self): +- return 'VolumeInfo("{}", {}, {})'.format( +- self.name, self.allocated_size, self.virtual_size ++ return 'VolumeInfo("{}", {}, {}, {})'.format( ++ self.name, self.allocated_size, self.virtual_size, ++ 'diskful' if self.is_diskful else 'diskless' + ) + + # -------------------------------------------------------------------------- +@@ -1332,9 +1335,9 @@ class LinstorVolumeManager(object): + .format(e) + ) + +- def find_up_to_date_diskfull_nodes(self, volume_uuid): ++ def find_up_to_date_diskful_nodes(self, volume_uuid): + """ +- Find all nodes that contain a specific volume using diskfull disks. ++ Find all nodes that contain a specific volume using diskful disks. + The disk must be up to data to be used. + :param str volume_uuid: The volume to use. + :return: The available nodes. +@@ -1716,6 +1719,8 @@ class LinstorVolumeManager(object): + else: + current = all_volume_info[resource.name] + ++ current.is_diskful = linstor.consts.FLAG_DISKLESS not in resource.flags ++ + for volume in resource.volumes: + # We ignore diskless pools of the form "DfltDisklessStorPool". + if volume.storage_pool_name == self._group_name: +diff --git a/drivers/util.py b/drivers/util.py +index 7151f368..6a9fc1a0 100755 +--- a/drivers/util.py ++++ b/drivers/util.py +@@ -1801,3 +1801,52 @@ def sessions_less_than_targets(other_config, device_config): + else: + return False + ++ ++def enable_and_start_service(name, start): ++ attempt = 0 ++ while True: ++ attempt += 1 ++ fn = 'enable' if start else 'disable' ++ args = ('systemctl', fn, '--now', name) ++ (ret, out, err) = doexec(args) ++ if ret == 0: ++ return ++ elif attempt >= 3: ++ raise Exception( ++ 'Failed to {} {}: {} {}'.format(fn, name, out, err) ++ ) ++ time.sleep(1) ++ ++ ++def stop_service(name): ++ args = ('systemctl', 'stop', name) ++ (ret, out, err) = doexec(args) ++ if ret == 0: ++ return ++ raise Exception('Failed to stop {}: {} {}'.format(name, out, err)) ++ ++ ++def restart_service(name): ++ attempt = 0 ++ while True: ++ attempt += 1 ++ SMlog('Restarting service {} {}...'.format(name, attempt)) ++ args = ('systemctl', 'restart', name) ++ (ret, out, err) = doexec(args) ++ if ret == 0: ++ return ++ elif attempt >= 3: ++ SMlog('Restart service FAILED {} {}'.format(name, attempt)) ++ raise Exception( ++ 'Failed to restart {}: {} {}'.format(name, out, err) ++ ) ++ time.sleep(1) ++ ++ ++def check_pid_exists(pid): ++ try: ++ os.kill(pid, 0) ++ except OSError: ++ return False ++ else: ++ return True +diff --git a/scripts/fork-log-daemon b/scripts/fork-log-daemon +new file mode 100755 +index 00000000..eb0f0b0f +--- /dev/null ++++ b/scripts/fork-log-daemon +@@ -0,0 +1,34 @@ ++#!/usr/bin/env python ++ ++import select ++import subprocess ++import sys ++import syslog ++ ++def main(): ++ process = subprocess.Popen(sys.argv[1:], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) ++ write_to_stdout = True ++ ++ while process.poll() is None: ++ while True: ++ output = process.stdout.readline() ++ if not output: ++ break ++ ++ if write_to_stdout: ++ try: ++ print(output) ++ sys.stdout.flush() ++ except Exception: ++ # Probably a broken pipe. So the process reading stdout is dead. ++ write_to_stdout = False ++ syslog.syslog(output) ++ ++if __name__ == "__main__": ++ syslog.openlog(ident=sys.argv[1], facility=syslog.LOG_DAEMON) ++ try: ++ main() ++ except Exception as e: ++ syslog.syslog(sys.argv[1] + ' terminated with exception: {}'.format(e)) ++ finally: ++ syslog.syslog(sys.argv[1] + ' is now terminated!') diff --git a/SOURCES/0051-fix-LinstorSR-find-controller-when-XAPI-unreachable-.patch b/SOURCES/0051-fix-LinstorSR-find-controller-when-XAPI-unreachable-.patch new file mode 100644 index 0000000..53bd174 --- /dev/null +++ b/SOURCES/0051-fix-LinstorSR-find-controller-when-XAPI-unreachable-.patch @@ -0,0 +1,184 @@ +From 44393c71d8419d61c746fc2699c450bd08183b3b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 3 Mar 2022 15:02:17 +0100 +Subject: [PATCH 051/177] fix(LinstorSR): find controller when XAPI unreachable + (XHA) + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 60 +++++++++++++++++++++++++++++---- + drivers/linstorvolumemanager.py | 34 ++++++++++++------- + 2 files changed, 74 insertions(+), 20 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 5bdf6769..a4f7afce 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -46,6 +46,7 @@ import traceback + import util + import VDI + import vhdutil ++import xml.etree.ElementTree as xml_parser + import xmlrpclib + import xs_errors + +@@ -56,6 +57,8 @@ from srmetadata import \ + + HIDDEN_TAG = 'hidden' + ++XHA_CONFIG_PATH = '/etc/xensource/xhad.conf' ++ + FORK_LOG_DAEMON = '/opt/xensource/libexec/fork-log-daemon' + + # ============================================================================== +@@ -248,6 +251,26 @@ def deflate(linstor, vdi_uuid, vdi_path, new_size, old_size): + # TODO: Change the LINSTOR volume size using linstor.resize_volume. + + ++def get_ips_from_xha_config_file(): ++ ips = [] ++ try: ++ # Ensure there is no dirty read problem. ++ # For example if the HA is reloaded. ++ tree = util.retry( ++ lambda: xml_parser.parse(XHA_CONFIG_PATH), ++ maxretry=10, ++ period=1 ++ ) ++ ++ for node in tree.getroot()[0]: ++ if node.tag == 'host': ++ for host_node in node: ++ if host_node.tag == 'IPaddress': ++ ips.append(host_node.text) ++ except: ++ pass ++ return ips ++ + # ============================================================================== + + # Usage example: +@@ -363,18 +386,41 @@ class LinstorSR(SR.SR): + if self.srcmd.cmd in ( + 'vdi_attach_from_config', 'vdi_detach_from_config' + ): +- # We must have a valid LINSTOR instance here without using +- # the XAPI. ++ def create_linstor(uri, attempt_count=30): ++ self._linstor = LinstorVolumeManager( ++ uri, ++ self._group_name, ++ logger=util.SMlog, ++ attempt_count=attempt_count ++ ) ++ + controller_uri = get_controller_uri() ++ if controller_uri: ++ create_linstor(controller_uri) ++ else: ++ def connect(): ++ # We must have a valid LINSTOR instance here without using ++ # the XAPI. Fallback with the HA config file. ++ for ip in get_ips_from_xha_config_file(): ++ controller_uri = 'linstor://' + ip ++ try: ++ util.SMlog('Connecting from config to LINSTOR controller using: {}'.format(ip)) ++ create_linstor(controller_uri, attempt_count=0) ++ return controller_uri ++ except: ++ pass ++ ++ controller_uri = util.retry(connect, maxretry=30, period=1) ++ if not controller_uri: ++ raise xs_errors.XenError( ++ 'SRUnavailable', ++ opterr='No valid controller URI to attach/detach from config' ++ ) ++ + self._journaler = LinstorJournaler( + controller_uri, self._group_name, logger=util.SMlog + ) + +- self._linstor = LinstorVolumeManager( +- controller_uri, +- self._group_name, +- logger=util.SMlog +- ) + return wrapped_method(self, *args, **kwargs) + + if not self._is_master: +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index da98e0b6..b4ee7830 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -186,15 +186,16 @@ def _get_controller_uri(): + # primary using drbdadm because we don't have all connections to the + # replicated volume. `drbdadm status xcp-persistent-database` returns + # 3 connections by default. +- session = util.get_localAPI_session() +- for host_ref, host_record in session.xenapi.host.get_all_records().items(): +- if distutils.util.strtobool( +- session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) +- ): +- return 'linstor://' + host_record['hostname'] +- +- # Not found, maybe we are trying to create the SR... +- ++ try: ++ session = util.get_localAPI_session() ++ for host_ref, host_record in session.xenapi.host.get_all_records().items(): ++ if distutils.util.strtobool( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) ++ ): ++ return 'linstor://' + host_record['hostname'] ++ except: ++ # Not found, maybe we are trying to create the SR... ++ pass + + def get_controller_uri(): + retries = 0 +@@ -349,7 +350,8 @@ class LinstorVolumeManager(object): + # -------------------------------------------------------------------------- + + def __init__( +- self, uri, group_name, repair=False, logger=default_logger.__func__ ++ self, uri, group_name, repair=False, logger=default_logger.__func__, ++ attempt_count=30 + ): + """ + Create a new LinstorVolumeManager object. +@@ -358,9 +360,12 @@ class LinstorVolumeManager(object): + :param bool repair: If true we try to remove bad volumes due to a crash + or unexpected behavior. + :param function logger: Function to log messages. ++ :param int attempt_count: Number of attempts to join the controller. + """ + +- self._linstor = self._create_linstor_instance(uri) ++ self._linstor = self._create_linstor_instance( ++ uri, attempt_count=attempt_count ++ ) + self._base_group_name = group_name + + # Ensure group exists. +@@ -2169,7 +2174,9 @@ class LinstorVolumeManager(object): + ]) + + @classmethod +- def _create_linstor_instance(cls, uri, keep_uri_unmodified=False): ++ def _create_linstor_instance( ++ cls, uri, keep_uri_unmodified=False, attempt_count=30 ++ ): + retry = False + + def connect(uri): +@@ -2193,7 +2200,8 @@ class LinstorVolumeManager(object): + + return util.retry( + lambda: connect(uri), +- maxretry=10, ++ maxretry=attempt_count, ++ period=1, + exceptions=[ + linstor.errors.LinstorNetworkError, + LinstorVolumeManagerError diff --git a/SOURCES/0052-fix-LinstorSR-use-IPs-instead-of-hostnames-in-NBD-se.patch b/SOURCES/0052-fix-LinstorSR-use-IPs-instead-of-hostnames-in-NBD-se.patch new file mode 100644 index 0000000..9d766fc --- /dev/null +++ b/SOURCES/0052-fix-LinstorSR-use-IPs-instead-of-hostnames-in-NBD-se.patch @@ -0,0 +1,205 @@ +From 58bf1f4dbf2f7860d18a55ff313363c6fa3f1305 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 24 Mar 2022 18:13:46 +0100 +Subject: [PATCH 052/177] fix(LinstorSR): use IPs instead of hostnames in NBD + server + +Without this patch we can't use XCP-ng hosts configured with static IPS. +Only servers with DHCP enabled can access the HTTP server(s) otherwise. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 101 ++++++++++++++++++++++++++++++++++++------- + drivers/util.py | 12 +++++ + 2 files changed, 97 insertions(+), 16 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index a4f7afce..c0bfc3f1 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -251,8 +251,15 @@ def deflate(linstor, vdi_uuid, vdi_path, new_size, old_size): + # TODO: Change the LINSTOR volume size using linstor.resize_volume. + + ++IPS_XHA_CACHE = None ++ ++ + def get_ips_from_xha_config_file(): +- ips = [] ++ if IPS_XHA_CACHE: ++ return IPS_XHA_CACHE ++ ++ ips = dict() ++ host_ip = None + try: + # Ensure there is no dirty read problem. + # For example if the HA is reloaded. +@@ -261,15 +268,50 @@ def get_ips_from_xha_config_file(): + maxretry=10, + period=1 + ) +- +- for node in tree.getroot()[0]: +- if node.tag == 'host': +- for host_node in node: +- if host_node.tag == 'IPaddress': +- ips.append(host_node.text) + except: +- pass +- return ips ++ return (host_ip, ips) ++ ++ def parse_host_nodes(ips, node): ++ current_id = None ++ current_ip = None ++ ++ for sub_node in node: ++ if sub_node.tag == 'IPaddress': ++ current_ip = sub_node.text ++ elif sub_node.tag == 'HostID': ++ current_id = sub_node.text ++ else: ++ continue ++ ++ if current_id and current_ip: ++ ips[current_id] = current_ip ++ return ++ util.SMlog('Ill-formed XHA file, missing IPaddress or/and HostID') ++ ++ def parse_common_config(ips, node): ++ for sub_node in node: ++ if sub_node.tag == 'host': ++ parse_host_nodes(ips, sub_node) ++ ++ def parse_local_config(ips, node): ++ for sub_node in node: ++ if sub_node.tag == 'localhost': ++ for host_node in sub_node: ++ if host_node.tag == 'HostID': ++ return host_node.text ++ ++ for node in tree.getroot(): ++ if node.tag == 'common-config': ++ parse_common_config(ips, node) ++ elif node.tag == 'local-config': ++ host_ip = parse_local_config(ips, node) ++ else: ++ continue ++ ++ if ips and host_ip: ++ break ++ ++ return (host_ip, ips) + + # ============================================================================== + +@@ -401,7 +443,7 @@ class LinstorSR(SR.SR): + def connect(): + # We must have a valid LINSTOR instance here without using + # the XAPI. Fallback with the HA config file. +- for ip in get_ips_from_xha_config_file(): ++ for ip in get_ips_from_xha_config_file()[1].values(): + controller_uri = 'linstor://' + ip + try: + util.SMlog('Connecting from config to LINSTOR controller using: {}'.format(ip)) +@@ -2402,10 +2444,23 @@ class LinstorVDI(VDI.VDI): + else: + port = '8077' + ++ try: ++ session = util.get_localAPI_session() ++ host_ip = util.get_this_host_address(session) ++ except: ++ # Fallback using the XHA file if session not available. ++ host_ip, _ = get_ips_from_xha_config_file() ++ if not host_ip: ++ raise Exception( ++ 'Cannot start persistent HTTP server: no XAPI session, nor XHA config file' ++ ) ++ + arguments = [ + 'http-disk-server', + '--disk', + '/dev/drbd/by-res/{}/0'.format(volume_name), ++ '--ip', ++ host_ip, + '--port', + port + ] +@@ -2432,7 +2487,10 @@ class LinstorVDI(VDI.VDI): + + if http_server: + # Kill process and children in this case... +- os.killpg(os.getpgid(http_server.pid), signal.SIGTERM) ++ try: ++ os.killpg(os.getpgid(http_server.pid), signal.SIGTERM) ++ except: ++ pass + + raise xs_errors.XenError( + 'VDIUnavailable', +@@ -2456,6 +2514,17 @@ class LinstorVDI(VDI.VDI): + else: + port = '8077' + ++ try: ++ session = util.get_localAPI_session() ++ ips = util.get_host_addresses(session) ++ except Exception as e: ++ _, ips = get_ips_from_xha_config_file() ++ if not ips: ++ raise Exception( ++ 'Cannot start persistent NBD server: no XAPI session, nor XHA config file ({})'.format(e) ++ ) ++ ips = ips.values() ++ + arguments = [ + 'nbd-http-server', + '--socket-path', +@@ -2463,10 +2532,7 @@ class LinstorVDI(VDI.VDI): + '--nbd-name', + volume_name, + '--urls', +- ','.join(map( +- lambda host: "http://" + host + ':' + port, +- self.sr._hosts +- )) ++ ','.join(map(lambda ip: 'http://' + ip + ':' + port, ips)) + ] + + util.SMlog('Starting {} using port {}...'.format(arguments[0], port)) +@@ -2515,7 +2581,10 @@ class LinstorVDI(VDI.VDI): + + if nbd_server: + # Kill process and children in this case... +- os.killpg(os.getpgid(nbd_server.pid), signal.SIGTERM) ++ try: ++ os.killpg(os.getpgid(nbd_server.pid), signal.SIGTERM) ++ except: ++ pass + + raise xs_errors.XenError( + 'VDIUnavailable', +diff --git a/drivers/util.py b/drivers/util.py +index 6a9fc1a0..7c52703c 100755 +--- a/drivers/util.py ++++ b/drivers/util.py +@@ -702,6 +702,18 @@ def get_hosts_attached_on(session, vdi_uuids): + host_refs[key[len('host_'):]] = True + return host_refs.keys() + ++def get_this_host_address(session): ++ host_uuid = get_this_host() ++ host_ref = session.xenapi.host.get_by_uuid(host_uuid) ++ return session.xenapi.host.get_record(host_ref)['address'] ++ ++def get_host_addresses(session): ++ addresses = [] ++ hosts = session.xenapi.host.get_all_records() ++ for record in hosts.itervalues(): ++ addresses.append(record['address']) ++ return addresses ++ + def get_this_host_ref(session): + host_uuid = get_this_host() + host_ref = session.xenapi.host.get_by_uuid(host_uuid) diff --git a/SOURCES/0053-fix-LinstorVolumeManager-ensure-we-always-use-IPs-in.patch b/SOURCES/0053-fix-LinstorVolumeManager-ensure-we-always-use-IPs-in.patch new file mode 100644 index 0000000..a9b3e06 --- /dev/null +++ b/SOURCES/0053-fix-LinstorVolumeManager-ensure-we-always-use-IPs-in.patch @@ -0,0 +1,27 @@ +From 972954832674cdd9c4792868a2fe4771a5c7eda0 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 31 Mar 2022 11:21:19 +0200 +Subject: [PATCH 053/177] fix(LinstorVolumeManager): ensure we always use IPs + in _get_controller_uri + +Otherwise if a hostname is returned, we can't use it if the XCP-ng pool +is configurer using static IPs instead of DHCP. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index b4ee7830..2d5c63ed 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -192,7 +192,7 @@ def _get_controller_uri(): + if distutils.util.strtobool( + session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) + ): +- return 'linstor://' + host_record['hostname'] ++ return 'linstor://' + host_record['address'] + except: + # Not found, maybe we are trying to create the SR... + pass diff --git a/SOURCES/0054-feat-linstor-manager-add-methods-to-add-remove-host-.patch b/SOURCES/0054-feat-linstor-manager-add-methods-to-add-remove-host-.patch new file mode 100644 index 0000000..adf7f93 --- /dev/null +++ b/SOURCES/0054-feat-linstor-manager-add-methods-to-add-remove-host-.patch @@ -0,0 +1,422 @@ +From df1b37be2e1b33838aa843f922de1de6cec3a793 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 6 Apr 2022 17:53:02 +0200 +Subject: [PATCH 054/177] feat(linstor-manager): add methods to add remove/host + from LINSTOR SR + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 6 +- + drivers/linstor-manager | 283 +++++++++++++++++++++++++++++++- + drivers/linstorvolumemanager.py | 54 ++++++ + 3 files changed, 338 insertions(+), 5 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index c0bfc3f1..413c5501 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -401,7 +401,9 @@ class LinstorSR(SR.SR): + self._ips = None + else: + self._ips = self.dconf['ips'].split(',') +- self._redundancy = int(self.dconf['redundancy'] or 1) ++ ++ if self.cmd == 'sr_create': ++ self._redundancy = int(self.dconf['redundancy']) or 1 + self._linstor = None # Ensure that LINSTOR attribute exists. + self._journaler = None + +@@ -1004,7 +1006,7 @@ class LinstorSR(SR.SR): + # ensures the displayed physical size is reachable by the user. + self.physical_size = \ + self._linstor.min_physical_size * len(self._hosts) / \ +- self._redundancy ++ self._linstor.redundancy + + self.physical_utilisation = self._linstor.allocated_volume_size + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 30230adb..7e34ce65 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -22,8 +22,7 @@ sys.path[0] = '/opt/xensource/sm/' + + import base64 + import distutils.util +-import time +-import subprocess ++import socket + import XenAPIPlugin + + from linstorjournaler import LinstorJournaler +@@ -72,6 +71,9 @@ def update_minidrbdcluster_service(start): + util.enable_and_start_service('minidrbdcluster', start) + + ++# ------------------------------------------------------------------------------ ++ ++ + def prepare_sr(session, args): + try: + update_all_ports(open=True) +@@ -338,6 +340,279 @@ def has_controller_running(session, args): + return str(ret == 0) + + ++def add_host(session, args): ++ group_name = args['groupName'] ++ ++ # 1. Find SR and PBDs. ++ srs = dict() ++ for sr_ref, sr in session.xenapi.SR.get_all_records().items(): ++ if sr.get('type') == 'linstor': ++ srs[sr_ref] = sr ++ ++ pbds = dict() ++ for pbd_ref, pbd in session.xenapi.PBD.get_all_records().items(): ++ device_config = pbd.get('device_config') ++ if ( ++ device_config and ++ device_config.get('group-name') == group_name ++ and pbd['SR'] in srs ++ ): ++ pbds[pbd_ref] = pbd ++ ++ # 2. Ensure there is at least one PBD and all PBDs are used in ++ # the same SR. ++ if not pbds: ++ raise Exception( ++ 'Failed to find PBDs of group `{}`'.format(group_name) ++ ) ++ ++ sr_ref = None ++ for pbd in pbds.values(): ++ if not sr_ref: ++ sr_ref = pbd['SR'] ++ elif pbd['SR'] != sr_ref: ++ raise Exception( ++ 'Group `{}` is used by many SRs!'.format(group_name) ++ ) ++ ++ # 3. Ensure node doesn't exist. ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ ++ node_name = socket.gethostname() ++ has_node = linstor.has_node(node_name) ++ ++ pbd_id = 0 ++ new_pbd_ref = None ++ ++ try: ++ # 4. Enable services. ++ update_all_ports(open=True) ++ update_minidrbdcluster_service(start=True) ++ update_linstor_satellite_service(start=True) ++ ++ # 5. Try to create local node. ++ if not has_node: ++ linstor.create_node(node_name, util.get_this_host_address(session)) ++ ++ # 6. Recreate PBDs. ++ # Use the redundancy given by Linstor instead of smapi config. ++ redundancy = linstor.redundancy ++ default_device_config = None ++ this_host = util.get_this_host_ref(session) ++ create_new_pbd = True ++ ++ assert pbds ++ pbds = pbds.items() ++ for pbd_ref, pbd in pbds: ++ device_config = pbd['device_config'] ++ ++ hosts = filter( ++ lambda host: len(host.strip()), ++ device_config.get('hosts', []).split(',') ++ ) ++ hosts.append(node_name) ++ hosts = ','.join(list(set(hosts))) ++ ++ # Should be the same on all hosts. ++ provisioning = device_config['provisioning'] ++ ++ if not default_device_config: ++ default_device_config = { ++ 'group-name': group_name, ++ 'redundancy': redundancy, ++ 'hosts': hosts, ++ 'provisioning': provisioning ++ } ++ ++ if pbd['currently_attached']: ++ session.xenapi.PBD.unplug(pbd_ref) ++ session.xenapi.PBD.destroy(pbd_ref) ++ pbd_id += 1 ++ ++ host = pbd['host'] ++ if host == this_host: ++ create_new_pbd = False ++ ++ pbd_ref = session.xenapi.PBD.create({ ++ 'host': host, ++ 'SR': sr_ref, ++ 'device_config': { ++ 'group-name': group_name, ++ 'redundancy': redundancy, ++ 'hosts': hosts, ++ 'provisioning': provisioning ++ } ++ }) ++ try: ++ session.xenapi.PBD.plug(pbd_ref) ++ except Exception as e: ++ util.SMlog('Failed to replug PBD: {}'.format(e)) ++ ++ # 7. Create new PBD. ++ if create_new_pbd: ++ new_pbd_ref = session.xenapi.PBD.create({ ++ 'host': this_host, ++ 'SR': sr_ref, ++ 'device_config': default_device_config ++ }) ++ try: ++ session.xenapi.PBD.plug(new_pbd_ref) ++ except Exception as e: ++ util.SMlog('Failed to plug new PBD: {}'.format(e)) ++ ++ return str(True) ++ except Exception as e: ++ stop_services = not has_node ++ if stop_services: ++ try: ++ linstor.destroy_node(node_name) ++ except Exception: ++ pass ++ ++ for pbd_ref, pbd in pbds[:pbd_id]: ++ try: ++ session.xenapi.PBD.unplug(pbd_ref) ++ except Exception: ++ pass ++ ++ try: ++ session.xenapi.PBD.destroy(pbd_ref) ++ except Exception: ++ pass ++ ++ try: ++ device_config = pbd['device_config'] ++ session.xenapi.PBD.create({ ++ 'host': host, ++ 'SR': sr_ref, ++ 'device_config': { ++ 'group-name': group_name, ++ 'redundancy': redundancy, ++ 'hosts': device_config['hosts'], ++ 'provisioning': device_config['provisioning'] ++ } ++ }) ++ except Exception as pbd_error: ++ util.SMlog('Failed to recreate PBD: {}'.format(pbd_error)) ++ pass ++ ++ try: ++ session.xenapi.PBD.plug(pbd_ref) ++ except Exception: ++ pass ++ ++ if new_pbd_ref: ++ try: ++ session.xenapi.PBD.unplug(new_pbd_ref) ++ except Exception: ++ pass ++ ++ try: ++ session.xenapi.PBD.destroy(new_pbd_ref) ++ except Exception: ++ pass ++ ++ try: ++ # If we failed to remove the node, we don't stop services. ++ if stop_services and not linstor.has_node(node_name): ++ update_linstor_satellite_service(start=False) ++ update_minidrbdcluster_service(start=False) ++ update_all_ports(open=False) ++ except Exception: ++ pass ++ ++ raise e ++ ++ ++def remove_host(session, args): ++ group_name = args['groupName'] ++ force = args.get('force') or False ++ ++ # 1. Find SRs and PBDs. ++ srs = dict() ++ for sr_ref, sr in session.xenapi.SR.get_all_records().items(): ++ if sr.get('type') == 'linstor': ++ srs[sr_ref] = sr ++ ++ pbds = dict() ++ for pbd_ref, pbd in session.xenapi.PBD.get_all_records().items(): ++ device_config = pbd.get('device_config') ++ if ( ++ device_config and ++ device_config.get('group-name') == group_name ++ and pbd['SR'] in srs ++ ): ++ pbds[pbd_ref] = pbd ++ ++ # 2. Remove node. ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ ++ node_name = socket.gethostname() ++ if linstor.has_node(node_name): ++ linstor.destroy_node(node_name) ++ if linstor.has_node(node_name): ++ raise Exception('Failed to remove node! Unknown error.') ++ ++ redundancy = linstor.redundancy ++ this_host = util.get_this_host_ref(session) ++ ++ # 3. Update PBDs. ++ for pbd_ref, pbd in pbds.items(): ++ host = pbd['host'] ++ if host == this_host: ++ if pbd['currently_attached']: ++ session.xenapi.PBD.unplug(pbd_ref) ++ session.xenapi.PBD.destroy(pbd_ref) ++ continue ++ ++ device_config = pbd['device_config'] ++ hosts = device_config.get('hosts', []).split(',') ++ try: ++ hosts.remove(node_name) ++ except Exception as e: ++ continue ++ hosts = ','.join(list(set(hosts))) ++ ++ if pbd['currently_attached']: ++ session.xenapi.PBD.unplug(pbd_ref) ++ session.xenapi.PBD.destroy(pbd_ref) ++ ++ pbd_ref = session.xenapi.PBD.create({ ++ 'host': host, ++ 'SR': pbd['SR'], ++ 'device_config': { ++ 'group-name': group_name, ++ 'redundancy': redundancy, ++ 'hosts': hosts, ++ 'provisioning': device_config['provisioning'] ++ } ++ }) ++ ++ try: ++ session.xenapi.PBD.plug(pbd_ref) ++ except Exception as e: ++ util.SMlog('Failed to replug PBD: {}'.format(e)) ++ ++ # 3. Stop services. ++ try: ++ update_linstor_satellite_service(start=False) ++ update_minidrbdcluster_service(start=False) ++ update_all_ports(open=False) ++ except Exception as e: ++ util.SMlog('Error while stopping services: {}'.format(e)) ++ pass ++ ++ return str('True') ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -357,5 +632,7 @@ if __name__ == '__main__': + 'getBlockBitmap': get_block_bitmap, + 'lockVdi': lock_vdi, + 'lsofResource': lsof_resource, +- 'hasControllerRunning': has_controller_running ++ 'hasControllerRunning': has_controller_running, ++ 'addHost': add_host, ++ 'removeHost': remove_host + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 2d5c63ed..6c0d5aa2 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -402,6 +402,15 @@ class LinstorVolumeManager(object): + """ + return self._base_group_name + ++ @property ++ def redundancy(self): ++ """ ++ Give the used redundancy. ++ :return: The redundancy. ++ :rtype: int ++ """ ++ return self._redundancy ++ + @property + def volumes(self): + """ +@@ -1376,6 +1385,51 @@ class LinstorVolumeManager(object): + """ + self._mark_resource_cache_as_dirty() + ++ def has_node(self, node_name): ++ """ ++ Check if a node exists in the LINSTOR database. ++ :rtype: bool ++ """ ++ result = self._linstor.node_list() ++ error_str = self._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Failed to list nodes using `{}`: {}' ++ .format(node_name, error_str) ++ ) ++ return bool(result[0].node(node_name)) ++ ++ def create_node(self, node_name, ip): ++ """ ++ Create a new node in the LINSTOR database. ++ :param str node_name: Node name to use. ++ :param str ip: Host IP to communicate. ++ """ ++ result = self._linstor.node_create( ++ node_name, ++ linstor.consts.VAL_NODE_TYPE_CMBD, ++ ip ++ ) ++ errors = self._filter_errors(result) ++ if errors: ++ error_str = self._get_error_str(errors) ++ raise LinstorVolumeManagerError( ++ 'Failed to create node `{}`: {}'.format(node_name, error_str) ++ ) ++ ++ def destroy_node(self, node_name): ++ """ ++ Destroy a node in the LINSTOR database. ++ :param str node_name: Node name to remove. ++ """ ++ result = self._linstor.node_delete(node_name) ++ errors = self._filter_errors(result) ++ if errors: ++ error_str = self._get_error_str(errors) ++ raise LinstorVolumeManagerError( ++ 'Failed to destroy node `{}`: {}'.format(node_name, error_str) ++ ) ++ + @classmethod + def create_sr( + cls, group_name, node_names, ips, redundancy, diff --git a/SOURCES/0055-feat-LinstorVolumeManager-support-SR-creation-with-d.patch b/SOURCES/0055-feat-LinstorVolumeManager-support-SR-creation-with-d.patch new file mode 100644 index 0000000..bb8bca2 --- /dev/null +++ b/SOURCES/0055-feat-LinstorVolumeManager-support-SR-creation-with-d.patch @@ -0,0 +1,127 @@ +From 1cacb61d45a0e8de6d99788a86eeffb7028edd85 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 13 Apr 2022 15:56:42 +0200 +Subject: [PATCH 055/177] feat(LinstorVolumeManager): support SR creation with + diskless nodes + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 66 +++++++++++++++++++++++++++------ + 1 file changed, 55 insertions(+), 11 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 6c0d5aa2..430e080b 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1543,9 +1543,14 @@ class LinstorVolumeManager(object): + ) + + # 2. Create storage pool on each node + resource group. ++ reg_volume_group_not_found = re.compile( ++ ".*Volume group '.*' not found$" ++ ) ++ + i = 0 + try: + # 2.a. Create storage pools. ++ storage_pool_count = 0 + while i < len(node_names): + node_name = node_names[i] + +@@ -1556,17 +1561,35 @@ class LinstorVolumeManager(object): + driver_pool_name=driver_pool_name + ) + +- error_str = cls._get_error_str(result) +- if error_str: +- raise LinstorVolumeManagerError( +- 'Could not create SP `{}` on node `{}`: {}'.format( +- group_name, +- node_name, +- error_str ++ errors = linstor.Linstor.filter_api_call_response_errors( ++ result ++ ) ++ if errors: ++ if len(errors) == 1 and errors[0].is_error( ++ linstor.consts.FAIL_STOR_POOL_CONFIGURATION_ERROR ++ ) and reg_volume_group_not_found.match(errors[0].message): ++ logger( ++ 'Volume group `{}` not found on `{}`. Ignoring...' ++ .format(group_name, node_name) + ) +- ) ++ cls._destroy_storage_pool(lin, group_name, node_name) ++ else: ++ error_str = cls._get_error_str(result) ++ raise LinstorVolumeManagerError( ++ 'Could not create SP `{}` on node `{}`: {}' ++ .format(group_name, node_name, error_str) ++ ) ++ else: ++ storage_pool_count += 1 + i += 1 + ++ if not storage_pool_count: ++ raise LinstorVolumeManagerError( ++ 'Unable to create SR `{}`: No VG group found'.format( ++ group_name, ++ ) ++ ) ++ + # 2.b. Create resource group. + result = lin.resource_group_create( + name=group_name, +@@ -2345,13 +2368,22 @@ class LinstorVolumeManager(object): + # "Not enough available nodes" + # I don't understand why but this command protect against this bug. + try: +- lin.storage_pool_list_raise(filter_by_stor_pools=[group_name]) ++ pools = lin.storage_pool_list_raise( ++ filter_by_stor_pools=[group_name] ++ ) + except Exception as e: + raise LinstorVolumeManagerError( + 'Failed to get storage pool list before database creation: {}' + .format(e) + ) + ++ # Ensure we have a correct list of storage pools. ++ nodes_with_pool = map(lambda pool: pool.node_name, pools.storage_pools) ++ assert nodes_with_pool # We must have at least one storage pool! ++ for node_name in nodes_with_pool: ++ assert node_name in node_names ++ util.SMlog('Nodes with storage pool: {}'.format(nodes_with_pool)) ++ + # Create the database definition. + size = cls.round_up_volume_size(DATABASE_SIZE) + cls._check_volume_creation_errors(lin.resource_group_spawn( +@@ -2364,14 +2396,26 @@ class LinstorVolumeManager(object): + + # Create real resources on the first nodes. + resources = [] +- for node_name in node_names[:redundancy]: ++ ++ diskful_nodes = [] ++ diskless_nodes = [] ++ for node_name in node_names: ++ if node_name in nodes_with_pool: ++ diskful_nodes.append(node_name) ++ else: ++ diskless_nodes.append(node_name) ++ ++ assert diskful_nodes ++ for node_name in diskful_nodes[:redundancy]: ++ util.SMlog('Create database diskful on {}'.format(node_name)) + resources.append(linstor.ResourceData( + node_name=node_name, + rsc_name=DATABASE_VOLUME_NAME, + storage_pool=group_name + )) + # Create diskless resources on the remaining set. +- for node_name in node_names[redundancy:]: ++ for node_name in diskful_nodes[redundancy:] + diskless_nodes: ++ util.SMlog('Create database diskless on {}'.format(node_name)) + resources.append(linstor.ResourceData( + node_name=node_name, + rsc_name=DATABASE_VOLUME_NAME, diff --git a/SOURCES/0056-feat-LinstorSR-add-a-config-var-to-disable-HTTP-NBD-.patch b/SOURCES/0056-feat-LinstorSR-add-a-config-var-to-disable-HTTP-NBD-.patch new file mode 100644 index 0000000..bc346b6 --- /dev/null +++ b/SOURCES/0056-feat-LinstorSR-add-a-config-var-to-disable-HTTP-NBD-.patch @@ -0,0 +1,51 @@ +From 2e4d783b1d7cd7dd9a7ac40504ac6ec71dcde60c Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 14 Apr 2022 10:30:23 +0200 +Subject: [PATCH 056/177] feat(LinstorSR): add a config var to disable HTTP/NBD + servers + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 15 +++++++++++++-- + 1 file changed, 13 insertions(+), 2 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 413c5501..927e4771 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -61,6 +61,13 @@ XHA_CONFIG_PATH = '/etc/xensource/xhad.conf' + + FORK_LOG_DAEMON = '/opt/xensource/libexec/fork-log-daemon' + ++# This flag can be disabled to debug the DRBD layer. ++# When this config var is False, the HA can only be used under ++# specific conditions: ++# - Only one heartbeat diskless VDI is present in the pool. ++# - The other hearbeat volumes must be diskful and limited to a maximum of 3. ++USE_HTTP_NBD_SERVERS = True ++ + # ============================================================================== + + # TODO: Supports 'VDI_INTRODUCE', 'VDI_RESET_ON_BOOT/2', 'SR_TRIM', +@@ -1769,7 +1776,11 @@ class LinstorVDI(VDI.VDI): + self.xenstore_data = {} + self.xenstore_data['storage-type'] = LinstorSR.DRIVER_TYPE + +- if attach_from_config and self.path.startswith('/dev/http-nbd/'): ++ if ( ++ USE_HTTP_NBD_SERVERS and ++ attach_from_config and ++ self.path.startswith('/dev/http-nbd/') ++ ): + return self._attach_using_http_nbd() + + if not util.pathexists(self.path): +@@ -1934,7 +1945,7 @@ class LinstorVDI(VDI.VDI): + # We can't increase this limitation, so we use a NBD/HTTP device + # instead. + volume_name = self._linstor.get_volume_name(self.uuid) +- if volume_name not in [ ++ if not USE_HTTP_NBD_SERVERS or volume_name not in [ + 'xcp-persistent-ha-statefile', 'xcp-persistent-redo-log' + ]: + if not self.path or not util.pathexists(self.path): diff --git a/SOURCES/0057-feat-LinstorSr-ensure-LVM-group-is-activated-during-.patch b/SOURCES/0057-feat-LinstorSr-ensure-LVM-group-is-activated-during-.patch new file mode 100644 index 0000000..e25e74e --- /dev/null +++ b/SOURCES/0057-feat-LinstorSr-ensure-LVM-group-is-activated-during-.patch @@ -0,0 +1,96 @@ +From c241c1495897b952e6ea11fd062cac28109394aa Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 14 Apr 2022 15:45:20 +0200 +Subject: [PATCH 057/177] feat(LinstorSr): ensure LVM group is activated during + SR.attach/create + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 23 +++++++++++++++++------ + drivers/linstor-manager | 2 ++ + 2 files changed, 19 insertions(+), 6 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 927e4771..e2d3d783 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -34,6 +34,7 @@ import cleanup + import distutils + import errno + import functools ++import lvutil + import os + import re + import scsiutil +@@ -320,6 +321,15 @@ def get_ips_from_xha_config_file(): + + return (host_ip, ips) + ++ ++def activate_lvm_group(group_name): ++ path = group_name.split('/') ++ assert path and len(path) <= 2 ++ try: ++ lvutil.setActiveVG(path[0], True) ++ except Exception as e: ++ util.SMlog('Cannot active VG `{}`: {}'.format(path[0], e)) ++ + # ============================================================================== + + # Usage example: +@@ -673,7 +683,7 @@ class LinstorSR(SR.SR): + # Ensure ports are opened and LINSTOR satellites + # are activated. In the same time the minidrbdcluster instances + # must be stopped. +- self._prepare_sr_on_all_hosts(enabled=True) ++ self._prepare_sr_on_all_hosts(self._group_name, enabled=True) + + # Create SR. + # Throw if the SR already exists. +@@ -798,6 +808,7 @@ class LinstorSR(SR.SR): + 'SRUnavailable', + opterr='no such group: {}'.format(self._group_name) + ) ++ activate_lvm_group(self._group_name) + + @_locked_load + def detach(self, uuid): +@@ -907,20 +918,20 @@ class LinstorSR(SR.SR): + opterr='Plugin {} failed'.format(self.MANAGER_PLUGIN) + ) + +- def _prepare_sr(self, host, enabled): ++ def _prepare_sr(self, host, group_name, enabled): + self._exec_manager_command( + host, + 'prepareSr' if enabled else 'releaseSr', +- {}, ++ {'groupName': group_name}, + 'SRUnavailable' + ) + +- def _prepare_sr_on_all_hosts(self, enabled): ++ def _prepare_sr_on_all_hosts(self, group_name, enabled): + master = util.get_master_ref(self.session) +- self._prepare_sr(master, enabled) ++ self._prepare_sr(master, group_name, enabled) + + for slave in util.get_all_slaves(self.session): +- self._prepare_sr(slave, enabled) ++ self._prepare_sr(slave, group_name, enabled) + + def _update_minidrbdcluster(self, host, enabled): + self._exec_manager_command( +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 7e34ce65..91731b1d 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -76,6 +76,8 @@ def update_minidrbdcluster_service(start): + + def prepare_sr(session, args): + try: ++ LinstorSR.activate_lvm_group(args['groupName']) ++ + update_all_ports(open=True) + # We don't want to enable and start minidrbdcluster daemon during + # SR creation. diff --git a/SOURCES/0058-feat-linstor-manager-add-method-to-create-LinstorSR-.patch b/SOURCES/0058-feat-linstor-manager-add-method-to-create-LinstorSR-.patch new file mode 100644 index 0000000..b225781 --- /dev/null +++ b/SOURCES/0058-feat-linstor-manager-add-method-to-create-LinstorSR-.patch @@ -0,0 +1,244 @@ +From cac2dda720b05ba479b272c9c65b5f14af180956 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 25 Apr 2022 14:47:51 +0200 +Subject: [PATCH 058/177] feat(linstor-manager): add method to create LinstorSR + + to list/destroy DRBD volumes + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 179 +++++++++++++++++++++++++++++++++++++--- + 1 file changed, 168 insertions(+), 11 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 91731b1d..7893ebc6 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -23,6 +23,7 @@ sys.path[0] = '/opt/xensource/sm/' + import base64 + import distutils.util + import socket ++import XenAPI + import XenAPIPlugin + + from linstorjournaler import LinstorJournaler +@@ -30,9 +31,13 @@ from linstorvolumemanager import get_controller_uri, LinstorVolumeManager + from lock import Lock + import json + import LinstorSR ++import re + import util + import vhdutil + ++BACKING_DISK_RE = re.compile('^/dev/([^/]+)/(?:[^/]+)$') ++LVM_PLUGIN = 'lvm' ++THIN_POOL = 'thin_pool' + + FIREWALL_PORT_SCRIPT = '/etc/xapi.d/plugins/firewall-port' + LINSTOR_PORTS = [3366, 3370, 3376, 3377, '7000:8000', 8076, 8077] +@@ -55,14 +60,6 @@ def update_all_ports(open): + update_port(port, open) + + +-def stop_service(name): +- args = ('systemctl', 'stop', name) +- (ret, out, err) = util.doexec(args) +- if ret == 0: +- return +- raise Exception('Failed to stop {}: {} {}'.format(name, out, err)) +- +- + def update_linstor_satellite_service(start): + util.enable_and_start_service('linstor-satellite', start) + +@@ -71,6 +68,111 @@ def update_minidrbdcluster_service(start): + util.enable_and_start_service('minidrbdcluster', start) + + ++def exec_create_sr(session, name, description, disks, volume_group, redundancy, thin, force): ++ disks = json.loads(disks) ++ disk_hostnames = disks.keys() ++ ++ # Create volumes. ++ hosts = session.xenapi.host.get_all_records() ++ hostnames = [] ++ for host_ref, host_record in hosts.items(): ++ hostname = host_record['hostname'] ++ if hostname not in disk_hostnames: ++ continue ++ ++ if force: ++ try: ++ session.xenapi.host.call_plugin( ++ host_ref, LVM_PLUGIN, 'destroy_volume_group', { ++ 'vg_name': volume_group, ++ 'force': 'True' ++ } ++ ) ++ except Exception as e: ++ try: ++ response = session.xenapi.host.call_plugin( ++ host_ref, LVM_PLUGIN, 'list_volume_groups', { ++ 'vg_name': volume_group ++ } ++ ) ++ if response != '{}': ++ raise e ++ except Exception: ++ raise e ++ ++ host_devices = ','.join(disks[hostname]) ++ session.xenapi.host.call_plugin( ++ host_ref, LVM_PLUGIN, 'create_physical_volume', { ++ 'devices': host_devices, ++ 'force': str(force) ++ } ++ ) ++ ++ session.xenapi.host.call_plugin( ++ host_ref, LVM_PLUGIN, 'create_volume_group', { ++ 'vg_name': volume_group, ++ 'devices': host_devices ++ } ++ ) ++ ++ if thin: ++ session.xenapi.host.call_plugin( ++ host_ref, LVM_PLUGIN, 'create_thin_pool', { ++ 'vg_name': volume_group, ++ 'lv_name': THIN_POOL ++ } ++ ) ++ ++ # Create SR. ++ master_ref = session.xenapi.pool.get_all_records().values()[0]['master'] ++ ++ device_config = { ++ 'redundancy': redundancy, ++ 'provisioning': 'thin' if thin else 'thick', ++ 'group-name': '{}/{}'.format(volume_group, THIN_POOL) if thin else volume_group, ++ 'hosts': ','.join(hostnames), ++ 'monitor-db-quorum': str(len(hostnames) > 2) ++ } ++ sr_ref = session.xenapi.SR.create( ++ master_ref, device_config, '0', name, description, 'linstor', '', True, {} ++ ) ++ return session.xenapi.SR.get_uuid(sr_ref) ++ ++ ++def get_drbd_volumes(volume_group=None): ++ drbd_volumes = {} ++ (ret, stdout, stderr) = util.doexec(['drbdsetup', 'show', '--json']) ++ if ret: ++ raise Exception('Failed to get JSON object: {}'.format(stderr)) ++ ++ config = json.loads(stdout) ++ for resource in config: ++ for volume in resource['_this_host']['volumes']: ++ backing_disk = volume['backing-disk'] ++ match = BACKING_DISK_RE.match(backing_disk) ++ if not match: ++ continue ++ ++ cur_volume_group = match.groups()[0] ++ if volume_group and cur_volume_group != volume_group: ++ continue ++ ++ minor = int(volume['device_minor']) ++ if cur_volume_group in drbd_volumes: ++ drbd_volumes[cur_volume_group].append(minor) ++ else: ++ drbd_volumes[cur_volume_group] = [minor] ++ return drbd_volumes ++ ++ ++def force_destroy_drbd_volume(minor): ++ (ret, stdout, stderr) = util.doexec(['drbdsetup', 'detach', minor, '--force']) ++ if ret: ++ raise Exception('Failed to detach volume: {}'.format(stderr)) ++ (ret, stdout, stderr) = util.doexec(['drbdsetup', 'del-minor', minor]) ++ if ret: ++ raise Exception('Failed to destroy volume: {}'.format(stderr)) ++ + # ------------------------------------------------------------------------------ + + +@@ -169,8 +271,8 @@ def destroy(session, args): + linstor.destroy() + return str(True) + except Exception as e: +- stop_service('linstor-controller') +- stop_service('var-lib-linstor.service') ++ util.stop_service('linstor-controller') ++ util.stop_service('var-lib-linstor.service') + util.SMlog('linstor-manager:destroy error: {}'.format(e)) + return str(False) + +@@ -615,6 +717,57 @@ def remove_host(session, args): + return str('True') + + ++def create_sr(session, args): ++ try: ++ name = args['name'] ++ description = args.get('description') or '' ++ disks = json.loads(args['disks']) ++ volume_group = args['volume_group'] ++ redundancy = int(args['redundancy']) ++ thin = distutils.util.strtobool(args.get('thin') or '0') ++ force = distutils.util.strtobool(args.get('force') or '0') ++ return json.dumps(exec_create_sr( ++ session, name, description, disks, volume_group, redundancy, thin, force ++ )) ++ except Exception as e: ++ util.SMlog('linstor-manager:create_sr error: {}'.format(e)) ++ raise ++ ++ ++def list_drbd_volumes(session, args): ++ try: ++ volume_group = args.get('volume_group') ++ return json.dumps(get_drbd_volumes(volume_group)) ++ except Exception as e: ++ util.SMlog('linstor-manager:list_drbd_volumes error: {}'.format(e)) ++ raise ++ ++ ++def destroy_drbd_volume(session, args): ++ try: ++ minor = args.get('minor') ++ if not minor: ++ raise Exception('Cannot destroy DRBD volume without minor.') ++ force_destroy_drbd_volume(minor) ++ return str(True) ++ except Exception as e: ++ util.SMlog('linstor-manager:destroy_drbd_volume error: {}'.format(e)) ++ return str(False) ++ ++ ++def destroy_drbd_volumes(session, args): ++ try: ++ volume_group = args.get('volume_group') ++ if not volume_group: ++ raise Exception('Cannot destroy DRBD volumes without volume group.') ++ for minor in get_drbd_volumes(volume_group).get(volume_group, []): ++ force_destroy_drbd_volume(str(minor)) ++ return str(True) ++ except Exception as e: ++ util.SMlog('linstor-manager:destroy_drbd_volumes error: {}'.format(e)) ++ return str(False) ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -636,5 +789,9 @@ if __name__ == '__main__': + 'lsofResource': lsof_resource, + 'hasControllerRunning': has_controller_running, + 'addHost': add_host, +- 'removeHost': remove_host ++ 'removeHost': remove_host, ++ 'createSr': create_sr, ++ 'listDrbdVolumes': list_drbd_volumes, ++ 'destroyDrbdVolume': destroy_drbd_volume, ++ 'destroyDrbdVolumes': destroy_drbd_volumes + }) diff --git a/SOURCES/0059-fix-LinstorSR-always-set-vdi_path-in-generate_config.patch b/SOURCES/0059-fix-LinstorSR-always-set-vdi_path-in-generate_config.patch new file mode 100644 index 0000000..14d1e35 --- /dev/null +++ b/SOURCES/0059-fix-LinstorSR-always-set-vdi_path-in-generate_config.patch @@ -0,0 +1,28 @@ +From f990493afc8187a52faab4913a6b353f70ab1583 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 26 Apr 2022 11:20:08 +0200 +Subject: [PATCH 059/177] fix(LinstorSR): always set vdi_path in + generate_config + +If the volume of a generated config is not related to HTTP/NBD +and if we already have a path to the resource, the VDI path is never +written to the config. So the config can't be used to attach the VDI... + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index e2d3d783..1855e3d9 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1970,7 +1970,7 @@ class LinstorVDI(VDI.VDI): + if not available: + raise xs_errors.XenError('VDIUnavailable') + +- resp['vdi_path'] = self.path ++ resp['vdi_path'] = self.path + else: + # Axiom: DRBD device is present on at least one host. + resp['vdi_path'] = '/dev/http-nbd/' + volume_name diff --git a/SOURCES/0060-fix-minidrbdcluster-supports-new-properties-like-for.patch b/SOURCES/0060-fix-minidrbdcluster-supports-new-properties-like-for.patch new file mode 100644 index 0000000..aa19534 --- /dev/null +++ b/SOURCES/0060-fix-minidrbdcluster-supports-new-properties-like-for.patch @@ -0,0 +1,34 @@ +From ed527602dea427ea63080a07395aaf7d99572953 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 13 May 2022 14:35:57 +0200 +Subject: [PATCH 060/177] fix(minidrbdcluster): supports new properties like + `force-io-failures` + +Signed-off-by: Ronan Abhamon +--- + scripts/minidrbdcluster | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/scripts/minidrbdcluster b/scripts/minidrbdcluster +index a04b6c1c..fb4de09b 100755 +--- a/scripts/minidrbdcluster ++++ b/scripts/minidrbdcluster +@@ -11,15 +11,15 @@ DRBDADM_OPEN_FAILED_RE = re.compile( + ) + MAY_PROMOT_RE = re.compile( + '(?:exists|change) resource name:((?:\\w|-)+) ' +- '(?:\\w+\\:\\w+ )*may_promote:(yes|no) promotion_score:(\\d+)' ++ '(?:(?:\\w|-)+\\:(?:\\w|-)+ )*may_promote:(yes|no) promotion_score:(\\d+)' + ) + PEER_ROLE_RE = re.compile( + '(?:exists|change) connection name:((?:\\w|-)+) peer-node-id:(?:\\d+) ' +- 'conn-name:(\\w+) (?:\\w+\\:\\w+ )*role:(Primary|Secondary|Unknown)' ++ 'conn-name:((?:\\w|-)+) (?:(?:\\w|-)+\\:(?:\\w|-)+ )*role:(Primary|Secondary|Unknown)' + ) + HAVE_QUORUM_RE = re.compile( + '(?:exists|change) device name:((?:\\w|-)+) ' +- '(?:\\w+\\:\\w+ )*quorum:(yes|no)' ++ '(?:(?:\\w|-)+\\:(?:\\w|-)+ )*quorum:(yes|no)' + ) + + diff --git a/SOURCES/0061-fix-LinstorSR-enabled-disable-minidrbcluster-with-fi.patch b/SOURCES/0061-fix-LinstorSR-enabled-disable-minidrbcluster-with-fi.patch new file mode 100644 index 0000000..0784c9b --- /dev/null +++ b/SOURCES/0061-fix-LinstorSR-enabled-disable-minidrbcluster-with-fi.patch @@ -0,0 +1,81 @@ +From 4a1220b9552323f2e79f9512ed030ae45ab4c800 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 18 May 2022 17:28:33 +0200 +Subject: [PATCH 061/177] fix(LinstorSR): enabled/disable minidrbcluster with + fixed order + +Ensure we disable minidrbdcluster during SR destruction on all hosts +with a deterministic execution to ensure linstor-controller is never restarted +on another host. It was possible before this patch because the host of the minidrbcluster +that was running the controller could be stopped before the others. Now the primary service +is stopped last. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 38 +++++++++++++++++++++++++++++++------- + 1 file changed, 31 insertions(+), 7 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 1855e3d9..57280e36 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -751,7 +751,9 @@ class LinstorSR(SR.SR): + ) + + try: +- self._update_minidrbdcluster_on_all_hosts(enabled=False) ++ self._update_minidrbdcluster_on_all_hosts( ++ controller_node_name=node_name, enabled=False ++ ) + + args = { + 'groupName': self._group_name, +@@ -761,7 +763,9 @@ class LinstorSR(SR.SR): + ) + except Exception as e: + try: +- self._update_minidrbdcluster_on_all_hosts(enabled=True) ++ self._update_minidrbdcluster_on_all_hosts( ++ controller_node_name=node_name, enabled=True ++ ) + except Exception as e2: + util.SMlog( + 'Failed to restart minidrbdcluster after destroy fail: {}' +@@ -941,12 +945,32 @@ class LinstorSR(SR.SR): + 'SRUnavailable' + ) + +- def _update_minidrbdcluster_on_all_hosts(self, enabled): +- master = util.get_master_ref(self.session) +- self._update_minidrbdcluster(master, enabled) ++ def _update_minidrbdcluster_on_all_hosts( ++ self, enabled, controller_node_name=None ++ ): ++ controller_host = None ++ secondary_hosts = [] + +- for slave in util.get_all_slaves(self.session): +- self._update_minidrbdcluster(slave, enabled) ++ hosts = self.session.xenapi.host.get_all_records() ++ for host_ref, host_rec in hosts.iteritems(): ++ if controller_node_name == host_rec['hostname']: ++ controller_host = host_ref ++ else: ++ secondary_hosts.append(host_ref) ++ ++ if enabled and controller_host: ++ # If enabled is true, we try to start the controller on the desired ++ # node name first. ++ self._update_minidrbdcluster(controller_host, enabled) ++ ++ for host in secondary_hosts: ++ self._update_minidrbdcluster(host, enabled) ++ ++ if not enabled and controller_host: ++ # If enabled is false, we disable the minidrbdcluster service of ++ # the controller host last. Why? Otherwise the linstor-controller ++ # of other nodes can be started, and we don't want that. ++ self._update_minidrbdcluster(controller_host, enabled) + + # -------------------------------------------------------------------------- + # Metadata. diff --git a/SOURCES/0062-fix-linstor-manager-change-linstor-satellite-start-b.patch b/SOURCES/0062-fix-linstor-manager-change-linstor-satellite-start-b.patch new file mode 100644 index 0000000..79490c5 --- /dev/null +++ b/SOURCES/0062-fix-linstor-manager-change-linstor-satellite-start-b.patch @@ -0,0 +1,38 @@ +From 77eddf395a3248f479ece8223c11da473ba7992b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 31 May 2022 14:01:45 +0200 +Subject: [PATCH 062/177] fix(linstor-manager): change linstor satellite start + behavior + +Ensure we don't have an invalid cache used by a satellite: +- We found an issue with a new added disk which used a volume group name + formerly involved by another disk. To avoid this kind of problem, we + always restart the satellite. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 11 ++++++++++- + 1 file changed, 10 insertions(+), 1 deletion(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 7893ebc6..c6d622f2 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -61,7 +61,16 @@ def update_all_ports(open): + + + def update_linstor_satellite_service(start): +- util.enable_and_start_service('linstor-satellite', start) ++ service = 'linstor-satellite' ++ ++ # Stop services in all cases first. ++ # Ensure we don't have an invalid cache used by a satellite. ++ # (We found an issue with a new added disk which used a volume group name ++ # formerly involved by another disk. To avoid this kind of problem, we ++ # always restart the satellite.) ++ util.enable_and_start_service(service, False) ++ if start: ++ util.enable_and_start_service(service, True) + + + def update_minidrbdcluster_service(start): diff --git a/SOURCES/0063-Fix-is_open-call-for-LinstorSR.patch b/SOURCES/0063-Fix-is_open-call-for-LinstorSR.patch new file mode 100644 index 0000000..5d55ce6 --- /dev/null +++ b/SOURCES/0063-Fix-is_open-call-for-LinstorSR.patch @@ -0,0 +1,103 @@ +From 4ee79f2016b9816380baaba51b75c4b35830ab5f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 2 Jun 2022 09:04:28 +0200 +Subject: [PATCH 063/177] Fix is_open call for LinstorSR + +1. Ensure LinstorSR driver is imported in `_is_open` definition to register it in the driver list. +Otherwise this function always fails with a SRUnknownType exception. + +2. Fetch the dconf of the target SR to retrieve VDI path, i.e. we can't use fake params like +other drivers, we must have a real LINSTOR connection to read in the DB the volume location. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 10 +++++++++- + drivers/on_slave.py | 17 ++++++++++++++++- + tests/test_on_slave.py | 10 +++++++++- + 3 files changed, 34 insertions(+), 3 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 57280e36..e5f6f85c 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -445,7 +445,11 @@ class LinstorSR(SR.SR): + def load(self, *args, **kwargs): + if not self._has_session: + if self.srcmd.cmd in ( +- 'vdi_attach_from_config', 'vdi_detach_from_config' ++ 'vdi_attach_from_config', ++ 'vdi_detach_from_config', ++ # When on-slave (is_open) is executed we have an ++ # empty command. ++ None + ): + def create_linstor(uri, attempt_count=30): + self._linstor = LinstorVolumeManager( +@@ -482,6 +486,10 @@ class LinstorSR(SR.SR): + controller_uri, self._group_name, logger=util.SMlog + ) + ++ if self.srcmd.cmd is None: ++ # Only useful on on-slave plugin (is_open). ++ self._vhdutil = LinstorVhdUtil(self.session, self._linstor) ++ + return wrapped_method(self, *args, **kwargs) + + if not self._is_master: +diff --git a/drivers/on_slave.py b/drivers/on_slave.py +index 3c7bd340..bbef4f7f 100755 +--- a/drivers/on_slave.py ++++ b/drivers/on_slave.py +@@ -78,6 +78,7 @@ def _is_open(session, args): + import EXTSR + import LargeBlockSR + import GlusterFSSR ++ import LinstorSR + import LVHDSR + import MooseFSSR + import NFSSR +@@ -108,8 +109,22 @@ def _is_open(session, args): + } + cmd.params = {"command": None} + ++ sr_uuid = srRec["uuid"] ++ ++ # Another ugly piece of code to load a real Linstor SR, otherwise ++ # we can't fetch the VDI path. ++ if srType == 'linstor': ++ host_ref = util.get_this_host_ref(session) ++ sr_ref = session.xenapi.SR.get_by_uuid(sr_uuid) ++ ++ pbd = util.find_my_pbd(session, host_ref, sr_ref) ++ if pbd is None: ++ raise util.SMException('Failed to find Linstor PBD') ++ ++ cmd.dconf = session.xenapi.PBD.get_device_config(pbd) ++ + driver = SR.driver(srType) +- sr = driver(cmd, srRec["uuid"]) ++ sr = driver(cmd, sr_uuid) + vdi = sr.vdi(vdiUuid) + tapdisk = blktap2.Tapdisk.find_by_path(vdi.path) + util.SMlog("Tapdisk for %s: %s" % (vdi.path, tapdisk)) +diff --git a/tests/test_on_slave.py b/tests/test_on_slave.py +index 54ebcd38..4c12d903 100644 +--- a/tests/test_on_slave.py ++++ b/tests/test_on_slave.py +@@ -13,7 +13,15 @@ import on_slave + + class Test_on_slave_is_open(unittest.TestCase): + +- MOCK_IMPORTS = ['SRCommand', 'SR', 'NFSSR', 'EXTSR', 'LVHDSR', 'blktap2'] ++ MOCK_IMPORTS = [ ++ 'SRCommand', ++ 'SR', ++ 'NFSSR', ++ 'EXTSR', ++ 'LVHDSR', ++ 'LinstorSR', ++ 'blktap2' ++ ] + + def fake_import(self, name, *args): + print 'Asked to import {}'.format(name) diff --git a/SOURCES/0064-fix-linstorvhdutil-fix-boolean-params-of-check-call.patch b/SOURCES/0064-fix-linstorvhdutil-fix-boolean-params-of-check-call.patch new file mode 100644 index 0000000..6f96b55 --- /dev/null +++ b/SOURCES/0064-fix-linstorvhdutil-fix-boolean-params-of-check-call.patch @@ -0,0 +1,47 @@ +From 1c0757759ea80cbf9ff35cc487a0b7d3cc2bfeb6 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 2 Jun 2022 09:28:32 +0200 +Subject: [PATCH 064/177] fix(linstorvhdutil): fix boolean params of `check` + call + +`ignoreMissingFooter` and `fast` must be string types to be used with XAPI plugin API. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 6 ++++-- + drivers/linstorvhdutil.py | 5 ++++- + 2 files changed, 8 insertions(+), 3 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index c6d622f2..63c0e3ed 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -289,8 +289,10 @@ def destroy(session, args): + def check(session, args): + try: + device_path = args['devicePath'] +- ignore_missing_footer = args['ignoreMissingFooter'] +- fast = args['fast'] ++ ignore_missing_footer = distutils.util.strtobool( ++ args['ignoreMissingFooter'] ++ ) ++ fast = distutils.util.strtobool(args['fast']) + return str(vhdutil.check(device_path, ignore_missing_footer, fast)) + except Exception as e: + util.SMlog('linstor-manager:check error: {}'.format(e)) +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 9ba0ac3b..f3d98705 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -111,7 +111,10 @@ class LinstorVhdUtil: + # -------------------------------------------------------------------------- + + def check(self, vdi_uuid, ignore_missing_footer=False, fast=False): +- kwargs = {'ignoreMissingFooter': ignore_missing_footer, 'fast': fast} ++ kwargs = { ++ 'ignoreMissingFooter': str(ignore_missing_footer), ++ 'fast': str(fast) ++ } + return self._check(vdi_uuid, **kwargs) + + @linstorhostcall(vhdutil.check, 'check') diff --git a/SOURCES/0065-feat-linstor-manager-robustify-exec_create_sr.patch b/SOURCES/0065-feat-linstor-manager-robustify-exec_create_sr.patch new file mode 100644 index 0000000..714a350 --- /dev/null +++ b/SOURCES/0065-feat-linstor-manager-robustify-exec_create_sr.patch @@ -0,0 +1,203 @@ +From 2f674984e2454621bd71eb4c3a0573f40704bb45 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 30 Jun 2022 17:09:51 +0200 +Subject: [PATCH 065/177] feat(linstor-manager): robustify exec_create_sr + +- Use lvm.py XCP-ng xapi plugins instead of lvm (old name) +- Check arguments to create the SR +- Fix param types given to SR.create +- lsof_resource use verbose output if there is a lock or problem +- Remove useless `force` param on remove_host + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 112 ++++++++++++++++++++++++++++------------ + 1 file changed, 78 insertions(+), 34 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 63c0e3ed..2930a9ed 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -36,7 +36,7 @@ import util + import vhdutil + + BACKING_DISK_RE = re.compile('^/dev/([^/]+)/(?:[^/]+)$') +-LVM_PLUGIN = 'lvm' ++LVM_PLUGIN = 'lvm.py' + THIN_POOL = 'thin_pool' + + FIREWALL_PORT_SCRIPT = '/etc/xapi.d/plugins/firewall-port' +@@ -77,17 +77,16 @@ def update_minidrbdcluster_service(start): + util.enable_and_start_service('minidrbdcluster', start) + + +-def exec_create_sr(session, name, description, disks, volume_group, redundancy, thin, force): +- disks = json.loads(disks) ++def exec_create_sr(session, name, description, disks, volume_group, redundancy, provisioning, force): + disk_hostnames = disks.keys() ++ thin = provisioning == 'thin' + + # Create volumes. + hosts = session.xenapi.host.get_all_records() + hostnames = [] + for host_ref, host_record in hosts.items(): + hostname = host_record['hostname'] +- if hostname not in disk_hostnames: +- continue ++ hostnames.append(hostname) + + if force: + try: +@@ -109,10 +108,24 @@ def exec_create_sr(session, name, description, disks, volume_group, redundancy, + except Exception: + raise e + +- host_devices = ','.join(disks[hostname]) ++ if hostname not in disk_hostnames or not disks[hostname]: ++ if force or session.xenapi.host.call_plugin( ++ host_ref, LVM_PLUGIN, 'list_volume_groups', { ++ 'vg_name': volume_group ++ } ++ ) == '{}': ++ continue ++ raise Exception('Volume group should not exist on `{}`, you must remove it manually'.format(hostname)) ++ ++ host_disks = disks[hostname] ++ if type(host_disks) is list: ++ host_disks = ','.join(disks[hostname]) ++ else: ++ raise Exception('Disk value of `{}` must be a disk list'.format(hostname)) ++ + session.xenapi.host.call_plugin( + host_ref, LVM_PLUGIN, 'create_physical_volume', { +- 'devices': host_devices, ++ 'devices': host_disks, + 'force': str(force) + } + ) +@@ -120,7 +133,7 @@ def exec_create_sr(session, name, description, disks, volume_group, redundancy, + session.xenapi.host.call_plugin( + host_ref, LVM_PLUGIN, 'create_volume_group', { + 'vg_name': volume_group, +- 'devices': host_devices ++ 'devices': host_disks + } + ) + +@@ -132,20 +145,20 @@ def exec_create_sr(session, name, description, disks, volume_group, redundancy, + } + ) + +- # Create SR. +- master_ref = session.xenapi.pool.get_all_records().values()[0]['master'] +- +- device_config = { +- 'redundancy': redundancy, +- 'provisioning': 'thin' if thin else 'thick', +- 'group-name': '{}/{}'.format(volume_group, THIN_POOL) if thin else volume_group, +- 'hosts': ','.join(hostnames), +- 'monitor-db-quorum': str(len(hostnames) > 2) +- } +- sr_ref = session.xenapi.SR.create( +- master_ref, device_config, '0', name, description, 'linstor', '', True, {} +- ) +- return session.xenapi.SR.get_uuid(sr_ref) ++ # Create SR. ++ master_ref = session.xenapi.pool.get_all_records().values()[0]['master'] ++ ++ device_config = { ++ 'redundancy': str(redundancy), ++ 'provisioning': 'thin' if thin else 'thick', ++ 'group-name': '{}/{}'.format(volume_group, THIN_POOL) if thin else volume_group, ++ 'hosts': ','.join(hostnames), ++ 'monitor-db-quorum': str(len(hostnames) > 2) ++ } ++ sr_ref = session.xenapi.SR.create( ++ master_ref, device_config, '0', name, description, 'linstor', '', True, {} ++ ) ++ return session.xenapi.SR.get_uuid(sr_ref) + + + def get_drbd_volumes(volume_group=None): +@@ -435,13 +448,13 @@ def lock_vdi(session, args): + def lsof_resource(session, args): + try: + drbd_path = args['drbdPath'] +- (ret, stdout, stderr) = util.doexec(['lsof', drbd_path]) ++ (ret, stdout, stderr) = util.doexec(['lsof', '-V', drbd_path]) + if ret == 0: + return 'DRBD resource `{}` is open: {}'.format( +- drbd_path, stdout ++ drbd_path, stdout.rstrip() + ) + return '`lsof` on DRBD resource `{}` returned {}: {}'.format( +- drbd_path, ret, stderr ++ drbd_path, ret, stdout.rstrip() + ) + except Exception as e: + util.SMlog('linstor-manager:lsof_drbd error: {}'.format(e)) +@@ -645,7 +658,6 @@ def add_host(session, args): + + def remove_host(session, args): + group_name = args['groupName'] +- force = args.get('force') or False + + # 1. Find SRs and PBDs. + srs = dict() +@@ -730,16 +742,48 @@ def remove_host(session, args): + + def create_sr(session, args): + try: +- name = args['name'] ++ # Use a complex parsing contrary to the other functions because ++ # this helper is a public method and is not easy to use. ++ name = args.get('name') ++ if not name: ++ raise Exception('`name` is empty') ++ + description = args.get('description') or '' +- disks = json.loads(args['disks']) +- volume_group = args['volume_group'] +- redundancy = int(args['redundancy']) +- thin = distutils.util.strtobool(args.get('thin') or '0') ++ ++ disks = args.get('disks') ++ if not disks: ++ raise Exception('`disks` is empty') ++ try: ++ disks = json.loads(disks) ++ except Exception as e: ++ raise Exception('failed to decode `disks`: {}'.format(e)) ++ if type(disks) is not dict: ++ raise Exception('`disks` must be a JSON object') ++ ++ volume_group = args.get('volume_group') ++ if not volume_group: ++ raise Exception('`volume_group` is empty') ++ ++ redundancy = args.get('redundancy') ++ if not redundancy: ++ raise Exception('`redundancy` is empty') ++ ++ try: ++ redundancy = int(redundancy) ++ except Exception: ++ raise Exception('`redundancy` is not a number') ++ ++ provisioning = args.get('provisioning') ++ if not provisioning: ++ provisioning = 'thin' ++ elif provisioning != 'thin' and provisioning != 'thick': ++ raise Exception('unsupported provisioning') ++ + force = distutils.util.strtobool(args.get('force') or '0') +- return json.dumps(exec_create_sr( +- session, name, description, disks, volume_group, redundancy, thin, force +- )) ++ ++ return exec_create_sr( ++ session, name, description, disks, volume_group, redundancy, provisioning, force ++ ) + except Exception as e: + util.SMlog('linstor-manager:create_sr error: {}'.format(e)) + raise diff --git a/SOURCES/0066-fix-cleanup-print-LINSTOR-VDI-UUID-if-error-during-i.patch b/SOURCES/0066-fix-cleanup-print-LINSTOR-VDI-UUID-if-error-during-i.patch new file mode 100644 index 0000000..d2b097e --- /dev/null +++ b/SOURCES/0066-fix-cleanup-print-LINSTOR-VDI-UUID-if-error-during-i.patch @@ -0,0 +1,24 @@ +From 21f073b1a262414c1910773a02ca54cd6f92c57e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 8 Jul 2022 14:52:25 +0200 +Subject: [PATCH 066/177] fix(cleanup): print LINSTOR VDI UUID if error during + info loading (not SR UUID) + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 9e3a5b07..7eeeee7f 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -2974,7 +2974,7 @@ class LinstorSR(SR): + except Exception as e: + Util.log( + ' [VDI {}: failed to load VDI info]: {}' +- .format(self.uuid, e) ++ .format(vdi_uuid, e) + ) + info = vhdutil.VHDInfo(vdi_uuid) + info.error = 1 diff --git a/SOURCES/0067-feat-cleanup-raise-and-dump-DRBD-openers-in-case-of-.patch b/SOURCES/0067-feat-cleanup-raise-and-dump-DRBD-openers-in-case-of-.patch new file mode 100644 index 0000000..b56947b --- /dev/null +++ b/SOURCES/0067-feat-cleanup-raise-and-dump-DRBD-openers-in-case-of-.patch @@ -0,0 +1,143 @@ +From 7b48c2a2c1635be52db7ab1625bc5d6d3b6c635e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 21 Jul 2022 11:39:20 +0200 +Subject: [PATCH 067/177] feat(cleanup): raise and dump DRBD openers in case of + bad coalesce + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 24 ++++++-------------- + drivers/linstor-manager | 40 ++++++++++++++++++++++++++++++++- + drivers/linstorvolumemanager.py | 24 ++++++++++++++++++++ + 3 files changed, 70 insertions(+), 18 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 7eeeee7f..0a586a67 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -3079,27 +3079,17 @@ class LinstorSR(SR): + + def _checkSlaves(self, vdi): + try: +- states = self._linstor.get_usage_states(vdi.uuid) +- for node_name, state in states.items(): +- self._checkSlave(node_name, vdi, state) ++ all_openers = self._linstor.get_volume_openers(vdi.uuid) ++ for openers in all_openers.itervalues(): ++ for opener in openers.values(): ++ if opener['process-name'] != 'tapdisk': ++ raise util.SMException( ++ 'VDI {} is in use: {}'.format(vdi.uuid, all_openers) ++ ) + except LinstorVolumeManagerError as e: + if e.code != LinstorVolumeManagerError.ERR_VOLUME_NOT_EXISTS: + raise + +- @staticmethod +- def _checkSlave(node_name, vdi, state): +- # If state is None, LINSTOR doesn't know the host state +- # (bad connection?). +- if state is None: +- raise util.SMException( +- 'Unknown state for VDI {} on {}'.format(vdi.uuid, node_name) +- ) +- +- if state: +- raise util.SMException( +- 'VDI {} is in use on {}'.format(vdi.uuid, node_name) +- ) +- + + ################################################################################ + # +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 2930a9ed..81789e7a 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -823,6 +823,43 @@ def destroy_drbd_volumes(session, args): + return str(False) + + ++def get_drbd_openers(session, args): ++ try: ++ resource_name = args.get('resourceName') ++ volume = args.get('volume') ++ if not resource_name or volume is None: ++ raise Exception('Cannot get DRBD openers without resource name and/or volume.') ++ ++ path = '/sys/kernel/debug/drbd/resources/{}/volumes/{}/openers'.format( ++ resource_name, volume ++ ) ++ ++ with open(path, 'r') as openers: ++ # Not a big cost, so read all lines directly. ++ lines = openers.readlines() ++ ++ result = {} ++ ++ opener_re = re.compile('(.*)\\s+([0-9]+)\\s+([0-9]+)') ++ for line in lines: ++ match = opener_re.match(line) ++ assert match ++ ++ groups = match.groups() ++ process_name = groups[0] ++ pid = groups[1] ++ open_duration_ms = groups[2] ++ result[pid] = { ++ 'process-name': process_name, ++ 'open-duration': open_duration_ms ++ } ++ ++ return json.dumps(result) ++ except Exception as e: ++ util.SMlog('linstor-manager:get_drbd_openers error: {}'.format(e)) ++ raise ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -848,5 +885,6 @@ if __name__ == '__main__': + 'createSr': create_sr, + 'listDrbdVolumes': list_drbd_volumes, + 'destroyDrbdVolume': destroy_drbd_volume, +- 'destroyDrbdVolumes': destroy_drbd_volumes ++ 'destroyDrbdVolumes': destroy_drbd_volumes, ++ 'getDrbdOpeners': get_drbd_openers + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 430e080b..d17845b5 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1048,6 +1048,30 @@ class LinstorVolumeManager(object): + + return states + ++ def get_volume_openers(self, volume_uuid): ++ """ ++ Get openers of a volume. ++ :param str volume_uuid: The volume uuid to monitor. ++ :return: A dictionnary that contains openers. ++ :rtype: dict(str, obj) ++ """ ++ ++ PLUGIN_CMD = 'getDrbdOpeners' ++ ++ openers = {} ++ ++ session = util.get_localAPI_session() ++ hosts = session.xenapi.host.get_all_records() ++ for host_ref, host_record in hosts.items(): ++ openers[host_record['hostname']] = json.loads( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, { ++ 'resourceName': self.get_volume_name(volume_uuid), ++ 'volume': '0' ++ }) ++ ) ++ ++ return openers ++ + def get_volumes_with_name(self): + """ + Give a volume dictionnary that contains names actually owned. diff --git a/SOURCES/0068-feat-linstorvhdutil-trace-DRBD-openers-in-case-of-ER.patch b/SOURCES/0068-feat-linstorvhdutil-trace-DRBD-openers-in-case-of-ER.patch new file mode 100644 index 0000000..c2feda0 --- /dev/null +++ b/SOURCES/0068-feat-linstorvhdutil-trace-DRBD-openers-in-case-of-ER.patch @@ -0,0 +1,132 @@ +From 17e39c0d21e3e0385d51d27c3192f8a28022bb3c Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 22 Jul 2022 10:26:20 +0200 +Subject: [PATCH 068/177] feat(linstorvhdutil): trace DRBD openers in case of + EROFS errors + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 8 +++++++ + drivers/linstorvhdutil.py | 48 +++++++++++++++++++++++++++++++-------- + 2 files changed, 47 insertions(+), 9 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 81789e7a..9022499f 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -868,6 +868,13 @@ if __name__ == '__main__': + 'attach': attach, + 'detach': detach, + 'destroy': destroy, ++ ++ # vhdutil wrappers called by linstorvhdutil. ++ # Note: When a VHD is open in RO mode (so for all vhdutil getters), ++ # the LVM layer is used directly to bypass DRBD verifications. ++ # In this case there can't be EROFS errors. ++ # Note 2: We assume linstorvhdutil executes remote calls on diskful ++ # DRBDs, otherwise we still have EROFS errors... + 'check': check, + 'getVHDInfo': get_vhd_info, + 'hasParent': has_parent, +@@ -877,6 +884,7 @@ if __name__ == '__main__': + 'getDepth': get_depth, + 'getKeyHash': get_key_hash, + 'getBlockBitmap': get_block_bitmap, ++ + 'lockVdi': lock_vdi, + 'lsofResource': lsof_resource, + 'hasControllerRunning': has_controller_running, +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index f3d98705..d6a21c26 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -26,6 +26,36 @@ import xs_errors + MANAGER_PLUGIN = 'linstor-manager' + + ++def call_vhd_util(linstor, func, device_path, *args, **kwargs): ++ try: ++ return func(device_path, *args, **kwargs) ++ except util.CommandException as e: ++ # Raise if we don't have a lock on the volume on another host. ++ if e.code != errno.EROFS: ++ raise ++ ++ # Volume is locked on a host, find openers. ++ e_with_openers = None ++ try: ++ volume_uuid = linstor.get_volume_uuid_from_device_path( ++ device_path ++ ) ++ e_with_openers = util.CommandException( ++ e.code, ++ e.cmd, ++ e.reason + ' (openers: {})'.format( ++ linstor.get_volume_openers(volume_uuid) ++ ) ++ ) ++ except Exception as illformed_e: ++ raise util.CommandException( ++ e.code, ++ e.cmd, ++ e.reason + ' (unable to get openers: {})'.format(illformed_e) ++ ) ++ raise e_with_openers # pylint: disable = E0702 ++ ++ + def linstorhostcall(local_method, remote_method): + def decorated(func): + def wrapper(*args, **kwargs): +@@ -46,7 +76,7 @@ def linstorhostcall(local_method, remote_method): + + try: + if not in_use or socket.gethostname() in node_names: +- return local_method(device_path, *args[2:], **kwargs) ++ return call_vhd_util(self._linstor, local_method, device_path, *args[2:], **kwargs) + except util.CommandException as e: + # EMEDIUMTYPE constant (124) is not available in python2. + if e.code != errno.EROFS and e.code != 124: +@@ -177,35 +207,35 @@ class LinstorVhdUtil: + + @linstormodifier() + def create(self, path, size, static, msize=0): +- return vhdutil.create(path, size, static, msize) ++ return call_vhd_util(self._linstor, vhdutil.create, path, size, static, msize) + + @linstormodifier() + def set_size_virt_fast(self, path, size): +- return vhdutil.setSizeVirtFast(path, size) ++ return call_vhd_util(self._linstor, vhdutil.setSizeVirtFast, path, size) + + @linstormodifier() + def set_size_phys(self, path, size, debug=True): +- return vhdutil.setSizePhys(path, size, debug) ++ return call_vhd_util(self._linstor, vhdutil.setSizePhys, path, size, debug) + + @linstormodifier() + def set_parent(self, path, parentPath, parentRaw): +- return vhdutil.setParent(path, parentPath, parentRaw) ++ return call_vhd_util(self._linstor, vhdutil.setParent, path, parentPath, parentRaw) + + @linstormodifier() + def set_hidden(self, path, hidden=True): +- return vhdutil.setHidden(path, hidden) ++ return call_vhd_util(self._linstor, vhdutil.setHidden, path, hidden) + + @linstormodifier() + def set_key(self, path, key_hash): +- return vhdutil.setKey(path, key_hash) ++ return call_vhd_util(self._linstor, vhdutil.setKey, path, key_hash) + + @linstormodifier() + def kill_data(self, path): +- return vhdutil.killData(path) ++ return call_vhd_util(self._linstor, vhdutil.killData, path) + + @linstormodifier() + def snapshot(self, path, parent, parentRaw, msize=0, checkEmpty=True): +- return vhdutil.snapshot(path, parent, parentRaw, msize, checkEmpty) ++ return call_vhd_util(self._linstor, vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) + + # -------------------------------------------------------------------------- + # Helpers. diff --git a/SOURCES/0069-fix-linstorvolumemanager-compute-correctly-size-in-a.patch b/SOURCES/0069-fix-linstorvolumemanager-compute-correctly-size-in-a.patch new file mode 100644 index 0000000..c50230c --- /dev/null +++ b/SOURCES/0069-fix-linstorvolumemanager-compute-correctly-size-in-a.patch @@ -0,0 +1,95 @@ +From 28090053e877e2bfb9bedf42af9a65a70b355ff5 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 29 Jul 2022 17:25:48 +0200 +Subject: [PATCH 069/177] fix(linstorvolumemanager): compute correctly size in + allocated_volume_size + +Remove replication count in computation. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 59 +++++++++++++-------------------- + 1 file changed, 23 insertions(+), 36 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index d17845b5..3806cc91 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -481,28 +481,6 @@ class LinstorVolumeManager(object): + size = current_size + return (size or 0) * 1024 + +- @property +- def total_volume_size(self): +- """ +- Give the sum of all created volumes. The place count is used. +- :return: The physical required size to use the volumes. +- :rtype: int +- """ +- +- size = 0 +- for resource in self._get_resource_cache().resources: +- for volume in resource.volumes: +- # We ignore diskless pools of the form "DfltDisklessStorPool". +- if volume.storage_pool_name == self._group_name: +- current_size = volume.usable_size +- if current_size < 0: +- raise LinstorVolumeManagerError( +- 'Failed to get usable size of `{}` on `{}`' +- .format(resource.name, volume.storage_pool_name) +- ) +- size += current_size +- return size * 1024 +- + @property + def allocated_volume_size(self): + """ +@@ -514,25 +492,34 @@ class LinstorVolumeManager(object): + :rtype: int + """ + +- size = 0 ++ # Paths: /res_name/vol_number/size ++ sizes = {} ++ + for resource in self._get_resource_cache().resources: +- volume_size = None ++ if resource.name not in sizes: ++ current = sizes[resource.name] = {} ++ else: ++ current = sizes[resource.name] ++ + for volume in resource.volumes: + # We ignore diskless pools of the form "DfltDisklessStorPool". +- if volume.storage_pool_name == self._group_name: +- current_size = volume.allocated_size +- if current_size < 0: +- raise LinstorVolumeManagerError( +- 'Failed to get allocated size of `{}` on `{}`' +- .format(resource.name, volume.storage_pool_name) +- ) ++ if volume.storage_pool_name != self._group_name: ++ continue ++ ++ current_size = volume.allocated_size ++ if current_size < 0: ++ raise LinstorVolumeManagerError( ++ 'Failed to get allocated size of `{}` on `{}`' ++ .format(resource.name, volume.storage_pool_name) ++ ) ++ current[volume.number] = max(current_size, current.get(volume.number) or 0) + +- if volume_size is None or current_size > volume_size: +- volume_size = current_size +- if volume_size is not None: +- size += volume_size ++ total_size = 0 ++ for volumes in sizes.itervalues(): ++ for size in volumes.itervalues(): ++ total_size += size + +- return size * 1024 ++ return total_size * 1024 + + @property + def metadata(self): diff --git a/SOURCES/0070-feat-LinstorSR-use-DRBD-openers-instead-of-lsof-to-l.patch b/SOURCES/0070-feat-LinstorSR-use-DRBD-openers-instead-of-lsof-to-l.patch new file mode 100644 index 0000000..6abefb4 --- /dev/null +++ b/SOURCES/0070-feat-LinstorSR-use-DRBD-openers-instead-of-lsof-to-l.patch @@ -0,0 +1,324 @@ +From 97e4f95765285fcd3a860d66a8f02f170a4e8e73 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 9 Aug 2022 11:07:57 +0200 +Subject: [PATCH 070/177] feat(LinstorSR): use DRBD openers instead of lsof to + log in blktap2 + +Signed-off-by: Ronan Abhamon +--- + drivers/blktap2.py | 4 +- + drivers/linstor-manager | 48 +-------- + drivers/linstorvolumemanager.py | 176 +++++++++++++++++--------------- + 3 files changed, 100 insertions(+), 128 deletions(-) + +diff --git a/drivers/blktap2.py b/drivers/blktap2.py +index 14c564e6..370f7fb8 100755 +--- a/drivers/blktap2.py ++++ b/drivers/blktap2.py +@@ -36,7 +36,7 @@ import json + import xs_errors + import XenAPI + import scsiutil +-from linstorvolumemanager import log_lsof_drbd ++from linstorvolumemanager import log_drbd_openers + from syslog import openlog, syslog + from stat import * # S_ISBLK(), ... + import nfs +@@ -833,7 +833,7 @@ class Tapdisk(object): + time.sleep(1) + continue + if err == errno.EROFS: +- log_lsof_drbd(path) ++ log_drbd_openers(path) + raise + try: + tapdisk = cls.__from_blktap(blktap) +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 9022499f..4d0ba299 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -27,7 +27,7 @@ import XenAPI + import XenAPIPlugin + + from linstorjournaler import LinstorJournaler +-from linstorvolumemanager import get_controller_uri, LinstorVolumeManager ++from linstorvolumemanager import get_controller_uri, get_local_volume_openers, LinstorVolumeManager + from lock import Lock + import json + import LinstorSR +@@ -445,22 +445,6 @@ def lock_vdi(session, args): + return str(False) + + +-def lsof_resource(session, args): +- try: +- drbd_path = args['drbdPath'] +- (ret, stdout, stderr) = util.doexec(['lsof', '-V', drbd_path]) +- if ret == 0: +- return 'DRBD resource `{}` is open: {}'.format( +- drbd_path, stdout.rstrip() +- ) +- return '`lsof` on DRBD resource `{}` returned {}: {}'.format( +- drbd_path, ret, stdout.rstrip() +- ) +- except Exception as e: +- util.SMlog('linstor-manager:lsof_drbd error: {}'.format(e)) +- raise +- +- + def has_controller_running(session, args): + (ret, stdout, stderr) = util.doexec([ + 'systemctl', 'is-active', '--quiet', 'linstor-controller' +@@ -827,34 +811,7 @@ def get_drbd_openers(session, args): + try: + resource_name = args.get('resourceName') + volume = args.get('volume') +- if not resource_name or volume is None: +- raise Exception('Cannot get DRBD openers without resource name and/or volume.') +- +- path = '/sys/kernel/debug/drbd/resources/{}/volumes/{}/openers'.format( +- resource_name, volume +- ) +- +- with open(path, 'r') as openers: +- # Not a big cost, so read all lines directly. +- lines = openers.readlines() +- +- result = {} +- +- opener_re = re.compile('(.*)\\s+([0-9]+)\\s+([0-9]+)') +- for line in lines: +- match = opener_re.match(line) +- assert match +- +- groups = match.groups() +- process_name = groups[0] +- pid = groups[1] +- open_duration_ms = groups[2] +- result[pid] = { +- 'process-name': process_name, +- 'open-duration': open_duration_ms +- } +- +- return json.dumps(result) ++ return get_local_volume_openers(resource_name, volume) + except Exception as e: + util.SMlog('linstor-manager:get_drbd_openers error: {}'.format(e)) + raise +@@ -886,7 +843,6 @@ if __name__ == '__main__': + 'getBlockBitmap': get_block_bitmap, + + 'lockVdi': lock_vdi, +- 'lsofResource': lsof_resource, + 'hasControllerRunning': has_controller_running, + 'addHost': add_host, + 'removeHost': remove_host, +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 3806cc91..6f4c5900 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -45,81 +45,56 @@ DRBD_BY_RES_PATH = '/dev/drbd/by-res/' + PLUGIN = 'linstor-manager' + + +-# Check if a path is a DRBD resource and log the process name/pid +-# that opened it. +-def log_lsof_drbd(path): +- PLUGIN_CMD = 'lsofResource' ++# ============================================================================== + +- # Ignore if it's not a symlink to DRBD resource. +- if not path.startswith(DRBD_BY_RES_PATH): +- return ++def get_local_volume_openers(resource_name, volume): ++ if not resource_name or volume is None: ++ raise Exception('Cannot get DRBD openers without resource name and/or volume.') + +- # Compute resource name. +- res_name_end = path.find('/', len(DRBD_BY_RES_PATH)) +- if res_name_end == -1: +- return +- res_name = path[len(DRBD_BY_RES_PATH):res_name_end] ++ path = '/sys/kernel/debug/drbd/resources/{}/volumes/{}/openers'.format( ++ resource_name, volume ++ ) + +- try: +- # Ensure path is a DRBD. +- drbd_path = os.path.realpath(path) +- stats = os.stat(drbd_path) +- if not stat.S_ISBLK(stats.st_mode) or os.major(stats.st_rdev) != 147: +- return ++ with open(path, 'r') as openers: ++ # Not a big cost, so read all lines directly. ++ lines = openers.readlines() + +- # Find where the device is open. +- (ret, stdout, stderr) = util.doexec(['drbdadm', 'status', res_name]) +- if ret != 0: +- util.SMlog('Failed to execute `drbdadm status` on `{}`: {}'.format( +- res_name, stderr +- )) +- return ++ result = {} + +- # Is it a local device? +- if stdout.startswith('{} role:Primary'.format(res_name)): +- (ret, stdout, stderr) = util.doexec(['lsof', drbd_path]) +- if ret == 0: +- util.SMlog( +- 'DRBD resource `{}` is open on local host: {}' +- .format(path, stdout) +- ) +- else: +- util.SMlog( +- '`lsof` on local DRBD resource `{}` returned {}: {}' +- .format(path, ret, stderr) +- ) +- return ++ opener_re = re.compile('(.*)\\s+([0-9]+)\\s+([0-9]+)') ++ for line in lines: ++ match = opener_re.match(line) ++ assert match + +- # Is it a remote device? +- res = REG_DRBDADM_PRIMARY.search(stdout) +- if not res: +- util.SMlog( +- 'Cannot find where is open DRBD resource `{}`' +- .format(path) +- ) +- return +- node_name = res.groups()[0] ++ groups = match.groups() ++ process_name = groups[0] ++ pid = groups[1] ++ open_duration_ms = groups[2] ++ result[pid] = { ++ 'process-name': process_name, ++ 'open-duration': open_duration_ms ++ } + +- session = util.get_localAPI_session() +- hosts = session.xenapi.host.get_all_records() +- for host_ref, host_record in hosts.items(): +- if node_name != host_record['hostname']: +- continue ++ return json.dumps(result) + +- ret = session.xenapi.host.call_plugin( +- host_ref, PLUGIN, PLUGIN_CMD, {'drbdPath': drbd_path}, +- ) +- util.SMlog('DRBD resource `{}` status on host `{}`: {}'.format( +- path, host_ref, ret +- )) +- return +- util.SMlog('Cannot find primary host of DRBD resource {}'.format(path)) +- except Exception as e: +- util.SMlog( +- 'Got exception while trying to determine where DRBD resource ' + +- '`{}` is open: {}'.format(path, e) ++def get_all_volume_openers(resource_name, volume): ++ PLUGIN_CMD = 'getDrbdOpeners' ++ ++ volume = str(volume) ++ openers = {} ++ ++ session = util.get_localAPI_session() ++ hosts = session.xenapi.host.get_all_records() ++ for host_ref, host_record in hosts.items(): ++ openers[host_record['hostname']] = json.loads( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, { ++ 'resourceName': resource_name, ++ 'volume': volume ++ }) + ) + ++ return openers ++ + + # ============================================================================== + +@@ -1042,22 +1017,8 @@ class LinstorVolumeManager(object): + :return: A dictionnary that contains openers. + :rtype: dict(str, obj) + """ ++ return get_all_volume_openers(self.get_volume_name(volume_uuid), '0') + +- PLUGIN_CMD = 'getDrbdOpeners' +- +- openers = {} +- +- session = util.get_localAPI_session() +- hosts = session.xenapi.host.get_all_records() +- for host_ref, host_record in hosts.items(): +- openers[host_record['hostname']] = json.loads( +- session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, { +- 'resourceName': self.get_volume_name(volume_uuid), +- 'volume': '0' +- }) +- ) +- +- return openers + + def get_volumes_with_name(self): + """ +@@ -2755,3 +2716,58 @@ class LinstorVolumeManager(object): + 'Failed to umount volume {} on {}: {}' + .format(volume_path, mountpoint, e) + ) ++ ++ ++# ============================================================================== ++ ++# Check if a path is a DRBD resource and log the process name/pid ++# that opened it. ++def log_drbd_openers(path): ++ # Ignore if it's not a symlink to DRBD resource. ++ if not path.startswith(DRBD_BY_RES_PATH): ++ return ++ ++ # Compute resource name. ++ res_name_end = path.find('/', len(DRBD_BY_RES_PATH)) ++ if res_name_end == -1: ++ return ++ res_name = path[len(DRBD_BY_RES_PATH):res_name_end] ++ ++ volume_end = path.rfind('/') ++ if volume_end == res_name_end: ++ return ++ volume = path[volume_end + 1:] ++ ++ try: ++ # Ensure path is a DRBD. ++ drbd_path = os.path.realpath(path) ++ stats = os.stat(drbd_path) ++ if not stat.S_ISBLK(stats.st_mode) or os.major(stats.st_rdev) != 147: ++ return ++ ++ # Find where the device is open. ++ (ret, stdout, stderr) = util.doexec(['drbdadm', 'status', res_name]) ++ if ret != 0: ++ util.SMlog('Failed to execute `drbdadm status` on `{}`: {}'.format( ++ res_name, stderr ++ )) ++ return ++ ++ # Is it a local device? ++ if stdout.startswith('{} role:Primary'.format(res_name)): ++ util.SMlog( ++ 'DRBD resource `{}` is open on local host: {}' ++ .format(path, get_local_volume_openers(res_name, volume)) ++ ) ++ return ++ ++ # Is it a remote device? ++ util.SMlog( ++ 'DRBD resource `{}` is open on hosts: {}' ++ .format(path, get_all_volume_openers(res_name, volume)) ++ ) ++ except Exception as e: ++ util.SMlog( ++ 'Got exception while trying to determine where DRBD resource ' + ++ '`{}` is open: {}'.format(path, e) ++ ) diff --git a/SOURCES/0071-feat-LinstorSR-support-cProfile-to-trace-calls-when-.patch b/SOURCES/0071-feat-LinstorSR-support-cProfile-to-trace-calls-when-.patch new file mode 100644 index 0000000..cd54b99 --- /dev/null +++ b/SOURCES/0071-feat-LinstorSR-support-cProfile-to-trace-calls-when-.patch @@ -0,0 +1,90 @@ +From 7af2ef8980077fc54ee58661502c9b49f7b973c9 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 25 Aug 2022 12:11:18 +0200 +Subject: [PATCH 071/177] feat(LinstorSR): support cProfile to trace calls when + a command is executed + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 11 ++++++++++- + drivers/util.py | 42 ++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 52 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index e5f6f85c..00554d73 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -69,6 +69,9 @@ FORK_LOG_DAEMON = '/opt/xensource/libexec/fork-log-daemon' + # - The other hearbeat volumes must be diskful and limited to a maximum of 3. + USE_HTTP_NBD_SERVERS = True + ++# Useful flag to trace calls using cProfile. ++TRACE_PERFS = False ++ + # ============================================================================== + + # TODO: Supports 'VDI_INTRODUCE', 'VDI_RESET_ON_BOOT/2', 'SR_TRIM', +@@ -2779,6 +2782,12 @@ class LinstorVDI(VDI.VDI): + + + if __name__ == '__main__': +- SRCommand.run(LinstorSR, DRIVER_INFO) ++ def run(): ++ SRCommand.run(LinstorSR, DRIVER_INFO) ++ ++ if not TRACE_PERFS: ++ run() ++ else: ++ util.make_profile('LinstorSR', run) + else: + SR.registerSR(LinstorSR) +diff --git a/drivers/util.py b/drivers/util.py +index 7c52703c..fe8e13d1 100755 +--- a/drivers/util.py ++++ b/drivers/util.py +@@ -1862,3 +1862,45 @@ def check_pid_exists(pid): + return False + else: + return True ++ ++ ++def make_profile(name, function): ++ """ ++ Helper to execute cProfile using unique log file. ++ """ ++ ++ import cProfile ++ import itertools ++ import os.path ++ import time ++ ++ assert name ++ assert function ++ ++ FOLDER = '/tmp/sm-perfs/' ++ makedirs(FOLDER) ++ ++ filename = time.strftime('{}_%Y%m%d_%H%M%S.prof'.format(name)) ++ ++ def gen_path(path): ++ yield path ++ root, ext = os.path.splitext(path) ++ for i in itertools.count(start=1, step=1): ++ yield root + '.{}.'.format(i) + ext ++ ++ for profile_path in gen_path(FOLDER + filename): ++ try: ++ file = open_atomic(profile_path, 'w') ++ file.close() ++ break ++ except OSError as e: ++ if e.errno == errno.EEXIST: ++ pass ++ else: ++ raise ++ ++ try: ++ SMlog('* Start profiling of {} ({}) *'.format(name, filename)) ++ cProfile.runctx('function()', None, locals(), profile_path) ++ finally: ++ SMlog('* End profiling of {} ({}) *'.format(name, filename)) diff --git a/SOURCES/0072-fix-LinstorJournaler-reset-namespace-when-get-is-cal.patch b/SOURCES/0072-fix-LinstorJournaler-reset-namespace-when-get-is-cal.patch new file mode 100644 index 0000000..bc686b8 --- /dev/null +++ b/SOURCES/0072-fix-LinstorJournaler-reset-namespace-when-get-is-cal.patch @@ -0,0 +1,23 @@ +From ff1e4c6c25e4d804f7b4635dbf8cd4d696beef0a Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 24 Aug 2022 17:09:11 +0200 +Subject: [PATCH 072/177] fix(LinstorJournaler): reset namespace when `get` is + called + +Otherwise, we can be in the wrong namespace and the key to find will be inaccessible. +--- + drivers/linstorjournaler.py | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/drivers/linstorjournaler.py b/drivers/linstorjournaler.py +index 3993f601..1e85ec96 100755 +--- a/drivers/linstorjournaler.py ++++ b/drivers/linstorjournaler.py +@@ -107,6 +107,7 @@ class LinstorJournaler: + ) + + def get(self, type, identifier): ++ self._reset_namespace() + return self._journal.get(self._get_key(type, identifier)) + + def get_all(self, type): diff --git a/SOURCES/0073-fix-linstorvhdutil-fix-coalesce-with-VM-running-unde.patch b/SOURCES/0073-fix-linstorvhdutil-fix-coalesce-with-VM-running-unde.patch new file mode 100644 index 0000000..27fe6b0 --- /dev/null +++ b/SOURCES/0073-fix-linstorvhdutil-fix-coalesce-with-VM-running-unde.patch @@ -0,0 +1,551 @@ +From 3bda00e34c79c8448b61d3f02c09ae7d910eb9ce Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 25 Aug 2022 10:54:56 +0200 +Subject: [PATCH 073/177] fix(linstorvhdutil): fix coalesce with VM running + under specific scenario: + +When a VM is running, we can't coalesce without this patch with a long chain +of VHDs because a parent can be in use on another host, and so a EROFS +can be emitted by vhd-util. + +So to fix this problem we run vhd-util on the remote host instead of the master +in case of failure in the cleanup algorithm. + +Impacted vhd-util functions: coalesce, getParent, repair. +--- + drivers/LinstorSR.py | 12 +- + drivers/cleanup.py | 22 +++- + drivers/linstor-manager | 37 ++++++ + drivers/linstorvhdutil.py | 269 ++++++++++++++++++++++++++------------ + drivers/vhdutil.py | 5 +- + 5 files changed, 251 insertions(+), 94 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 00554d73..47ac3c85 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -163,7 +163,9 @@ def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + # If the virtual VHD size is lower than the LINSTOR volume size, + # there is nothing to do. + vhd_size = compute_volume_size( +- LinstorVhdUtil(session, linstor).get_size_virt(vdi_uuid), ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 ++ LinstorVhdUtil(session, linstor).get_size_virt(vdi_uuid), # pylint: disable = E1120 + image_type + ) + +@@ -207,7 +209,9 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + + device_path = linstor.get_device_path(vdi_uuid) + new_volume_size = LinstorVolumeManager.round_up_volume_size( +- LinstorVhdUtil(session, linstor).get_size_phys(device_path) ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 ++ LinstorVhdUtil(session, linstor).get_size_phys(device_path) # pylint: disable = E1120 + ) + + volume_info = linstor.get_volume_info(vdi_uuid) +@@ -1231,8 +1235,10 @@ class LinstorSR(SR.SR): + self.vdis[vdi_uuid] = vdi + + if vdi.vdi_type == vhdutil.VDI_TYPE_VHD: ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 + vdi.sm_config_override['key_hash'] = \ +- self._vhdutil.get_key_hash(vdi_uuid) ++ self._vhdutil.get_key_hash(vdi_uuid) # pylint: disable = E1120 + + # 4.c. Update CBT status of disks either just added + # or already in XAPI. +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 0a586a67..73293632 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -840,12 +840,14 @@ class VDI(object): + xapi.message.create(msg_name, "3", "SR", vdi.sr.uuid, msg_body) + _reportCoalesceError = staticmethod(_reportCoalesceError) + ++ def coalesce(self): ++ vhdutil.coalesce(self.path) ++ + def _doCoalesceVHD(vdi): + try: +- + startTime = time.time() + vhdSize = vdi.getSizeVHD() +- vhdutil.coalesce(vdi.path) ++ vdi.coalesce() + endTime = time.time() + vdi.sr.recordStorageSpeed(startTime, endTime, vhdSize) + except util.CommandException, ce: +@@ -1437,6 +1439,9 @@ class LinstorVDI(VDI): + ) + return super(LinstorVDI, self).pause(failfast) + ++ def coalesce(self): ++ self.sr._vhdutil.force_coalesce(self.path) ++ + def _relinkSkip(self): + abortFlag = IPCFlag(self.sr.uuid) + for child in self.children: +@@ -1461,6 +1466,19 @@ class LinstorVDI(VDI): + blktap2.VDI.tap_unpause(session, sr_uuid, vdi_uuid) + self.children = [] + ++ def _setParent(self, parent): ++ self.sr._vhdutil.force_parent(self.path, parent.path) ++ self.parent = parent ++ self.parentUuid = parent.uuid ++ parent.children.append(self) ++ try: ++ self.setConfig(self.DB_VHD_PARENT, self.parentUuid) ++ Util.log("Updated the vhd-parent field for child %s with %s" % \ ++ (self.uuid, self.parentUuid)) ++ except: ++ Util.log("Failed to update %s with vhd-parent field %s" % \ ++ (self.uuid, self.parentUuid)) ++ + def _setHidden(self, hidden=True): + HIDDEN_TAG = 'hidden' + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 4d0ba299..5485b900 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -416,6 +416,37 @@ def get_block_bitmap(session, args): + raise + + ++def set_parent(session, args): ++ try: ++ device_path = args['devicePath'] ++ parent_path = args['parentPath'] ++ vhdutil.setParent(device_path, parent_path, False) ++ return '' ++ except Exception as e: ++ util.SMlog('linstor-manager:set_parent error: {}'.format(e)) ++ raise ++ ++ ++def coalesce(session, args): ++ try: ++ device_path = args['devicePath'] ++ vhdutil.coalesce(device_path) ++ return '' ++ except Exception as e: ++ util.SMlog('linstor-manager:coalesce error: {}'.format(e)) ++ raise ++ ++ ++def repair(session, args): ++ try: ++ device_path = args['devicePath'] ++ vhdutil.repair(device_path) ++ return '' ++ except Exception as e: ++ util.SMlog('linstor-manager:repair error: {}'.format(e)) ++ raise ++ ++ + def lock_vdi(session, args): + lock = None + try: +@@ -842,6 +873,12 @@ if __name__ == '__main__': + 'getKeyHash': get_key_hash, + 'getBlockBitmap': get_block_bitmap, + ++ # Called by cleanup.py to coalesce when a primary ++ # is opened on a non-local host. ++ 'setParent': set_parent, ++ 'coalesce': coalesce, ++ 'repair': repair, ++ + 'lockVdi': lock_vdi, + 'hasControllerRunning': has_controller_running, + 'addHost': add_host, +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index d6a21c26..4d031e12 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -25,39 +25,30 @@ import xs_errors + + MANAGER_PLUGIN = 'linstor-manager' + ++# EMEDIUMTYPE constant (124) is not available in python2. ++EMEDIUMTYPE = 124 + +-def call_vhd_util(linstor, func, device_path, *args, **kwargs): +- try: +- return func(device_path, *args, **kwargs) +- except util.CommandException as e: +- # Raise if we don't have a lock on the volume on another host. +- if e.code != errno.EROFS: +- raise +- +- # Volume is locked on a host, find openers. +- e_with_openers = None ++ ++def call_vhd_util_on_host(session, host_ref, method, device_path, args): + try: +- volume_uuid = linstor.get_volume_uuid_from_device_path( +- device_path ++ response = session.xenapi.host.call_plugin( ++ host_ref, MANAGER_PLUGIN, method, args + ) +- e_with_openers = util.CommandException( +- e.code, +- e.cmd, +- e.reason + ' (openers: {})'.format( +- linstor.get_volume_openers(volume_uuid) +- ) +- ) +- except Exception as illformed_e: +- raise util.CommandException( +- e.code, +- e.cmd, +- e.reason + ' (unable to get openers: {})'.format(illformed_e) +- ) +- raise e_with_openers # pylint: disable = E0702 ++ except Exception as e: ++ util.SMlog('call-plugin ({} with {}) exception: {}'.format( ++ method, args, e ++ )) ++ raise ++ ++ util.SMlog('call-plugin ({} with {}) returned: {}'.format( ++ method, args, response ++ )) ++ ++ return response + + + def linstorhostcall(local_method, remote_method): +- def decorated(func): ++ def decorated(response_parser): + def wrapper(*args, **kwargs): + self = args[0] + vdi_uuid = args[1] +@@ -76,45 +67,27 @@ def linstorhostcall(local_method, remote_method): + + try: + if not in_use or socket.gethostname() in node_names: +- return call_vhd_util(self._linstor, local_method, device_path, *args[2:], **kwargs) ++ return self._call_local_vhd_util(local_method, device_path, *args[2:], **kwargs) + except util.CommandException as e: +- # EMEDIUMTYPE constant (124) is not available in python2. +- if e.code != errno.EROFS and e.code != 124: ++ if e.code != errno.EROFS and e.code != EMEDIUMTYPE: + raise + + # B. Execute the plugin on master or slave. +- def exec_remote_method(): ++ remote_args = { ++ 'devicePath': device_path, ++ 'groupName': self._linstor.group_name ++ } ++ remote_args.update(**kwargs) ++ remote_args = {str(key): str(value) for key, value in remote_args.iteritems()} ++ ++ def remote_call(): + host_ref = self._get_readonly_host( + vdi_uuid, device_path, node_names + ) +- args = { +- 'devicePath': device_path, +- 'groupName': self._linstor.group_name +- } +- args.update(**kwargs) +- +- try: +- response = self._session.xenapi.host.call_plugin( +- host_ref, MANAGER_PLUGIN, remote_method, args +- ) +- except Exception as e: +- util.SMlog('call-plugin ({} with {}) exception: {}'.format( +- remote_method, args, e +- )) +- raise ++ return call_vhd_util_on_host(self._session, host_ref, remote_method, device_path, remote_args) ++ response = util.retry(remote_call, 5, 2) + +- util.SMlog('call-plugin ({} with {}) returned: {}'.format( +- remote_method, args, response +- )) +- if response == 'False': +- raise xs_errors.XenError( +- 'VDIUnavailable', +- opterr='Plugin {} failed'.format(MANAGER_PLUGIN) +- ) +- kwargs['response'] = response +- +- util.retry(exec_remote_method, 5, 3) +- return func(*args, **kwargs) ++ return response_parser(self, vdi_uuid, response) + return wrapper + return decorated + +@@ -137,7 +110,7 @@ class LinstorVhdUtil: + self._linstor = linstor + + # -------------------------------------------------------------------------- +- # Getters. ++ # Getters: read locally and try on another host in case of failure. + # -------------------------------------------------------------------------- + + def check(self, vdi_uuid, ignore_missing_footer=False, fast=False): +@@ -153,11 +126,13 @@ class LinstorVhdUtil: + + def get_vhd_info(self, vdi_uuid, include_parent=True): + kwargs = {'includeParent': str(include_parent)} +- return self._get_vhd_info(vdi_uuid, self._extract_uuid, **kwargs) ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 ++ return self._get_vhd_info(vdi_uuid, self._extract_uuid, **kwargs) # pylint: disable = E1123 + + @linstorhostcall(vhdutil.getVHDInfo, 'getVHDInfo') +- def _get_vhd_info(self, vdi_uuid, *args, **kwargs): +- obj = json.loads(kwargs['response']) ++ def _get_vhd_info(self, vdi_uuid, response): ++ obj = json.loads(response) + + vhd_info = vhdutil.VHDInfo(vdi_uuid) + vhd_info.sizeVirt = obj['sizeVirt'] +@@ -171,71 +146,91 @@ class LinstorVhdUtil: + return vhd_info + + @linstorhostcall(vhdutil.hasParent, 'hasParent') +- def has_parent(self, vdi_uuid, **kwargs): +- return distutils.util.strtobool(kwargs['response']) ++ def has_parent(self, vdi_uuid, response): ++ return distutils.util.strtobool(response) + + def get_parent(self, vdi_uuid): + return self._get_parent(vdi_uuid, self._extract_uuid) + + @linstorhostcall(vhdutil.getParent, 'getParent') +- def _get_parent(self, vdi_uuid, *args, **kwargs): +- return kwargs['response'] ++ def _get_parent(self, vdi_uuid, response): ++ return response + + @linstorhostcall(vhdutil.getSizeVirt, 'getSizeVirt') +- def get_size_virt(self, vdi_uuid, **kwargs): +- return int(kwargs['response']) ++ def get_size_virt(self, vdi_uuid, response): ++ return int(response) + + @linstorhostcall(vhdutil.getSizePhys, 'getSizePhys') +- def get_size_phys(self, vdi_uuid, **kwargs): +- return int(kwargs['response']) ++ def get_size_phys(self, vdi_uuid, response): ++ return int(response) + + @linstorhostcall(vhdutil.getDepth, 'getDepth') +- def get_depth(self, vdi_uuid, **kwargs): +- return int(kwargs['response']) ++ def get_depth(self, vdi_uuid, response): ++ return int(response) + + @linstorhostcall(vhdutil.getKeyHash, 'getKeyHash') +- def get_key_hash(self, vdi_uuid, **kwargs): +- return kwargs['response'] or None ++ def get_key_hash(self, vdi_uuid, response): ++ return response or None + + @linstorhostcall(vhdutil.getBlockBitmap, 'getBlockBitmap') +- def get_block_bitmap(self, vdi_uuid, **kwargs): +- return base64.b64decode(kwargs['response']) ++ def get_block_bitmap(self, vdi_uuid, response): ++ return base64.b64decode(response) + + # -------------------------------------------------------------------------- +- # Setters. ++ # Setters: only used locally. + # -------------------------------------------------------------------------- + + @linstormodifier() + def create(self, path, size, static, msize=0): +- return call_vhd_util(self._linstor, vhdutil.create, path, size, static, msize) ++ return self._call_local_vhd_util(vhdutil.create, path, size, static, msize) + + @linstormodifier() + def set_size_virt_fast(self, path, size): +- return call_vhd_util(self._linstor, vhdutil.setSizeVirtFast, path, size) ++ return self._call_local_vhd_util(vhdutil.setSizeVirtFast, path, size) + + @linstormodifier() + def set_size_phys(self, path, size, debug=True): +- return call_vhd_util(self._linstor, vhdutil.setSizePhys, path, size, debug) ++ return self._call_local_vhd_util(vhdutil.setSizePhys, path, size, debug) + + @linstormodifier() +- def set_parent(self, path, parentPath, parentRaw): +- return call_vhd_util(self._linstor, vhdutil.setParent, path, parentPath, parentRaw) ++ def set_parent(self, path, parentPath, parentRaw=False): ++ return self._call_local_vhd_util(vhdutil.setParent, path, parentPath, parentRaw) + + @linstormodifier() + def set_hidden(self, path, hidden=True): +- return call_vhd_util(self._linstor, vhdutil.setHidden, path, hidden) ++ return self._call_local_vhd_util(vhdutil.setHidden, path, hidden) + + @linstormodifier() + def set_key(self, path, key_hash): +- return call_vhd_util(self._linstor, vhdutil.setKey, path, key_hash) ++ return self._call_local_vhd_util(vhdutil.setKey, path, key_hash) + + @linstormodifier() + def kill_data(self, path): +- return call_vhd_util(self._linstor, vhdutil.killData, path) ++ return self._call_local_vhd_util(vhdutil.killData, path) + + @linstormodifier() + def snapshot(self, path, parent, parentRaw, msize=0, checkEmpty=True): +- return call_vhd_util(self._linstor, vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) ++ return self._call_local_vhd_util(vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) ++ ++ # -------------------------------------------------------------------------- ++ # Remote setters: write locally and try on another host in case of failure. ++ # -------------------------------------------------------------------------- ++ ++ @linstormodifier() ++ def force_parent(self, path, parentPath, parentRaw=False): ++ kwargs = { ++ 'parentPath': str(parentPath), ++ 'parentRaw': parentRaw ++ } ++ return self._call_vhd_util(vhdutil.setParent, 'setParent', path, **kwargs) ++ ++ @linstormodifier() ++ def force_coalesce(self, path): ++ return self._call_vhd_util(vhdutil.coalesce, 'coalesce', path) ++ ++ @linstormodifier() ++ def force_repair(self, path): ++ return self._call_vhd_util(vhdutil.repair, 'repair', path) + + # -------------------------------------------------------------------------- + # Helpers. +@@ -273,3 +268,105 @@ class LinstorVhdUtil: + opterr='Unable to find a valid host from VDI: {} (path={})' + .format(vdi_uuid, device_path) + ) ++ ++ # -------------------------------------------------------------------------- ++ ++ def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): ++ try: ++ def local_call(): ++ return local_method(device_path, *args, **kwargs) ++ return util.retry(local_call, 5, 2) ++ except util.CommandException as e: ++ if e.code != errno.EROFS and e.code != EMEDIUMTYPE: ++ raise ++ ++ # Volume is locked on a host, find openers. ++ e_with_openers = None ++ try: ++ volume_uuid = self._linstor.get_volume_uuid_from_device_path( ++ device_path ++ ) ++ e_with_openers = util.CommandException( ++ e.code, ++ e.cmd, ++ e.reason + ' (openers: {})'.format( ++ self._linstor.get_volume_openers(volume_uuid) ++ ) ++ ) ++ except Exception as illformed_e: ++ raise util.CommandException( ++ e.code, ++ e.cmd, ++ e.reason + ' (unable to get openers: {})'.format(illformed_e) ++ ) ++ raise e_with_openers # pylint: disable = E0702 ++ ++ def _call_vhd_util(self, local_method, remote_method, device_path, *args, **kwargs): ++ # A. Try to write locally... ++ try: ++ def local_call(): ++ return local_method(device_path, *args, **kwargs) ++ return util.retry(local_call, 5, 2) ++ except util.CommandException as e: ++ if e.code != errno.EROFS and e.code != EMEDIUMTYPE: ++ raise ++ ++ # B. Execute the command on another host. ++ # B.1. Get host list. ++ try: ++ hosts = self._session.xenapi.host.get_all_records() ++ except Exception as e: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Unable to get host list to run vhd-util command `{}` (path={}): {}' ++ .format(remote_method, device_path, e) ++ ) ++ ++ # B.2. Prepare remote args. ++ remote_args = { ++ 'devicePath': device_path, ++ 'groupName': self._linstor.group_name ++ } ++ remote_args.update(**kwargs) ++ remote_args = {str(key): str(value) for key, value in remote_args.iteritems()} ++ ++ volume_uuid = self._linstor.get_volume_uuid_from_device_path( ++ device_path ++ ) ++ ++ # B.3. Call! ++ def remote_call(): ++ try: ++ all_openers = self._linstor.get_volume_openers(volume_uuid) ++ except Exception as e: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Unable to get DRBD openers to run vhd-util command `{}` (path={}): {}' ++ .format(remote_method, device_path, e) ++ ) ++ ++ no_host_found = True ++ for hostname, openers in all_openers.iteritems(): ++ if not openers: ++ continue ++ ++ try: ++ host_ref = next(ref for ref, rec in hosts.iteritems() if rec['hostname'] == hostname) ++ except StopIteration: ++ continue ++ ++ no_host_found = False ++ try: ++ return call_vhd_util_on_host(self._session, host_ref, remote_method, device_path, remote_args) ++ except Exception: ++ pass ++ ++ if no_host_found: ++ return local_method(device_path, *args, **kwargs) ++ ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='No valid host found to run vhd-util command `{}` (path={}): {}' ++ .format(remote_method, device_path, e) ++ ) ++ return util.retry(remote_call, 5, 2) +diff --git a/drivers/vhdutil.py b/drivers/vhdutil.py +index 0a8fe918..d75edb11 100755 +--- a/drivers/vhdutil.py ++++ b/drivers/vhdutil.py +@@ -97,9 +97,8 @@ def calcOverheadFull(virtual_size): + def fullSizeVHD(virtual_size): + return virtual_size + calcOverheadFull(virtual_size) + +-def ioretry(cmd): +- return util.ioretry(lambda: util.pread2(cmd), +- errlist = [errno.EIO, errno.EROFS, errno.EAGAIN]) ++def ioretry(cmd, errlist=[errno.EIO, errno.EAGAIN]): ++ return util.ioretry(lambda: util.pread2(cmd), errlist) + + def getVHDInfo(path, extractUuidFunction, includeParent = True): + """Get the VHD info. The parent info may optionally be omitted: vhd-util diff --git a/SOURCES/0074-fix-linstorvolumemanager-_get_volumes_info-doesn-t-r.patch b/SOURCES/0074-fix-linstorvolumemanager-_get_volumes_info-doesn-t-r.patch new file mode 100644 index 0000000..cdb76c2 --- /dev/null +++ b/SOURCES/0074-fix-linstorvolumemanager-_get_volumes_info-doesn-t-r.patch @@ -0,0 +1,47 @@ +From 94ede43831ca0996b88a27626a9559d3f6b1f403 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 5 Sep 2022 15:09:17 +0200 +Subject: [PATCH 074/177] fix(linstorvolumemanager): `_get_volumes_info` + doesn't raise with offline host + +Ensure this method doesn't raise an exception when a host is offline. +Otherwise we can't use properly the HA when a host is unreachable and it's a +problem to restart VMs on a valid host. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 21 ++++++++++++--------- + 1 file changed, 12 insertions(+), 9 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 6f4c5900..a1bc151e 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1789,15 +1789,18 @@ class LinstorVolumeManager(object): + max(current.allocated_size, allocated_size) or \ + allocated_size + +- if volume.usable_size < 0: +- raise LinstorVolumeManagerError( +- 'Failed to get usable size of `{}` on `{}`' +- .format(resource.name, volume.storage_pool_name) +- ) +- virtual_size = volume.usable_size +- +- current.virtual_size = current.virtual_size and \ +- min(current.virtual_size, virtual_size) or virtual_size ++ usable_size = volume.usable_size ++ if usable_size > 0 and ( ++ usable_size < current.virtual_size or ++ not current.virtual_size ++ ): ++ current.virtual_size = usable_size ++ ++ if current.virtual_size <= 0: ++ raise LinstorVolumeManagerError( ++ 'Failed to get usable size of `{}` on `{}`' ++ .format(resource.name, volume.storage_pool_name) ++ ) + + for current in all_volume_info.values(): + current.allocated_size *= 1024 diff --git a/SOURCES/0075-fix-linstorvolumemanager-remove-double-prefix-on-kv-.patch b/SOURCES/0075-fix-linstorvolumemanager-remove-double-prefix-on-kv-.patch new file mode 100644 index 0000000..f0161d2 --- /dev/null +++ b/SOURCES/0075-fix-linstorvolumemanager-remove-double-prefix-on-kv-.patch @@ -0,0 +1,37 @@ +From 5d26e090abc329bd612d52e2c82fd4607c01ef17 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 12 Sep 2022 15:56:09 +0200 +Subject: [PATCH 075/177] fix(linstorvolumemanager): remove double prefix on kv + group name + +- Before this patch, when the kv store was created/accessed, a double "xcp-sr-" prefix was used. +- This change is not compatible with existing LINSTOR SR instances! + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 5 +---- + 1 file changed, 1 insertion(+), 4 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index a1bc151e..3ee5d248 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2195,7 +2195,7 @@ class LinstorVolumeManager(object): + + def _create_linstor_kv(self, namespace): + return linstor.KV( +- self._get_store_name(), ++ self._group_name, + uri=self._linstor.controller_host(), + namespace=namespace + ) +@@ -2205,9 +2205,6 @@ class LinstorVolumeManager(object): + properties.namespace = self._build_volume_namespace(volume_uuid) + return properties + +- def _get_store_name(self): +- return 'xcp-sr-{}'.format(self._group_name) +- + @classmethod + def _build_sr_namespace(cls): + return '/{}/'.format(cls.NAMESPACE_SR) diff --git a/SOURCES/0076-feat-LinstorSR-add-linstor-kv-dump-helper-to-print-k.patch b/SOURCES/0076-feat-LinstorSR-add-linstor-kv-dump-helper-to-print-k.patch new file mode 100644 index 0000000..67a255e --- /dev/null +++ b/SOURCES/0076-feat-LinstorSR-add-linstor-kv-dump-helper-to-print-k.patch @@ -0,0 +1,69 @@ +From cbe616a7450f988cc267eb02b73de8e03f546fd6 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 12 Sep 2022 17:54:57 +0200 +Subject: [PATCH 076/177] feat(LinstorSR): add linstor-kv-dump helper to print + kv store + +Signed-off-by: Ronan Abhamon +--- + Makefile | 1 + + scripts/linstor-kv-dump | 38 ++++++++++++++++++++++++++++++++++++++ + 2 files changed, 39 insertions(+) + create mode 100755 scripts/linstor-kv-dump + +diff --git a/Makefile b/Makefile +index af1011a1..aa71f809 100755 +--- a/Makefile ++++ b/Makefile +@@ -240,6 +240,7 @@ install: precheck + install -m 755 drivers/fcoelib.py $(SM_STAGING)$(SM_DEST) + mkdir -p $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/fork-log-daemon $(SM_STAGING)$(LIBEXEC) ++ install -m 755 scripts/linstor-kv-dump $(SM_STAGING)$(BIN_DEST) + install -m 755 scripts/local-device-change $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/check-device-sharing $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/usb_change $(SM_STAGING)$(LIBEXEC) +diff --git a/scripts/linstor-kv-dump b/scripts/linstor-kv-dump +new file mode 100755 +index 00000000..93598d7c +--- /dev/null ++++ b/scripts/linstor-kv-dump +@@ -0,0 +1,38 @@ ++#!/usr/bin/env python ++# ++# Copyright (C) 2022 Vates SAS ++# ++# This program is free software: you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation, either version 3 of the License, or ++# (at your option) any later version. ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++ ++import argparse ++import json ++import linstor ++ ++def dump_kv(controller_uri, group_name, namespace): ++ kv = linstor.KV( ++ group_name, ++ uri=controller_uri, ++ namespace=namespace ++ ) ++ print(json.dumps(kv, sort_keys=True, indent=2)) ++ ++def main(): ++ parser = argparse.ArgumentParser() ++ parser.add_argument('-u', '--uri', required=True) ++ parser.add_argument('-g', '--group-name', required=True) ++ parser.add_argument('-n', '--namespace', default='/') ++ args = parser.parse_args() ++ dump_kv(args.uri, args.group_name, args.namespace) ++ ++if __name__ == '__main__': ++ main() diff --git a/SOURCES/0077-fix-LinstorSR-disable-VHD-key-hash-usage-to-limit-ex.patch b/SOURCES/0077-fix-LinstorSR-disable-VHD-key-hash-usage-to-limit-ex.patch new file mode 100644 index 0000000..5484c6d --- /dev/null +++ b/SOURCES/0077-fix-LinstorSR-disable-VHD-key-hash-usage-to-limit-ex.patch @@ -0,0 +1,34 @@ +From 5232782abcfd52a3ff75a8f83906af61c5a0a77d Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 14 Sep 2022 10:17:18 +0200 +Subject: [PATCH 077/177] fix(LinstorSR): disable VHD key hash usage to limit + exec time + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 47ac3c85..374d6cb9 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -72,6 +72,9 @@ USE_HTTP_NBD_SERVERS = True + # Useful flag to trace calls using cProfile. + TRACE_PERFS = False + ++# Enable/Disable VHD key hash support. ++USE_KEY_HASH = False ++ + # ============================================================================== + + # TODO: Supports 'VDI_INTRODUCE', 'VDI_RESET_ON_BOOT/2', 'SR_TRIM', +@@ -1234,7 +1237,7 @@ class LinstorSR(SR.SR): + vdi = self.vdi(vdi_uuid) + self.vdis[vdi_uuid] = vdi + +- if vdi.vdi_type == vhdutil.VDI_TYPE_VHD: ++ if USE_KEY_HASH and vdi.vdi_type == vhdutil.VDI_TYPE_VHD: + # TODO: Replace pylint comment with this feature when possible: + # https://github.com/PyCQA/pylint/pull/2926 + vdi.sm_config_override['key_hash'] = \ diff --git a/SOURCES/0078-fix-minidrbdcluster-ensure-SIGINT-is-handled-correct.patch b/SOURCES/0078-fix-minidrbdcluster-ensure-SIGINT-is-handled-correct.patch new file mode 100644 index 0000000..8d64f08 --- /dev/null +++ b/SOURCES/0078-fix-minidrbdcluster-ensure-SIGINT-is-handled-correct.patch @@ -0,0 +1,96 @@ +From 9c9b5a3ea111d22c550e6f983ca7539b14908737 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 15 Sep 2022 11:34:25 +0200 +Subject: [PATCH 078/177] fix(minidrbdcluster): ensure SIGINT is handled + correctly + +This patch is here to make sure no LINSTOR controller survives when +systemd asks to minidrbdcluster to stop with `SIGINT`. + +- Remove `os.system`, it's totally unsafe, all signals are ignored with it. +- Use `subprocess.Popen` instead and catch correctly signal exceptions, it works + because `wait` call doesn't hide the signals. +- Ensure `SIGINT` is only sent to the main process, not to the subprocesses. +- Ensure `SIGKILL` is NEVER sent to minidrbdcluster. + +Signed-off-by: Ronan Abhamon +--- + scripts/minidrbdcluster | 35 ++++++++++++++++++++++++--------- + systemd/minidrbdcluster.service | 1 + + 2 files changed, 27 insertions(+), 9 deletions(-) + +diff --git a/scripts/minidrbdcluster b/scripts/minidrbdcluster +index fb4de09b..4cdc59e6 100755 +--- a/scripts/minidrbdcluster ++++ b/scripts/minidrbdcluster +@@ -1,7 +1,6 @@ + #! /usr/bin/env python2 + + import configparser +-import os + import re + import signal + import subprocess +@@ -35,24 +34,42 @@ def sig_handler(sig, frame): + ) + + ++def preexec_subprocess(): ++ signal.signal(signal.SIGINT, signal.SIG_IGN) ++ ++ ++def exec_subprocess(args): ++ proc = subprocess.Popen(args, preexec_fn=preexec_subprocess) ++ raise_sigint = False ++ while True: ++ try: ++ proc.wait() ++ break ++ except KeyboardInterrupt: ++ raise_sigint = True ++ except: # noqa: E722 ++ pass ++ ++ if raise_sigint: ++ raise KeyboardInterrupt ++ ++ return proc.returncode ++ ++ + def call_systemd(operation, service): + verbose = operation in ('start', 'stop') + if verbose: + print('Trying to %s %s' % (operation, service)) +- r = os.system('systemctl %s %s' % (operation, service)) ++ ret = exec_subprocess(['systemctl', operation, service]) + if verbose: + print('%s for %s %s' % ( +- 'success' if r == 0 else 'failure', operation, service ++ 'success' if ret == 0 else 'failure', operation, service + )) +- return r == 0 ++ return ret == 0 + + + def ensure_systemd_started(service): +- args = ['systemctl', 'is-active', '--quiet', service] +- +- proc = subprocess.Popen(args) +- proc.wait() +- if not proc.returncode: ++ if not exec_subprocess(['systemctl', 'is-active', '--quiet', service]): + return True # Already active. + + return call_systemd('start', service) +diff --git a/systemd/minidrbdcluster.service b/systemd/minidrbdcluster.service +index 3de6ac4f..1ddf91f3 100644 +--- a/systemd/minidrbdcluster.service ++++ b/systemd/minidrbdcluster.service +@@ -10,6 +10,7 @@ Environment=PYTHONUNBUFFERED=1 + ExecStart=/opt/xensource/libexec/minidrbdcluster + KillMode=process + KillSignal=SIGINT ++SendSIGKILL=no + StandardOutput=journal + StandardError=journal + SyslogIdentifier=minidrbdcluster diff --git a/SOURCES/0079-feat-minidrbdcluster-stop-resource-services-at-start.patch b/SOURCES/0079-feat-minidrbdcluster-stop-resource-services-at-start.patch new file mode 100644 index 0000000..8c1780f --- /dev/null +++ b/SOURCES/0079-feat-minidrbdcluster-stop-resource-services-at-start.patch @@ -0,0 +1,132 @@ +From 5f8a99eff7f2f137b7afac5738d45c1c56a8846b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 15 Sep 2022 11:49:34 +0200 +Subject: [PATCH 079/177] feat(minidrbdcluster): stop resource services at + startup + +- Ensure all services are stopped when minidrbcluster is started. +- Clean code to parse only once the systemd unit string. +- Log unhandled exceptions. + +Signed-off-by: Ronan Abhamon +--- + scripts/minidrbdcluster | 50 ++++++++++++++++++++++++++--------------- + 1 file changed, 32 insertions(+), 18 deletions(-) + +diff --git a/scripts/minidrbdcluster b/scripts/minidrbdcluster +index 4cdc59e6..eae7cbfe 100755 +--- a/scripts/minidrbdcluster ++++ b/scripts/minidrbdcluster +@@ -83,8 +83,7 @@ def show_status(services, status): + print('%s is %s' % (res_name, status[res_name])) + + +-def clean_up(services): +- print('exiting:') ++def stop_services(services): + for systemd_unit in reversed(services): + call_systemd('stop', systemd_unit) + +@@ -98,18 +97,17 @@ def get_systemd_units(systemd_units_str): + return systemd_units + + +-def process(events2, resources, services, status): ++def process(events2, resources, running_services, status): + line = events2.stdout.readline() + m = MAY_PROMOT_RE.match(line) + if m: + res_name, may_promote, promotion_score = m.groups() + if res_name in resources and may_promote == 'yes': +- systemd_units_str = resources[res_name]['systemd-units'] +- for systemd_unit in get_systemd_units(systemd_units_str): ++ for systemd_unit in resources[res_name]['systemd-units']: + if not ensure_systemd_started(systemd_unit): + break +- if systemd_unit not in services: +- services.append(systemd_unit) ++ if systemd_unit not in running_services: ++ running_services.append(systemd_unit) + m = PEER_ROLE_RE.match(line) + if m: + res_name, conn_name, role = m.groups() +@@ -119,15 +117,14 @@ def process(events2, resources, services, status): + if m: + res_name, have_quorum = m.groups() + if res_name in resources and have_quorum == 'no': +- systemd_units_str = resources[res_name]['systemd-units'] +- systemd_units = get_systemd_units(systemd_units_str) +- to_stop = [x for x in systemd_units if x in services] ++ systemd_units = resources[res_name]['systemd-units'] ++ to_stop = [x for x in systemd_units if x in running_services] + if to_stop: + print('Lost quorum on %s' % (res_name)) + for systemd_unit in reversed(to_stop): + r = call_systemd('stop', systemd_unit) + if r: +- services.remove(systemd_unit) ++ running_services.remove(systemd_unit) + + + def active_drbd_volume(res_name): +@@ -152,8 +149,7 @@ def active_drbd_volume(res_name): + + + def main(): +- services = [] +- status = dict() ++ # 1. Load minidrbdcluster config. + config = configparser.ConfigParser() + config.read('/etc/minidrbdcluster.ini') + resources = config._sections +@@ -162,12 +158,28 @@ def main(): + 'No resources to watch, maybe /etc/minidrbdcluster.ini missing' + ) + print('Managing DRBD resources: %s' % (' '.join(resources))) +- for res_name in resources: ++ ++ # 2. Prepare resources. ++ status = dict() ++ all_services = [] # Contains common services between each DRBD volumes. ++ for res_name, resource in resources.iteritems(): + status[res_name] = dict() + active_drbd_volume(res_name) ++ systemd_units = get_systemd_units(resource['systemd-units']) ++ resource['systemd-units'] = systemd_units ++ ++ for systemd_unit in systemd_units: ++ if systemd_unit not in all_services: ++ all_services.append(systemd_unit) + ++ # 3. Ensure all services are stopped. ++ stop_services(all_services) ++ ++ # 4. Run! + signal.signal(signal.SIGHUP, sig_handler) + ++ running_services = [] ++ + print('Starting process...') + events2 = subprocess.Popen( + ['drbdsetup', 'events2'], stdout=subprocess.PIPE +@@ -175,14 +187,16 @@ def main(): + run = True + while run: + try: +- process(events2, resources, services, status) ++ process(events2, resources, running_services, status) + except KeyboardInterrupt: + run = False + except SigHupException: +- show_status(services, status) +- +- clean_up(services) ++ show_status(running_services, status) ++ except Exception: ++ print('Unhandled exception: %s' % str(e)) + ++ print('Exiting...') ++ stop_services(running_services) + + if __name__ == '__main__': + main() diff --git a/SOURCES/0080-feat-linstor-manager-add-new-healthCheck-function-to.patch b/SOURCES/0080-feat-linstor-manager-add-new-healthCheck-function-to.patch new file mode 100644 index 0000000..de206c3 --- /dev/null +++ b/SOURCES/0080-feat-linstor-manager-add-new-healthCheck-function-to.patch @@ -0,0 +1,358 @@ +From 488877ba4b8602148c23e3238a7530e67ad5ea8f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 23 Sep 2022 17:45:08 +0200 +Subject: [PATCH 080/177] feat(linstor-manager): add new `healthCheck` function + to monitor pool (#26) + +Print a JSON output to monitor state of LINSTOR SRs: + - Display nodes, storage pool and resources + - Print human readable warns and errors + +Usage example: + +``` +xe host-call-plugin host-uuid=c96ec4dd-28ac-4df4-b73c-4371bd202728 plugin=linstor-manager fn=healthCheck args:groupName=linstor_group +{ + "errors": [], + "warns": [], + "controller-uri": "linstor://172.16.210.14", + "storage-pools": { + "r620-s1": [ + { + "free-size": 999125155840, + "storage-pool-name": "xcp-sr-linstor_group", + "capacity": 1000203091968, + "uuid": "994a5c45-ba52-4f17-8e46-74c7dec0a1e7" + } + ], + "r620-s3": [ + { + "free-size": 999125155840, + "storage-pool-name": "xcp-sr-linstor_group", + "capacity": 1000203091968, + "uuid": "ad78adad-a9f6-4513-9f96-e8eb8fe716dc" + } + ], + "r620-s2": [ + { + "free-size": 999125155840, + "storage-pool-name": "xcp-sr-linstor_group", + "capacity": 1000203091968, + "uuid": "f76048f9-8821-484b-9a51-670a49df7a6e" + } + ] + }, + "nodes": { + "r620-s1": "ONLINE", + "r620-s3": "ONLINE", + "r620-s2": "ONLINE" + }, + "resources": { + "xcp-persistent-database": { + "r620-s1": { + "tie-breaker": false, + "in-use": true, + "volumes": [ + { + "storage-pool-name": "xcp-sr-linstor_group", + "uuid": "1a436f23-eb81-4a8f-8ab6-de317282b5d5", + "device-path": "/dev/drbd1000", + "number": 0, + "disk-state": "UpToDate", + "allocated-size": 1077936128, + "usable-size": 1073741824 + } + ], + "diskful": true + }, + "r620-s3": { + "tie-breaker": false, + "in-use": false, + "volumes": [ + { + "storage-pool-name": "xcp-sr-linstor_group", + "uuid": "31a05bd1-20b6-471a-86b9-bbcdccfaab96", + "device-path": "/dev/drbd1000", + "number": 0, + "disk-state": "UpToDate", + "allocated-size": 1077936128, + "usable-size": 1073741824 + } + ], + "diskful": true + }, + "r620-s2": { + "tie-breaker": false, + "in-use": false, + "volumes": [ + { + "storage-pool-name": "xcp-sr-linstor_group", + "uuid": "0420f252-9762-4063-bdd4-732e40373ffb", + "device-path": "/dev/drbd1000", + "number": 0, + "disk-state": "UpToDate", + "allocated-size": 1077936128, + "usable-size": 1073741824 + } + ], + "diskful": true + } + } + } +} +``` + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 116 +++++++++++++++++++++++++++++++- + drivers/linstorvolumemanager.py | 105 +++++++++++++++++++++++++++++ + 2 files changed, 220 insertions(+), 1 deletion(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 5485b900..7abc1054 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -848,6 +848,119 @@ def get_drbd_openers(session, args): + raise + + ++def health_check(session, args): ++ group_name = args['groupName'] ++ ++ result = { ++ 'controller-uri': '', ++ 'nodes': {}, ++ 'storage-pools': {}, ++ 'warnings': [], ++ 'errors': [] ++ } ++ ++ def format_result(): ++ return json.dumps(result) ++ ++ # 1. Get controller. ++ try: ++ controller_uri = get_controller_uri() ++ ++ result['controller-uri'] = controller_uri ++ try: ++ if controller_uri == 'linstor://localhost': ++ # Replace `localhost` with IP to give a better info for users. ++ result['controller-uri'] = 'linstor://' + util.get_this_host_address(session) ++ except Exception: ++ # Ignore error: can be a XAPI restart or something else. ++ pass ++ ++ linstor = LinstorVolumeManager( ++ controller_uri, ++ group_name, ++ logger=util.SMlog ++ ) ++ except Exception as e: ++ # Probably a network issue, or offline controller. ++ result['errors'].append('Cannot join SR: `{}`.'.format(e)) ++ return format_result() ++ ++ try: ++ # 2. Check node statuses. ++ nodes = linstor.get_nodes_info() ++ result['nodes'] = nodes ++ for node_name, status in nodes.items(): ++ if status != 'ONLINE': ++ result['warnings'].append('Node `{}` is {}.'.format(node_name, status)) ++ ++ # 3. Check storage pool statuses. ++ storage_pools_per_node = linstor.get_storage_pools_info() ++ result['storage-pools'] = storage_pools_per_node ++ for node_name, storage_pools in storage_pools_per_node.items(): ++ for storage_pool in storage_pools: ++ free_size = storage_pool['free-size'] ++ capacity = storage_pool['capacity'] ++ if free_size < 0 or capacity <= 0: ++ result['errors'].append( ++ 'Cannot get free size and/or capacity of storage pool `{}`.' ++ .format(storage_pool['uuid']) ++ ) ++ elif free_size > capacity: ++ result['errors'].append( ++ 'Free size of storage pool `{}` is greater than capacity.' ++ .format(storage_pool['uuid']) ++ ) ++ else: ++ remaining_percent = free_size / float(capacity) * 100.0 ++ threshold = 10.0 ++ if remaining_percent < threshold: ++ result['warnings'].append( ++ 'Remaining size of storage pool `{}` is below {}% of its capacity.' ++ .format(storage_pool['uuid'], threshold) ++ ) ++ ++ # 4. Check resource statuses. ++ all_resources = linstor.get_resources_info() ++ result['resources'] = all_resources ++ ++ for resource_name, resource_by_node in all_resources.items(): ++ for node_name, resource in resource_by_node.items(): ++ for volume_index, volume in enumerate(resource['volumes']): ++ disk_state = volume['disk-state'] ++ if disk_state in ['UpToDate', 'Created', 'Attached']: ++ continue ++ if disk_state == 'DUnknown': ++ result['warnings'].append( ++ 'Unknown state for volume `{}` at index {} for resource `{}` on node `{}`' ++ .format(volume['device-path'], volume_index, resource_name, node_name) ++ ) ++ continue ++ if disk_state in ['Inconsistent', 'Failed', 'To: Creating', 'To: Attachable', 'To: Attaching']: ++ result['errors'].append( ++ 'Invalid state `{}` for volume `{}` at index {} for resource `{}` on node `{}`' ++ .format(disk_state, volume['device-path'], volume_index, resource_name, node_name) ++ ) ++ continue ++ if disk_state == 'Diskless': ++ if resource['diskful']: ++ result['errors'].append( ++ 'Unintentional diskless state detected for volume `{}` at index {} for resource `{}` on node `{}`' ++ .format(volume['device-path'], volume_index, resource_name, node_name) ++ ) ++ elif resource['tie-breaker']: ++ volume['disk-state'] = 'TieBreaker' ++ continue ++ result['warnings'].append( ++ 'Unhandled state `{}` for volume `{}` at index {} for resource `{}` on node `{}`' ++ .format(disk_state, volume['device-path'], volume_index, resource_name, node_name) ++ ) ++ ++ except Exception as e: ++ result['errors'].append('Unexpected error: `{}`'.format(e)) ++ ++ return format_result() ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -887,5 +1000,6 @@ if __name__ == '__main__': + 'listDrbdVolumes': list_drbd_volumes, + 'destroyDrbdVolume': destroy_drbd_volume, + 'destroyDrbdVolumes': destroy_drbd_volumes, +- 'getDrbdOpeners': get_drbd_openers ++ 'getDrbdOpeners': get_drbd_openers, ++ 'healthCheck': health_check + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 3ee5d248..efe5d53b 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1402,6 +1402,111 @@ class LinstorVolumeManager(object): + 'Failed to destroy node `{}`: {}'.format(node_name, error_str) + ) + ++ def get_nodes_info(self): ++ """ ++ Get all nodes + statuses, used or not by the pool. ++ :rtype: dict(str, dict) ++ """ ++ try: ++ nodes = {} ++ for node in self._linstor.node_list_raise().nodes: ++ nodes[node.name] = node.connection_status ++ return nodes ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to get all nodes: `{}`'.format(e) ++ ) ++ ++ def get_storage_pools_info(self): ++ """ ++ Give all storage pools of current group name. ++ :rtype: dict(str, list) ++ """ ++ storage_pools = {} ++ for pool in self._get_storage_pools(force=True): ++ if pool.node_name not in storage_pools: ++ storage_pools[pool.node_name] = [] ++ ++ size = -1 ++ capacity = -1 ++ ++ space = pool.free_space ++ if space: ++ size = space.free_capacity ++ if size < 0: ++ size = -1 ++ else: ++ size *= 1024 ++ capacity = space.total_capacity ++ if capacity <= 0: ++ capacity = -1 ++ else: ++ capacity *= 1024 ++ ++ storage_pools[pool.node_name].append({ ++ 'storage-pool-name': pool.name, ++ 'uuid': pool.uuid, ++ 'free-size': size, ++ 'capacity': capacity ++ }) ++ ++ return storage_pools ++ ++ def get_resources_info(self): ++ """ ++ Give all resources of current group name. ++ :rtype: dict(str, list) ++ """ ++ resources = {} ++ resource_list = self._linstor.resource_list_raise() ++ for resource in resource_list.resources: ++ if resource.name not in resources: ++ resources[resource.name] = {} ++ ++ resources[resource.name][resource.node_name] = { ++ 'volumes': [], ++ 'diskful': linstor.consts.FLAG_DISKLESS not in resource.flags, ++ 'tie-breaker': linstor.consts.FLAG_TIE_BREAKER in resource.flags ++ } ++ ++ for volume in resource.volumes: ++ # We ignore diskless pools of the form "DfltDisklessStorPool". ++ if volume.storage_pool_name != self._group_name: ++ continue ++ ++ usable_size = volume.usable_size ++ if usable_size < 0: ++ usable_size = -1 ++ else: ++ usable_size *= 1024 ++ ++ allocated_size = volume.allocated_size ++ if allocated_size < 0: ++ allocated_size = -1 ++ else: ++ allocated_size *= 1024 ++ ++ resources[resource.name][resource.node_name]['volumes'].append({ ++ 'storage-pool-name': volume.storage_pool_name, ++ 'uuid': volume.uuid, ++ 'number': volume.number, ++ 'device-path': volume.device_path, ++ 'usable-size': usable_size, ++ 'allocated-size': allocated_size ++ }) ++ ++ for resource_state in resource_list.resource_states: ++ resource = resources[resource_state.rsc_name][resource_state.node_name] ++ resource['in-use'] = resource_state.in_use ++ ++ volumes = resource['volumes'] ++ for volume_state in resource_state.volume_states: ++ volume = next((x for x in volumes if x['number'] == volume_state.number), None) ++ if volume: ++ volume['disk-state'] = volume_state.disk_state ++ ++ return resources ++ + @classmethod + def create_sr( + cls, group_name, node_names, ips, redundancy, diff --git a/SOURCES/0081-fix-LinstorSR-fix-xha-conf-parsing-return-host-ip-no.patch b/SOURCES/0081-fix-LinstorSR-fix-xha-conf-parsing-return-host-ip-no.patch new file mode 100644 index 0000000..b0adf8b --- /dev/null +++ b/SOURCES/0081-fix-LinstorSR-fix-xha-conf-parsing-return-host-ip-no.patch @@ -0,0 +1,51 @@ +From a5be376a1404d031c41abd728e758f8d296a09db Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 4 Oct 2022 11:01:33 +0200 +Subject: [PATCH 081/177] fix(LinstorSR): fix xha conf parsing => return host + ip, not the UUID + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 374d6cb9..d32771fe 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -277,7 +277,7 @@ def get_ips_from_xha_config_file(): + return IPS_XHA_CACHE + + ips = dict() +- host_ip = None ++ host_id = None + try: + # Ensure there is no dirty read problem. + # For example if the HA is reloaded. +@@ -287,7 +287,7 @@ def get_ips_from_xha_config_file(): + period=1 + ) + except: +- return (host_ip, ips) ++ return (None, ips) + + def parse_host_nodes(ips, node): + current_id = None +@@ -322,14 +322,14 @@ def get_ips_from_xha_config_file(): + if node.tag == 'common-config': + parse_common_config(ips, node) + elif node.tag == 'local-config': +- host_ip = parse_local_config(ips, node) ++ host_id = parse_local_config(ips, node) + else: + continue + +- if ips and host_ip: ++ if ips and host_id: + break + +- return (host_ip, ips) ++ return (host_id and ips.get(host_id), ips) + + + def activate_lvm_group(group_name): diff --git a/SOURCES/0082-fix-LinstorSR-start-correctly-HA-servers-HTTP-NBD-af.patch b/SOURCES/0082-fix-LinstorSR-start-correctly-HA-servers-HTTP-NBD-af.patch new file mode 100644 index 0000000..fa2cd7b --- /dev/null +++ b/SOURCES/0082-fix-LinstorSR-start-correctly-HA-servers-HTTP-NBD-af.patch @@ -0,0 +1,41 @@ +From 65d5ff02da3d1c084573924a82735da84f1ec5c4 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 4 Oct 2022 18:48:09 +0200 +Subject: [PATCH 082/177] fix(LinstorSR): start correctly HA servers (HTTP/NBD) + after reboot + +Use a timeout call after a reboot to get a XAPI session because +it can be impossible to initialize the connection at startup +or if the current host has been ejected (in this case the call can +block indefinitely). + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 7 +++++-- + 1 file changed, 5 insertions(+), 2 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index d32771fe..ae253858 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -2513,7 +2513,10 @@ class LinstorVDI(VDI.VDI): + port = '8077' + + try: +- session = util.get_localAPI_session() ++ # Use a timeout call because XAPI may be unusable on startup ++ # or if the host has been ejected. So in this case the call can ++ # block indefinitely. ++ session = util.timeout_call(5, util.get_localAPI_session) + host_ip = util.get_this_host_address(session) + except: + # Fallback using the XHA file if session not available. +@@ -2583,7 +2586,7 @@ class LinstorVDI(VDI.VDI): + port = '8077' + + try: +- session = util.get_localAPI_session() ++ session = util.timeout_call(5, util.get_localAPI_session) + ips = util.get_host_addresses(session) + except Exception as e: + _, ips = get_ips_from_xha_config_file() diff --git a/SOURCES/0083-fix-linstorvolumemanager-use-an-array-to-store-diskf.patch b/SOURCES/0083-fix-linstorvolumemanager-use-an-array-to-store-diskf.patch new file mode 100644 index 0000000..177326d --- /dev/null +++ b/SOURCES/0083-fix-linstorvolumemanager-use-an-array-to-store-diskf.patch @@ -0,0 +1,75 @@ +From 90cd3fd85ce5c73b4727a579acda75721252963a Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 5 Oct 2022 10:45:50 +0200 +Subject: [PATCH 083/177] fix(linstorvolumemanager): use an array to store + diskful volumes info + +Otherwise the `is_diskful` attr only reflects the info of one host +after a call to `get_volume_info`... And we therefore lose this +information concerning the others. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 4 +++- + drivers/linstorvolumemanager.py | 9 +++++---- + 2 files changed, 8 insertions(+), 5 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index ae253858..b8558077 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -39,6 +39,7 @@ import os + import re + import scsiutil + import signal ++import socket + import SR + import SRCommand + import subprocess +@@ -2729,7 +2730,8 @@ class LinstorVDI(VDI.VDI): + .format(self.uuid, e) + ) + +- must_get_device_path = volume_info.is_diskful ++ hostname = socket.gethostname() ++ must_get_device_path = hostname in volume_info.diskful + + drbd_path = None + if must_get_device_path or self.sr._is_master: +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index efe5d53b..e577f63c 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -307,19 +307,19 @@ class LinstorVolumeManager(object): + 'allocated_size', # Allocated size, place count is not used. + 'virtual_size', # Total virtual available size of this volume + # (i.e. the user size at creation). +- 'is_diskful' ++ 'diskful' # Array of nodes that have a diskful volume. + ) + + def __init__(self, name): + self.name = name + self.allocated_size = 0 + self.virtual_size = 0 +- self.is_diskful = False ++ self.diskful = [] + + def __repr__(self): + return 'VolumeInfo("{}", {}, {}, {})'.format( + self.name, self.allocated_size, self.virtual_size, +- 'diskful' if self.is_diskful else 'diskless' ++ self.diskful + ) + + # -------------------------------------------------------------------------- +@@ -1878,7 +1878,8 @@ class LinstorVolumeManager(object): + else: + current = all_volume_info[resource.name] + +- current.is_diskful = linstor.consts.FLAG_DISKLESS not in resource.flags ++ if linstor.consts.FLAG_DISKLESS not in resource.flags: ++ current.diskful.append(resource.node_name) + + for volume in resource.volumes: + # We ignore diskless pools of the form "DfltDisklessStorPool". diff --git a/SOURCES/0084-feat-linstorvolumemanager-support-snaps-when-a-host-.patch b/SOURCES/0084-feat-linstorvolumemanager-support-snaps-when-a-host-.patch new file mode 100644 index 0000000..bc46485 --- /dev/null +++ b/SOURCES/0084-feat-linstorvolumemanager-support-snaps-when-a-host-.patch @@ -0,0 +1,30 @@ +From 8e4f41d2b6caa3830f1073e3ab463657e9bb0d67 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 6 Oct 2022 17:54:10 +0200 +Subject: [PATCH 084/177] feat(linstorvolumemanager): support snaps when a host + is offline + +- Don't create diskless volumes during clone, delay it. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 6 ------ + 1 file changed, 6 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index e577f63c..09aad42f 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1199,12 +1199,6 @@ class LinstorVolumeManager(object): + rsc_name=clone_volume_name, + storage_pool=self._group_name + )) +- for node_name in diskless_node_names: +- resources.append(linstor.ResourceData( +- node_name=node_name, +- rsc_name=clone_volume_name, +- diskless=True +- )) + + # 5. Create resources! + def clean(): diff --git a/SOURCES/0085-fix-linstorvolumemanager-support-offline-hosts-when-.patch b/SOURCES/0085-fix-linstorvolumemanager-support-offline-hosts-when-.patch new file mode 100644 index 0000000..a32829a --- /dev/null +++ b/SOURCES/0085-fix-linstorvolumemanager-support-offline-hosts-when-.patch @@ -0,0 +1,112 @@ +From 53755a3d8e40a2493bc006139e9631ed7c3ae531 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 7 Oct 2022 17:18:37 +0200 +Subject: [PATCH 085/177] fix(linstorvolumemanager): support offline hosts when + plugins are called + +- Robustify plugin calls +- Add a timeout on session getter + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 69 +++++++++++++++++++++++++-------- + 1 file changed, 52 insertions(+), 17 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 09aad42f..58c02382 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -83,15 +83,30 @@ def get_all_volume_openers(resource_name, volume): + volume = str(volume) + openers = {} + +- session = util.get_localAPI_session() ++ # Make sure this call never stucks because this function can be called ++ # during HA init and in this case we can wait forever. ++ session = util.timeout_call(10, util.get_localAPI_session) ++ + hosts = session.xenapi.host.get_all_records() + for host_ref, host_record in hosts.items(): +- openers[host_record['hostname']] = json.loads( +- session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, { +- 'resourceName': resource_name, +- 'volume': volume +- }) +- ) ++ node_name = host_record['hostname'] ++ try: ++ if not session.xenapi.host_metrics.get_record( ++ host_record['metrics'] ++ )['live']: ++ # Ensure we call plugin on online hosts only. ++ continue ++ ++ openers[node_name] = json.loads( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, { ++ 'resourceName': resource_name, ++ 'volume': volume ++ }) ++ ) ++ except Exception as e: ++ util.SMlog('Failed to get openers of `{}` on `{}`: {}'.format( ++ resource_name, node_name, e ++ )) + + return openers + +@@ -162,12 +177,20 @@ def _get_controller_uri(): + # replicated volume. `drbdadm status xcp-persistent-database` returns + # 3 connections by default. + try: +- session = util.get_localAPI_session() ++ session = util.timeout_call(10, util.get_localAPI_session) ++ + for host_ref, host_record in session.xenapi.host.get_all_records().items(): +- if distutils.util.strtobool( +- session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) +- ): +- return 'linstor://' + host_record['address'] ++ node_name = host_record['hostname'] ++ try: ++ if distutils.util.strtobool( ++ session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) ++ ): ++ return 'linstor://' + host_record['address'] ++ except Exception as e: ++ # Can throw and exception if a host is offline. So catch it. ++ util.SMlog('Unable to search controller on `{}`: {}'.format( ++ node_name, e ++ )) + except: + # Not found, maybe we are trying to create the SR... + pass +@@ -200,12 +223,24 @@ def get_controller_node_name(): + if res: + return res.groups()[0] + +- session = util.get_localAPI_session() ++ session = util.timeout_call(5, util.get_localAPI_session) ++ + for host_ref, host_record in session.xenapi.host.get_all_records().items(): +- if distutils.util.strtobool( +- session.xenapi.host.call_plugin(host_ref, PLUGIN, PLUGIN_CMD, {}) +- ): +- return host_record['hostname'] ++ node_name = host_record['hostname'] ++ try: ++ if not session.xenapi.host_metrics.get_record( ++ host_record['metrics'] ++ )['live']: ++ continue ++ ++ if distutils.util.strtobool(session.xenapi.host.call_plugin( ++ host_ref, PLUGIN, PLUGIN_CMD, {} ++ )): ++ return node_name ++ except Exception as e: ++ util.SMlog('Failed to call plugin to get controller on `{}`: {}'.format( ++ node_name, e ++ )) + + # ============================================================================== + diff --git a/SOURCES/0086-fix-linstorvolumemanager-define-_base_group_name-mem.patch b/SOURCES/0086-fix-linstorvolumemanager-define-_base_group_name-mem.patch new file mode 100644 index 0000000..f7fa187 --- /dev/null +++ b/SOURCES/0086-fix-linstorvolumemanager-define-_base_group_name-mem.patch @@ -0,0 +1,31 @@ +From 61f2aa4e8c0ee7ed0a7bb514a8c5d6e7d8b1e789 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 7 Oct 2022 17:45:26 +0200 +Subject: [PATCH 086/177] fix(linstorvolumemanager): define _base_group_name + member at SR creation + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 58c02382..d19effbe 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1622,6 +1622,7 @@ class LinstorVolumeManager(object): + ) + + driver_pool_name = group_name ++ base_group_name = group_name + group_name = cls._build_group_name(group_name) + pools = lin.storage_pool_list_raise(filter_by_stor_pools=[group_name]) + pools = pools.storage_pools +@@ -1784,6 +1785,7 @@ class LinstorVolumeManager(object): + instance._linstor = lin + instance._logger = logger + instance._redundancy = redundancy ++ instance._base_group_name = base_group_name + instance._group_name = group_name + instance._volumes = set() + instance._storage_pools_time = 0 diff --git a/SOURCES/0087-feat-linstorvhdutil-modify-logic-of-local-vhdutil-ca.patch b/SOURCES/0087-feat-linstorvhdutil-modify-logic-of-local-vhdutil-ca.patch new file mode 100644 index 0000000..19f1160 --- /dev/null +++ b/SOURCES/0087-feat-linstorvhdutil-modify-logic-of-local-vhdutil-ca.patch @@ -0,0 +1,145 @@ +From f42a4f663b05544178d929d46740a16abc5d6ca9 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 10 Oct 2022 14:33:24 +0200 +Subject: [PATCH 087/177] feat(linstorvhdutil): modify logic of local vhdutil + calls + +- Always log openers when we can't call vhdutil locally +- Remove "raise" lines after a local fail, always retry command calls on a remote + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvhdutil.py | 72 ++++++++++++++++++++++++--------------- + 1 file changed, 44 insertions(+), 28 deletions(-) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 4d031e12..2687cadf 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -65,12 +65,17 @@ def linstorhostcall(local_method, remote_method): + (node_names, in_use) = \ + self._linstor.find_up_to_date_diskful_nodes(vdi_uuid) + ++ local_e = None + try: + if not in_use or socket.gethostname() in node_names: +- return self._call_local_vhd_util(local_method, device_path, *args[2:], **kwargs) +- except util.CommandException as e: +- if e.code != errno.EROFS and e.code != EMEDIUMTYPE: +- raise ++ # Don't call `_call_local_vhd_util`, we don't want to ++ # trace failed calls now using opener files. It can be ++ # normal to have an exception. ++ def local_call(): ++ return local_method(device_path, *args[2:], **kwargs) ++ return util.retry(local_call, 5, 2) ++ except util.CommandException as local_e: ++ self._handle_local_vhd_util_error(local_e) + + # B. Execute the plugin on master or slave. + remote_args = { +@@ -80,12 +85,13 @@ def linstorhostcall(local_method, remote_method): + remote_args.update(**kwargs) + remote_args = {str(key): str(value) for key, value in remote_args.iteritems()} + +- def remote_call(): +- host_ref = self._get_readonly_host( +- vdi_uuid, device_path, node_names +- ) +- return call_vhd_util_on_host(self._session, host_ref, remote_method, device_path, remote_args) +- response = util.retry(remote_call, 5, 2) ++ try: ++ def remote_call(): ++ host_ref = self._get_readonly_host(vdi_uuid, device_path, node_names) ++ return call_vhd_util_on_host(self._session, host_ref, remote_method, device_path, remote_args) ++ response = util.retry(remote_call, 5, 2) ++ except Exception as remote_e: ++ self._raise_openers_exception(device_path, local_e or remote_e) + + return response_parser(self, vdi_uuid, response) + return wrapper +@@ -271,22 +277,13 @@ class LinstorVhdUtil: + + # -------------------------------------------------------------------------- + +- def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): +- try: +- def local_call(): +- return local_method(device_path, *args, **kwargs) +- return util.retry(local_call, 5, 2) +- except util.CommandException as e: +- if e.code != errno.EROFS and e.code != EMEDIUMTYPE: +- raise +- +- # Volume is locked on a host, find openers. ++ def _raise_openers_exception(self, device_path, e): + e_with_openers = None + try: + volume_uuid = self._linstor.get_volume_uuid_from_device_path( + device_path + ) +- e_with_openers = util.CommandException( ++ e_wrapper = util.CommandException( + e.code, + e.cmd, + e.reason + ' (openers: {})'.format( +@@ -294,12 +291,29 @@ class LinstorVhdUtil: + ) + ) + except Exception as illformed_e: +- raise util.CommandException( ++ e_wrapper = util.CommandException( + e.code, + e.cmd, + e.reason + ' (unable to get openers: {})'.format(illformed_e) + ) +- raise e_with_openers # pylint: disable = E0702 ++ util.SMlog('raise opener exception: {} ({})'.format(e_wrapper, e_wrapper.reason)) ++ raise e_wrapper # pylint: disable = E0702 ++ ++ @staticmethod ++ def _handle_local_vhd_util_error(e): ++ if e.code != errno.EROFS and e.code != EMEDIUMTYPE: ++ util.SMlog('failed to execute locally vhd-util (sys {})'.format(e.code)) ++ ++ def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): ++ try: ++ def local_call(): ++ return local_method(device_path, *args, **kwargs) ++ return util.retry(local_call, 5, 2) ++ except util.CommandException as e: ++ self._handle_local_vhd_util_error(e) ++ ++ # Volume is locked on a host, find openers. ++ self._raise_openers_exception(device_path, e) + + def _call_vhd_util(self, local_method, remote_method, device_path, *args, **kwargs): + # A. Try to write locally... +@@ -308,8 +322,7 @@ class LinstorVhdUtil: + return local_method(device_path, *args, **kwargs) + return util.retry(local_call, 5, 2) + except util.CommandException as e: +- if e.code != errno.EROFS and e.code != EMEDIUMTYPE: +- raise ++ self._handle_local_vhd_util_error(e) + + # B. Execute the command on another host. + # B.1. Get host list. +@@ -362,11 +375,14 @@ class LinstorVhdUtil: + pass + + if no_host_found: +- return local_method(device_path, *args, **kwargs) ++ try: ++ return local_method(device_path, *args, **kwargs) ++ except Exception as e: ++ self._raise_openers_exception(device_path, e) + + raise xs_errors.XenError( + 'VDIUnavailable', +- opterr='No valid host found to run vhd-util command `{}` (path={}): {}' +- .format(remote_method, device_path, e) ++ opterr='No valid host found to run vhd-util command `{}` (path=`{}`, openers=`{}`): {}' ++ .format(remote_method, device_path, openers, e) + ) + return util.retry(remote_call, 5, 2) diff --git a/SOURCES/0088-fix-linstorvolumemanager-robustify-failed-snapshots.patch b/SOURCES/0088-fix-linstorvolumemanager-robustify-failed-snapshots.patch new file mode 100644 index 0000000..d03ce1c --- /dev/null +++ b/SOURCES/0088-fix-linstorvolumemanager-robustify-failed-snapshots.patch @@ -0,0 +1,50 @@ +From f5a1ae173fe0af8c91f125796f1785b6971d1405 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 17 Oct 2022 18:14:16 +0200 +Subject: [PATCH 088/177] fix(linstorvolumemanager): robustify failed snapshots + +- Ensure we can always rename a failed snap, so we must check if + we have metadata in the KV-store. Otherwise an error is triggered + because we are trying to copy a None object. +- If we can't delete a volume, rename it with a DELETED_ suffix, + it's mandatory in specific cases to rollback snapshot transactions. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 16 +++++++++++++++- + 1 file changed, 15 insertions(+), 1 deletion(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index d19effbe..44b247e8 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -951,7 +951,11 @@ class LinstorVolumeManager(object): + volume_properties[self.PROP_UPDATING_UUID_SRC] = volume_uuid + + # 4. Copy the properties. +- volume_properties[self.PROP_METADATA] = metadata ++ # Note: On new volumes, during clone for example, the metadata ++ # may be missing. So we must test it to avoid this error: ++ # "None has to be a str/unicode, but is " ++ if metadata: ++ volume_properties[self.PROP_METADATA] = metadata + volume_properties[self.PROP_VOLUME_NAME] = volume_name + + # 5. Ok! +@@ -2289,6 +2293,16 @@ class LinstorVolumeManager(object): + 'Cannot clean volume {}: {}'.format(volume_uuid, e) + ) + ++ # The volume can't be removed, maybe it's still in use, ++ # in this case rename it with the "DELETED_" prefix. ++ # This prefix is mandatory if it exists a snap transaction to ++ # rollback because the original VDI UUID can try to be renamed ++ # with the UUID we are trying to delete... ++ if not volume_uuid.startswith('DELETED_'): ++ self.update_volume_uuid( ++ volume_uuid, 'DELETED_' + volume_uuid, force=True ++ ) ++ + for dest_uuid, src_uuid in updating_uuid_volumes.items(): + dest_namespace = self._build_volume_namespace(dest_uuid) + diff --git a/SOURCES/0089-fix-linstorvolumemanager-use-a-namespace-for-volumes.patch b/SOURCES/0089-fix-linstorvolumemanager-use-a-namespace-for-volumes.patch new file mode 100644 index 0000000..0dac5ab --- /dev/null +++ b/SOURCES/0089-fix-linstorvolumemanager-use-a-namespace-for-volumes.patch @@ -0,0 +1,26 @@ +From f7cf38ee5935f024250d13a40be3f2b81eaf6a7e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 8 Nov 2022 17:31:45 +0100 +Subject: [PATCH 089/177] fix(linstorvolumemanager): use a namespace for + volumes + +- This change is not compatible with existing LINSTOR SR instances! + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 44b247e8..8c253d49 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -308,7 +308,7 @@ class LinstorVolumeManager(object): + + # Property namespaces. + NAMESPACE_SR = 'xcp/sr' +- NAMESPACE_VOLUME = 'volume' ++ NAMESPACE_VOLUME = 'xcp/volume' + + # Regex to match properties. + REG_PROP = '^([^/]+)/{}$' diff --git a/SOURCES/0090-feat-linstor-kv-dump-rename-to-linstor-kv-tool-add-r.patch b/SOURCES/0090-feat-linstor-kv-dump-rename-to-linstor-kv-tool-add-r.patch new file mode 100644 index 0000000..154bfee --- /dev/null +++ b/SOURCES/0090-feat-linstor-kv-dump-rename-to-linstor-kv-tool-add-r.patch @@ -0,0 +1,94 @@ +From ef7f0c07eab1c25393611a57162e1290c6b54308 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 14 Nov 2022 17:18:48 +0100 +Subject: [PATCH 090/177] feat(linstor-kv-dump): rename to linstor-kv-tool + + add remove volume helpers + +--- + Makefile | 2 +- + scripts/{linstor-kv-dump => linstor-kv-tool} | 42 +++++++++++++++++++- + 2 files changed, 42 insertions(+), 2 deletions(-) + rename scripts/{linstor-kv-dump => linstor-kv-tool} (51%) + +diff --git a/Makefile b/Makefile +index aa71f809..42058c82 100755 +--- a/Makefile ++++ b/Makefile +@@ -240,7 +240,7 @@ install: precheck + install -m 755 drivers/fcoelib.py $(SM_STAGING)$(SM_DEST) + mkdir -p $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/fork-log-daemon $(SM_STAGING)$(LIBEXEC) +- install -m 755 scripts/linstor-kv-dump $(SM_STAGING)$(BIN_DEST) ++ install -m 755 scripts/linstor-kv-tool $(SM_STAGING)$(BIN_DEST) + install -m 755 scripts/local-device-change $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/check-device-sharing $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/usb_change $(SM_STAGING)$(LIBEXEC) +diff --git a/scripts/linstor-kv-dump b/scripts/linstor-kv-tool +similarity index 51% +rename from scripts/linstor-kv-dump +rename to scripts/linstor-kv-tool +index 93598d7c..128d8992 100755 +--- a/scripts/linstor-kv-dump ++++ b/scripts/linstor-kv-tool +@@ -18,6 +18,7 @@ import argparse + import json + import linstor + ++ + def dump_kv(controller_uri, group_name, namespace): + kv = linstor.KV( + group_name, +@@ -26,13 +27,52 @@ def dump_kv(controller_uri, group_name, namespace): + ) + print(json.dumps(kv, sort_keys=True, indent=2)) + ++ ++def remove_volume(controller_uri, group_name, vdi_name): ++ assert vdi_name ++ kv = linstor.KV( ++ group_name, ++ uri=controller_uri, ++ namespace='/xcp/volume/{}'.format(vdi_name) ++ ) ++ ++ for key, value in list(kv.items()): ++ del kv[key] ++ ++ ++def remove_all_volumes(controller_uri, group_name): ++ kv = linstor.KV( ++ group_name, ++ uri=controller_uri, ++ namespace='/' ++ ) ++ ++ for key, value in list(kv.items()): ++ if key.startswith('xcp/volume/'): ++ size = key.rindex('/') ++ kv.namespace = key[:size] ++ del kv[key[size + 1:]] ++ ++ + def main(): + parser = argparse.ArgumentParser() + parser.add_argument('-u', '--uri', required=True) + parser.add_argument('-g', '--group-name', required=True) + parser.add_argument('-n', '--namespace', default='/') ++ ++ action = parser.add_mutually_exclusive_group(required=True) ++ action.add_argument('--dump-volumes', action='store_true') ++ action.add_argument('--remove-volume', metavar='VDI_UUID') ++ action.add_argument('--remove-all-volumes', action='store_true') ++ + args = parser.parse_args() +- dump_kv(args.uri, args.group_name, args.namespace) ++ if args.dump_volumes: ++ dump_kv(args.uri, args.group_name, args.namespace) ++ elif args.remove_volume: ++ remove_volume(args.uri, args.group_name, args.remove_volume) ++ elif args.remove_all_volumes: ++ remove_all_volumes(args.uri, args.group_name) ++ + + if __name__ == '__main__': + main() diff --git a/SOURCES/0091-fix-LinstorSR-handle-correctly-localhost-during-star.patch b/SOURCES/0091-fix-LinstorSR-handle-correctly-localhost-during-star.patch new file mode 100644 index 0000000..9ed5986 --- /dev/null +++ b/SOURCES/0091-fix-LinstorSR-handle-correctly-localhost-during-star.patch @@ -0,0 +1,72 @@ +From 196dce016c304f57e9e60a60397fc58c992d459f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 16 Nov 2022 12:12:12 +0100 +Subject: [PATCH 091/177] fix(LinstorSR): handle correctly localhost during + start/stop of minidrbdcluster + +Otherwise another controller can be started during `xe sr-destroy` call. +And in this case we can get an exception during the umount/remount of /var/lib/linstor +to destroy the persistent database because a controller uses it. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 31 +++++++++++++++++++++++++++---- + 1 file changed, 27 insertions(+), 4 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index b8558077..ba284524 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -967,25 +967,48 @@ class LinstorSR(SR.SR): + def _update_minidrbdcluster_on_all_hosts( + self, enabled, controller_node_name=None + ): ++ if controller_node_name == 'localhost': ++ controller_node_name = self.session.xenapi.host.get_record( ++ util.get_this_host_ref(self.session) ++ )['hostname'] ++ assert controller_node_name ++ assert controller_node_name != 'localhost' ++ + controller_host = None + secondary_hosts = [] + + hosts = self.session.xenapi.host.get_all_records() + for host_ref, host_rec in hosts.iteritems(): +- if controller_node_name == host_rec['hostname']: ++ hostname = host_rec['hostname'] ++ if controller_node_name == hostname: + controller_host = host_ref + else: +- secondary_hosts.append(host_ref) ++ secondary_hosts.append((host_ref, hostname)) ++ ++ action_name = 'Starting' if enabled else 'Stopping' ++ if controller_node_name and not controller_host: ++ util.SMlog('Failed to find controller host: `{}`'.format( ++ controller_node_name ++ )) + + if enabled and controller_host: ++ util.SMlog('{} minidrbdcluster on controller host `{}`...'.format( ++ action_name, controller_node_name ++ )) + # If enabled is true, we try to start the controller on the desired + # node name first. + self._update_minidrbdcluster(controller_host, enabled) + +- for host in secondary_hosts: +- self._update_minidrbdcluster(host, enabled) ++ for host_ref, hostname in secondary_hosts: ++ util.SMlog('{} minidrbdcluster on host {}...'.format( ++ action_name, hostname ++ )) ++ self._update_minidrbdcluster(host_ref, enabled) + + if not enabled and controller_host: ++ util.SMlog('{} minidrbdcluster on controller host `{}`...'.format( ++ action_name, controller_node_name ++ )) + # If enabled is false, we disable the minidrbdcluster service of + # the controller host last. Why? Otherwise the linstor-controller + # of other nodes can be started, and we don't want that. diff --git a/SOURCES/0092-fix-cleanup.py-call-repair-on-another-host-when-EROF.patch b/SOURCES/0092-fix-cleanup.py-call-repair-on-another-host-when-EROF.patch new file mode 100644 index 0000000..b841295 --- /dev/null +++ b/SOURCES/0092-fix-cleanup.py-call-repair-on-another-host-when-EROF.patch @@ -0,0 +1,59 @@ +From 628668575b779ebb479e4621170b78ae2636b928 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 17 Nov 2022 15:43:25 +0100 +Subject: [PATCH 092/177] fix(cleanup.py): call repair on another host when + EROFS is returned (DRBD) + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 20 ++++++++++++++++++-- + 1 file changed, 18 insertions(+), 2 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 73293632..f6c4346b 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -726,6 +726,12 @@ class VDI(object): + lock.Lock.cleanupAll(self.uuid) + self._clear() + ++ def getParent(self): ++ return vhdutil.getParent(self.path, lambda x: x.strip()) ++ ++ def repair(self, parent): ++ vhdutil.repair(parent) ++ + def __str__(self): + strHidden = "" + if self.hidden: +@@ -876,11 +882,11 @@ class VDI(object): + # Try a repair and reraise the exception + parent = "" + try: +- parent = vhdutil.getParent(self.path, lambda x: x.strip()) ++ parent = self.getParent() + # Repair error is logged and ignored. Error reraised later + util.SMlog('Coalesce failed on %s, attempting repair on ' \ + 'parent %s' % (self.uuid, parent)) +- vhdutil.repair(parent) ++ self.repair(parent) + except Exception, e: + util.SMlog('(error ignored) Failed to repair parent %s ' \ + 'after failed coalesce on %s, err: %s' % +@@ -1442,6 +1448,16 @@ class LinstorVDI(VDI): + def coalesce(self): + self.sr._vhdutil.force_coalesce(self.path) + ++ def getParent(self): ++ return self.sr._vhdutil.get_parent( ++ self.sr._linstor.get_volume_uuid_from_device_path(self.path) ++ ) ++ ++ def repair(self, parent_uuid): ++ self.sr._vhdutil.force_repair( ++ self.sr._linstor.get_device_path(parent_uuid) ++ ) ++ + def _relinkSkip(self): + abortFlag = IPCFlag(self.sr.uuid) + for child in self.children: diff --git a/SOURCES/0093-fix-LinstorSR-avoid-introduction-of-DELETED-volumes.patch b/SOURCES/0093-fix-LinstorSR-avoid-introduction-of-DELETED-volumes.patch new file mode 100644 index 0000000..0fc1210 --- /dev/null +++ b/SOURCES/0093-fix-LinstorSR-avoid-introduction-of-DELETED-volumes.patch @@ -0,0 +1,24 @@ +From a7b89628ae53027de00b1cfe03d6bedc38e02f4a Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 17 Nov 2022 15:46:02 +0100 +Subject: [PATCH 093/177] fix(LinstorSR): avoid introduction of DELETED volumes + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index ba284524..9e5b3cda 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1170,6 +1170,9 @@ class LinstorSR(SR.SR): + ) + continue + ++ if vdi_uuid.startswith('DELETED_'): ++ continue ++ + util.SMlog( + 'Trying to introduce VDI {} as it is present in ' + 'LINSTOR and not in XAPI...' diff --git a/SOURCES/0094-feat-linstor-kv-tool-remove-all-volumes-supports-jou.patch b/SOURCES/0094-feat-linstor-kv-tool-remove-all-volumes-supports-jou.patch new file mode 100644 index 0000000..e6374eb --- /dev/null +++ b/SOURCES/0094-feat-linstor-kv-tool-remove-all-volumes-supports-jou.patch @@ -0,0 +1,27 @@ +From e609d0fca2a6dd7efbadf3d97ff9b0f85549a216 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 18 Nov 2022 10:40:58 +0100 +Subject: [PATCH 094/177] feat(linstor-kv-tool): remove-all-volumes supports + journals now + +Not yet supported for remove-volume, not sure about the consequences +for the moment. + +Signed-off-by: Ronan Abhamon +--- + scripts/linstor-kv-tool | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/scripts/linstor-kv-tool b/scripts/linstor-kv-tool +index 128d8992..c9070270 100755 +--- a/scripts/linstor-kv-tool ++++ b/scripts/linstor-kv-tool +@@ -48,7 +48,7 @@ def remove_all_volumes(controller_uri, group_name): + ) + + for key, value in list(kv.items()): +- if key.startswith('xcp/volume/'): ++ if key.startswith('xcp/volume/') or key.startswith('xcp/sr/journal/'): + size = key.rindex('/') + kv.namespace = key[:size] + del kv[key[size + 1:]] diff --git a/SOURCES/0095-fix-linstorvhdutil-due-to-bad-refactoring-check-call.patch b/SOURCES/0095-fix-linstorvhdutil-due-to-bad-refactoring-check-call.patch new file mode 100644 index 0000000..b86def5 --- /dev/null +++ b/SOURCES/0095-fix-linstorvhdutil-due-to-bad-refactoring-check-call.patch @@ -0,0 +1,30 @@ +From fb05d2668ccc6f24109d5c30611fbfc167e34340 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 23 Nov 2022 15:26:51 +0100 +Subject: [PATCH 095/177] fix(linstorvhdutil): due to bad refactoring, check + call was broken + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvhdutil.py | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 2687cadf..a883ca4d 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -124,11 +124,11 @@ class LinstorVhdUtil: + 'ignoreMissingFooter': str(ignore_missing_footer), + 'fast': str(fast) + } +- return self._check(vdi_uuid, **kwargs) ++ return self._check(vdi_uuid, **kwargs) # pylint: disable = E1123 + + @linstorhostcall(vhdutil.check, 'check') +- def _check(self, vdi_uuid, **kwargs): +- return distutils.util.strtobool(kwargs['response']) ++ def _check(self, vdi_uuid, response): ++ return distutils.util.strtobool(response) + + def get_vhd_info(self, vdi_uuid, include_parent=True): + kwargs = {'includeParent': str(include_parent)} diff --git a/SOURCES/0096-feat-linstorvhdutil-ensure-we-use-VHD-parent-to-find.patch b/SOURCES/0096-feat-linstorvhdutil-ensure-we-use-VHD-parent-to-find.patch new file mode 100644 index 0000000..8bbeed3 --- /dev/null +++ b/SOURCES/0096-feat-linstorvhdutil-ensure-we-use-VHD-parent-to-find.patch @@ -0,0 +1,66 @@ +From a765c54efe219c79f7a40550aadd7311f29757fe Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 23 Nov 2022 15:28:23 +0100 +Subject: [PATCH 096/177] feat(linstorvhdutil): ensure we use VHD parent to + find host where to coalesce + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvhdutil.py | 20 +++++++++++++++----- + 1 file changed, 15 insertions(+), 5 deletions(-) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index a883ca4d..c2e9665f 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -228,15 +228,15 @@ class LinstorVhdUtil: + 'parentPath': str(parentPath), + 'parentRaw': parentRaw + } +- return self._call_vhd_util(vhdutil.setParent, 'setParent', path, **kwargs) ++ return self._call_vhd_util(vhdutil.setParent, 'setParent', path, use_parent=False, **kwargs) + + @linstormodifier() + def force_coalesce(self, path): +- return self._call_vhd_util(vhdutil.coalesce, 'coalesce', path) ++ return self._call_vhd_util(vhdutil.coalesce, 'coalesce', path, use_parent=True) + + @linstormodifier() + def force_repair(self, path): +- return self._call_vhd_util(vhdutil.repair, 'repair', path) ++ return self._call_vhd_util(vhdutil.repair, 'repair', path, use_parent=False) + + # -------------------------------------------------------------------------- + # Helpers. +@@ -315,7 +315,12 @@ class LinstorVhdUtil: + # Volume is locked on a host, find openers. + self._raise_openers_exception(device_path, e) + +- def _call_vhd_util(self, local_method, remote_method, device_path, *args, **kwargs): ++ def _call_vhd_util(self, local_method, remote_method, device_path, use_parent, *args, **kwargs): ++ # Note: `use_parent` exists to know if the VHD parent is used by the local/remote method. ++ # Normally in case of failure, if the parent is unused we try to execute the method on ++ # another host using the DRBD opener list. In the other case, if the parent is required, ++ # we must check where this last one is open instead of the child. ++ + # A. Try to write locally... + try: + def local_call(): +@@ -346,11 +351,16 @@ class LinstorVhdUtil: + volume_uuid = self._linstor.get_volume_uuid_from_device_path( + device_path + ) ++ parent_volume_uuid = None ++ if use_parent: ++ parent_volume_uuid = self.get_parent(volume_uuid) ++ ++ openers_uuid = parent_volume_uuid if use_parent else volume_uuid + + # B.3. Call! + def remote_call(): + try: +- all_openers = self._linstor.get_volume_openers(volume_uuid) ++ all_openers = self._linstor.get_volume_openers(openers_uuid) + except Exception as e: + raise xs_errors.XenError( + 'VDIUnavailable', diff --git a/SOURCES/0097-feat-linstorvolumemanager-force-DRBD-demote-after-fa.patch b/SOURCES/0097-feat-linstorvolumemanager-force-DRBD-demote-after-fa.patch new file mode 100644 index 0000000..5e88dc2 --- /dev/null +++ b/SOURCES/0097-feat-linstorvolumemanager-force-DRBD-demote-after-fa.patch @@ -0,0 +1,236 @@ +From be33740647adac8be43e998a0ce620524f43db24 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 5 Dec 2022 18:40:11 +0100 +Subject: [PATCH 097/177] feat(linstorvolumemanager): force DRBD demote after + failed volume creation/clone + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 13 ++++ + drivers/linstorvolumemanager.py | 101 ++++++++++++++++++++++++-------- + 2 files changed, 91 insertions(+), 23 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 7abc1054..5c4c5c90 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -804,6 +804,18 @@ def create_sr(session, args): + raise + + ++def demote_drbd_resource(session, args): ++ try: ++ resource_name = args['resource_name'] ++ (ret, stdout, stderr) = util.doexec(['drbdsetup', 'secondary', resource_name]) ++ if ret: ++ raise Exception('Failed to demote resource: {}'.format(stderr)) ++ return str(True) ++ except Exception as e: ++ util.SMlog('linstor-manager:demote_drbd_resource error: {}'.format(e)) ++ return str(False) ++ ++ + def list_drbd_volumes(session, args): + try: + volume_group = args.get('volume_group') +@@ -998,6 +1010,7 @@ if __name__ == '__main__': + 'removeHost': remove_host, + 'createSr': create_sr, + 'listDrbdVolumes': list_drbd_volumes, ++ 'demoteDrbdResource': demote_drbd_resource, + 'destroyDrbdVolume': destroy_drbd_volume, + 'destroyDrbdVolumes': destroy_drbd_volumes, + 'getDrbdOpeners': get_drbd_openers, +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 8c253d49..2e2feb23 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -242,6 +242,29 @@ def get_controller_node_name(): + node_name, e + )) + ++ ++def demote_drbd_resource(node_name, resource_name): ++ PLUGIN_CMD = 'demoteDrbdResource' ++ ++ session = util.timeout_call(5, util.get_localAPI_session) ++ ++ for host_ref, host_record in session.xenapi.host.get_all_records().items(): ++ if host_record['hostname'] != node_name: ++ continue ++ ++ try: ++ session.xenapi.host.call_plugin( ++ host_ref, PLUGIN, PLUGIN_CMD, {'resource_name': resource_name} ++ ) ++ except Exception as e: ++ util.SMlog('Failed to demote resource `{}` on `{}`: {}'.format( ++ resource_name, node_name, e ++ )) ++ raise Exception( ++ 'Can\'t demote resource `{}`, unable to find node `{}`' ++ .format(resource_name, node_name) ++ ) ++ + # ============================================================================== + + class LinstorVolumeManagerError(Exception): +@@ -615,6 +638,7 @@ class LinstorVolumeManager(object): + no_diskless=no_diskless + ) + ++ # Volume created! Now try to find the device path. + try: + self._logger( + 'Find device path of LINSTOR volume {}...'.format(volume_uuid) +@@ -627,8 +651,10 @@ class LinstorVolumeManager(object): + 'LINSTOR volume {} created!'.format(volume_uuid) + ) + return device_path +- except Exception: +- self._force_destroy_volume(volume_uuid) ++ except Exception as e: ++ # There is an issue to find the path. ++ # At this point the volume has just been created, so force flag can be used. ++ self._destroy_volume(volume_uuid, force=True) + raise + + def mark_volume_as_persistent(self, volume_uuid): +@@ -1242,7 +1268,7 @@ class LinstorVolumeManager(object): + # 5. Create resources! + def clean(): + try: +- self._destroy_volume(clone_uuid) ++ self._destroy_volume(clone_uuid, force=True) + except Exception as e: + self._logger( + 'Unable to destroy volume {} after shallow clone fail: {}' +@@ -1250,12 +1276,16 @@ class LinstorVolumeManager(object): + ) + + def create(): +- try: +- volume_properties = self._create_volume_with_properties( +- clone_uuid, clone_volume_name, size, +- place_resources=False +- ) ++ # Note: placed outside try/except block because we create only definition first. ++ # There is no reason to call `clean` before the real resource creation. ++ volume_properties = self._create_volume_with_properties( ++ clone_uuid, clone_volume_name, size, ++ place_resources=False ++ ) + ++ # After this point, `clean` can be called for any fail because the clone UUID ++ # is really unique. No risk to remove existing data. ++ try: + result = self._linstor.resource_create(resources) + error_str = self._get_error_str(result) + if error_str: +@@ -1298,6 +1328,7 @@ class LinstorVolumeManager(object): + resource_names = self._fetch_resource_names() + for volume_uuid, volume_name in self.get_volumes_with_name().items(): + if not volume_name or volume_name not in resource_names: ++ # Don't force, we can be sure of what's happening. + self.destroy_volume(volume_uuid) + + def destroy(self): +@@ -2064,7 +2095,7 @@ class LinstorVolumeManager(object): + # B.2. Create volume! + def clean(): + try: +- self._destroy_volume(volume_uuid) ++ self._destroy_volume(volume_uuid, force=True) + except Exception as e: + self._logger( + 'Unable to destroy volume {} after creation fail: {}' +@@ -2136,7 +2167,7 @@ class LinstorVolumeManager(object): + # It can only happen if the same volume uuid is used in the same + # call in another host. + if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: +- self._force_destroy_volume(volume_uuid) ++ self._destroy_volume(volume_uuid, force=True) + raise + + def _find_device_path(self, volume_uuid, volume_name): +@@ -2184,22 +2215,52 @@ class LinstorVolumeManager(object): + # Contains a path of the /dev/drbd form. + return resources[0].volumes[0].device_path + +- def _destroy_resource(self, resource_name): +- self._mark_resource_cache_as_dirty() ++ def _destroy_resource(self, resource_name, force=False): + result = self._linstor.resource_dfn_delete(resource_name) + error_str = self._get_error_str(result) +- if error_str: ++ if not error_str: ++ self._mark_resource_cache_as_dirty() ++ return ++ ++ if not force: ++ self._mark_resource_cache_as_dirty() + raise LinstorVolumeManagerError( +- 'Could not destroy resource `{}` from SR `{}`: {}' ++ 'Could not destroy resource `{}` from SR `{}`: {}' + .format(resource_name, self._group_name, error_str) + ) + +- def _destroy_volume(self, volume_uuid): ++ # If force is used, ensure there is no opener. ++ all_openers = get_all_volume_openers(resource_name, '0') ++ for openers in all_openers.itervalues(): ++ if openers: ++ self._mark_resource_cache_as_dirty() ++ raise LinstorVolumeManagerError( ++ 'Could not force destroy resource `{}` from SR `{}`: {} (openers=`{}`)' ++ .format(resource_name, self._group_name, error_str, all_openers) ++ ) ++ ++ # Maybe the resource is blocked in primary mode. DRBD/LINSTOR issue? ++ resource_states = filter( ++ lambda resource_state: resource_state.name == resource_name, ++ self._get_resource_cache().resource_states ++ ) ++ ++ # Mark only after computation of states. ++ self._mark_resource_cache_as_dirty() ++ ++ for resource_state in resource_states: ++ volume_state = resource_state.volume_states[0] ++ if resource_state.in_use: ++ demote_drbd_resource(resource_state.node_name, resource_name) ++ break ++ self._destroy_resource(resource_name) ++ ++ def _destroy_volume(self, volume_uuid, force=False): + volume_properties = self._get_volume_properties(volume_uuid) + try: + volume_name = volume_properties.get(self.PROP_VOLUME_NAME) + if volume_name in self._fetch_resource_names(): +- self._destroy_resource(volume_name) ++ self._destroy_resource(volume_name, force) + + # Assume this call is atomic. + volume_properties.clear() +@@ -2208,12 +2269,6 @@ class LinstorVolumeManager(object): + 'Cannot destroy volume `{}`: {}'.format(volume_uuid, e) + ) + +- def _force_destroy_volume(self, volume_uuid): +- try: +- self._destroy_volume(volume_uuid) +- except Exception as e: +- self._logger('Ignore fail: {}'.format(e)) +- + def _build_volumes(self, repair): + properties = self._kv_cache + resource_names = self._fetch_resource_names() +@@ -2283,7 +2338,7 @@ class LinstorVolumeManager(object): + # Little optimization, don't call `self._destroy_volume`, + # we already have resource name list. + if volume_name in resource_names: +- self._destroy_resource(volume_name) ++ self._destroy_resource(volume_name, force=True) + + # Assume this call is atomic. + properties.clear() diff --git a/SOURCES/0098-fix-linstorvhdutil-ensure-we-retry-creation-in-all-s.patch b/SOURCES/0098-fix-linstorvhdutil-ensure-we-retry-creation-in-all-s.patch new file mode 100644 index 0000000..7c4420d --- /dev/null +++ b/SOURCES/0098-fix-linstorvhdutil-ensure-we-retry-creation-in-all-s.patch @@ -0,0 +1,118 @@ +From 11cd26e0cd28d9aa622bdc31ace20008c11ab02e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 6 Dec 2022 11:22:15 +0100 +Subject: [PATCH 098/177] fix(linstorvhdutil): ensure we retry creation in all + situations + +Without this patch, a basic resource creation is never restarted +after a failure. The classic situation is when a DRBD error is raised like +`Failed to adjust DRBD resource`. This problem is rare but it exists. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 74 ++++++++++++++++----------------- + 1 file changed, 35 insertions(+), 39 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 2e2feb23..81cce802 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2044,9 +2044,25 @@ class LinstorVolumeManager(object): + self, volume_uuid, volume_name, size, place_resources, + no_diskless=False + ): ++ if no_diskless and not place_resources: ++ raise LinstorVolumeManagerError( ++ 'Could not create volume `{}` from SR `{}`: it\'s impossible ' ++ .format(volume_uuid, self._group_name) + ++ 'to force no diskless without placing resources' ++ ) ++ + size = self.round_up_volume_size(size) + self._mark_resource_cache_as_dirty() + ++ resources = [] ++ if no_diskless: ++ for node_name in self._get_node_names(): ++ resources.append(linstor.ResourceData( ++ node_name=node_name, ++ rsc_name=volume_name, ++ storage_pool=self._group_name ++ )) ++ + def create_definition(): + self._check_volume_creation_errors( + self._linstor.resource_group_spawn( +@@ -2060,39 +2076,6 @@ class LinstorVolumeManager(object): + ) + self._increase_volume_peer_slots(self._linstor, volume_name) + +- # A. Basic case when we use the default redundancy of the group. +- if not no_diskless: +- create_definition() +- if place_resources: +- self._check_volume_creation_errors( +- self._linstor.resource_auto_place( +- rsc_name=volume_name, +- place_count=self._redundancy, +- diskless_on_remaining=not no_diskless +- ), +- volume_uuid, +- self._group_name +- ) +- return +- +- # B. Complex case. +- if not place_resources: +- raise LinstorVolumeManagerError( +- 'Could not create volume `{}` from SR `{}`: it\'s impossible ' +- .format(volume_uuid, self._group_name) + +- 'to force no diskless without placing resources' +- ) +- +- # B.1. Create resource list. +- resources = [] +- for node_name in self._get_node_names(): +- resources.append(linstor.ResourceData( +- node_name=node_name, +- rsc_name=volume_name, +- storage_pool=self._group_name +- )) +- +- # B.2. Create volume! + def clean(): + try: + self._destroy_volume(volume_uuid, force=True) +@@ -2105,13 +2088,26 @@ class LinstorVolumeManager(object): + def create(): + try: + create_definition() +- result = self._linstor.resource_create(resources) +- error_str = self._get_error_str(result) +- if error_str: +- raise LinstorVolumeManagerError( +- 'Could not create volume `{}` from SR `{}`: {}'.format( +- volume_uuid, self._group_name, error_str ++ if no_diskless: ++ # Create a physical resource on each node. ++ result = self._linstor.resource_create(resources) ++ error_str = self._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not create volume `{}` from SR `{}`: {}'.format( ++ volume_uuid, self._group_name, error_str ++ ) + ) ++ elif place_resources: ++ # Basic case when we use the default redundancy of the group. ++ self._check_volume_creation_errors( ++ self._linstor.resource_auto_place( ++ rsc_name=volume_name, ++ place_count=self._redundancy, ++ diskless_on_remaining=not no_diskless ++ ), ++ volume_uuid, ++ self._group_name + ) + except LinstorVolumeManagerError as e: + if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: diff --git a/SOURCES/0099-fix-linstorvhdutil-don-t-retry-local-vhdutil-call-wh.patch b/SOURCES/0099-fix-linstorvhdutil-don-t-retry-local-vhdutil-call-wh.patch new file mode 100644 index 0000000..04815de --- /dev/null +++ b/SOURCES/0099-fix-linstorvhdutil-don-t-retry-local-vhdutil-call-wh.patch @@ -0,0 +1,155 @@ +From 9a688cb7ccdf0d6dacda5f92caff8fe6a058967d Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 7 Dec 2022 17:56:39 +0100 +Subject: [PATCH 099/177] fix(linstorvhdutil): don't retry local vhdutil call + when EROFS is detected + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvhdutil.py | 79 ++++++++++++++++++++++++--------------- + 1 file changed, 48 insertions(+), 31 deletions(-) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index c2e9665f..63d59ab5 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -47,6 +47,14 @@ def call_vhd_util_on_host(session, host_ref, method, device_path, args): + return response + + ++class ErofsLinstorCallException(Exception): ++ def __init__(self, cmd_err): ++ self.cmd_err = cmd_err ++ ++ def __str__(self): ++ return str(self.cmd_err) ++ ++ + def linstorhostcall(local_method, remote_method): + def decorated(response_parser): + def wrapper(*args, **kwargs): +@@ -68,14 +76,17 @@ def linstorhostcall(local_method, remote_method): + local_e = None + try: + if not in_use or socket.gethostname() in node_names: +- # Don't call `_call_local_vhd_util`, we don't want to +- # trace failed calls now using opener files. It can be +- # normal to have an exception. +- def local_call(): +- return local_method(device_path, *args[2:], **kwargs) +- return util.retry(local_call, 5, 2) +- except util.CommandException as local_e: +- self._handle_local_vhd_util_error(local_e) ++ return self._call_local_vhd_util(local_method, device_path, *args[2:], **kwargs) ++ except ErofsLinstorCallException as e: ++ local_e = e.cmd_err ++ except Exception as e: ++ local_e = e ++ ++ util.SMlog( ++ 'unable to execute `{}` locally, retry using a readable host... (cause: {})'.format( ++ remote_method, local_e if local_e else 'local diskless + in use or not up to date' ++ ) ++ ) + + # B. Execute the plugin on master or slave. + remote_args = { +@@ -188,35 +199,35 @@ class LinstorVhdUtil: + + @linstormodifier() + def create(self, path, size, static, msize=0): +- return self._call_local_vhd_util(vhdutil.create, path, size, static, msize) ++ return self._call_local_vhd_util_or_fail(vhdutil.create, path, size, static, msize) + + @linstormodifier() + def set_size_virt_fast(self, path, size): +- return self._call_local_vhd_util(vhdutil.setSizeVirtFast, path, size) ++ return self._call_local_vhd_util_or_fail(vhdutil.setSizeVirtFast, path, size) + + @linstormodifier() + def set_size_phys(self, path, size, debug=True): +- return self._call_local_vhd_util(vhdutil.setSizePhys, path, size, debug) ++ return self._call_local_vhd_util_or_fail(vhdutil.setSizePhys, path, size, debug) + + @linstormodifier() + def set_parent(self, path, parentPath, parentRaw=False): +- return self._call_local_vhd_util(vhdutil.setParent, path, parentPath, parentRaw) ++ return self._call_local_vhd_util_or_fail(vhdutil.setParent, path, parentPath, parentRaw) + + @linstormodifier() + def set_hidden(self, path, hidden=True): +- return self._call_local_vhd_util(vhdutil.setHidden, path, hidden) ++ return self._call_local_vhd_util_or_fail(vhdutil.setHidden, path, hidden) + + @linstormodifier() + def set_key(self, path, key_hash): +- return self._call_local_vhd_util(vhdutil.setKey, path, key_hash) ++ return self._call_local_vhd_util_or_fail(vhdutil.setKey, path, key_hash) + + @linstormodifier() + def kill_data(self, path): +- return self._call_local_vhd_util(vhdutil.killData, path) ++ return self._call_local_vhd_util_or_fail(vhdutil.killData, path) + + @linstormodifier() + def snapshot(self, path, parent, parentRaw, msize=0, checkEmpty=True): +- return self._call_local_vhd_util(vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) ++ return self._call_local_vhd_util_or_fail(vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) + + # -------------------------------------------------------------------------- + # Remote setters: write locally and try on another host in case of failure. +@@ -299,21 +310,27 @@ class LinstorVhdUtil: + util.SMlog('raise opener exception: {} ({})'.format(e_wrapper, e_wrapper.reason)) + raise e_wrapper # pylint: disable = E0702 + +- @staticmethod +- def _handle_local_vhd_util_error(e): +- if e.code != errno.EROFS and e.code != EMEDIUMTYPE: +- util.SMlog('failed to execute locally vhd-util (sys {})'.format(e.code)) +- + def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): + try: + def local_call(): +- return local_method(device_path, *args, **kwargs) +- return util.retry(local_call, 5, 2) ++ try: ++ return local_method(device_path, *args, **kwargs) ++ except util.CommandException as e: ++ if e.code == errno.EROFS or e.code == EMEDIUMTYPE: ++ raise ErofsLinstorCallException(e) # Break retry calls. ++ raise e ++ # Retry only locally if it's not an EROFS exception. ++ return util.retry(local_call, 5, 2, exceptions=[util.CommandException]) + except util.CommandException as e: +- self._handle_local_vhd_util_error(e) ++ util.SMlog('failed to execute locally vhd-util (sys {})'.format(e.code)) ++ raise e + +- # Volume is locked on a host, find openers. +- self._raise_openers_exception(device_path, e) ++ def _call_local_vhd_util_or_fail(self, local_method, device_path, *args, **kwargs): ++ try: ++ return self._call_local_vhd_util(local_method, device_path, *args, **kwargs) ++ except ErofsLinstorCallException as e: ++ # Volume is locked on a host, find openers. ++ self._raise_openers_exception(device_path, e) + + def _call_vhd_util(self, local_method, remote_method, device_path, use_parent, *args, **kwargs): + # Note: `use_parent` exists to know if the VHD parent is used by the local/remote method. +@@ -323,11 +340,11 @@ class LinstorVhdUtil: + + # A. Try to write locally... + try: +- def local_call(): +- return local_method(device_path, *args, **kwargs) +- return util.retry(local_call, 5, 2) +- except util.CommandException as e: +- self._handle_local_vhd_util_error(e) ++ return self._call_local_vhd_util(local_method, device_path, *args, **kwargs) ++ except Exception: ++ pass ++ ++ util.SMlog('unable to execute `{}` locally, retry using a writable host...'.format(remote_method)) + + # B. Execute the command on another host. + # B.1. Get host list. diff --git a/SOURCES/0100-feat-fork-log-daemon-ignore-SIGTERM.patch b/SOURCES/0100-feat-fork-log-daemon-ignore-SIGTERM.patch new file mode 100644 index 0000000..88f65fe --- /dev/null +++ b/SOURCES/0100-feat-fork-log-daemon-ignore-SIGTERM.patch @@ -0,0 +1,32 @@ +From 3ff1971e28327c89e72f1df2d952339aed48ebeb Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 15 Dec 2022 14:36:04 +0100 +Subject: [PATCH 100/177] feat(fork-log-daemon): ignore SIGTERM + +Without this patch, the output logs of the fork-log-daemon child +are never displayed when SIGTERM is sent to the PGID. + +Signed-off-by: Ronan Abhamon +--- + scripts/fork-log-daemon | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/scripts/fork-log-daemon b/scripts/fork-log-daemon +index eb0f0b0f..665a60ba 100755 +--- a/scripts/fork-log-daemon ++++ b/scripts/fork-log-daemon +@@ -1,12 +1,14 @@ + #!/usr/bin/env python + + import select ++import signal + import subprocess + import sys + import syslog + + def main(): + process = subprocess.Popen(sys.argv[1:], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) ++ signal.signal(signal.SIGTERM, signal.SIG_IGN) + write_to_stdout = True + + while process.poll() is None: diff --git a/SOURCES/0101-feat-LinstorSR-wait-for-http-disk-server-startup.patch b/SOURCES/0101-feat-LinstorSR-wait-for-http-disk-server-startup.patch new file mode 100644 index 0000000..56dc35a --- /dev/null +++ b/SOURCES/0101-feat-LinstorSR-wait-for-http-disk-server-startup.patch @@ -0,0 +1,113 @@ +From 2023aa15d2bbd899eb6af814584b88a3deb3f6bf Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 16 Dec 2022 16:52:50 +0100 +Subject: [PATCH 101/177] feat(LinstorSR): wait for http-disk-server startup + +Avoid a race condition with NBD server. +We must be sure the HTTP server is reachable before the NBD server execution, +otherwise the HA activation may fail. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 35 +++++++++++++++++------------------ + 1 file changed, 17 insertions(+), 18 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 9e5b3cda..f336534e 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -2527,13 +2527,10 @@ class LinstorVDI(VDI.VDI): + + @staticmethod + def _start_persistent_http_server(volume_name): +- null = None + pid_path = None + http_server = None + + try: +- null = open(os.devnull, 'w') +- + if volume_name == 'xcp-persistent-ha-statefile': + port = '8076' + else: +@@ -2566,8 +2563,8 @@ class LinstorVDI(VDI.VDI): + util.SMlog('Starting {} on port {}...'.format(arguments[0], port)) + http_server = subprocess.Popen( + [FORK_LOG_DAEMON] + arguments, +- stdout=null, +- stderr=null, ++ stdout=subprocess.PIPE, ++ stderr=subprocess.STDOUT, + # Ensure we use another group id to kill this process without + # touch the current one. + preexec_fn=os.setsid +@@ -2576,6 +2573,17 @@ class LinstorVDI(VDI.VDI): + pid_path = '/run/http-server-{}.pid'.format(volume_name) + with open(pid_path, 'w') as pid_file: + pid_file.write(str(http_server.pid)) ++ ++ def is_ready(): ++ while http_server.poll() is None: ++ if http_server.stdout.readline().rstrip() == 'Server ready!': ++ return True ++ return False ++ try: ++ if not util.timeout_call(10, is_ready): ++ raise Exception('Failed to wait HTTP server startup, bad output') ++ except util.TimeoutException: ++ raise Exception('Failed to wait for HTTP server startup during given delay') + except Exception as e: + if pid_path: + try: +@@ -2594,19 +2602,13 @@ class LinstorVDI(VDI.VDI): + 'VDIUnavailable', + opterr='Failed to start http-server: {}'.format(e) + ) +- finally: +- if null: +- null.close() + + def _start_persistent_nbd_server(self, volume_name): +- null = None + pid_path = None + nbd_path = None + nbd_server = None + + try: +- null = open(os.devnull, 'w') +- + if volume_name == 'xcp-persistent-ha-statefile': + port = '8076' + else: +@@ -2643,6 +2645,10 @@ class LinstorVDI(VDI.VDI): + preexec_fn=os.setsid + ) + ++ pid_path = '/run/nbd-server-{}.pid'.format(volume_name) ++ with open(pid_path, 'w') as pid_file: ++ pid_file.write(str(nbd_server.pid)) ++ + reg_nbd_path = re.compile("^NBD `(/dev/nbd[0-9]+)` is now attached.$") + def get_nbd_path(): + while nbd_server.poll() is None: +@@ -2658,10 +2664,6 @@ class LinstorVDI(VDI.VDI): + except util.TimeoutException: + raise Exception('Unable to read NBD path') + +- pid_path = '/run/nbd-server-{}.pid'.format(volume_name) +- with open(pid_path, 'w') as pid_file: +- pid_file.write(str(nbd_server.pid)) +- + util.SMlog('Create symlink: {} -> {}'.format(self.path, nbd_path)) + os.symlink(nbd_path, self.path) + except Exception as e: +@@ -2688,9 +2690,6 @@ class LinstorVDI(VDI.VDI): + 'VDIUnavailable', + opterr='Failed to start nbd-server: {}'.format(e) + ) +- finally: +- if null: +- null.close() + + @classmethod + def _kill_persistent_server(self, type, volume_name, sig): diff --git a/SOURCES/0102-fix-LinstorSR-handle-inflate-resize-actions-correctl.patch b/SOURCES/0102-fix-LinstorSR-handle-inflate-resize-actions-correctl.patch new file mode 100644 index 0000000..df9479c --- /dev/null +++ b/SOURCES/0102-fix-LinstorSR-handle-inflate-resize-actions-correctl.patch @@ -0,0 +1,131 @@ +From 574565e1b754b653ab88a8ad6d897a5725462fb4 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 16 Jan 2023 17:58:51 +0100 +Subject: [PATCH 102/177] fix(LinstorSR): handle inflate + resize actions + correctly + +- Ensure LINSTOR set the expected new volume size when inflate is executed, + otherwise we log and we use the returned size. +- Repair deflate calls in case of journal entry, cache is not usable. +- Logs VHD and DRBD volume size in create/resize methods. +- Ensure resize is only executed on master. +- Use utilisation size instead of capacity in journal entries. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 49 +++++++++++++++++++++++++++++++------------- + 1 file changed, 35 insertions(+), 14 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index f336534e..72ec9de7 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -234,7 +234,7 @@ def inflate(journaler, linstor, vdi_uuid, vdi_path, new_size, old_size): + return + + util.SMlog( +- 'Inflate {} (new VHD size={}, previous={})' ++ 'Inflate {} (size={}, previous={})' + .format(vdi_uuid, new_size, old_size) + ) + +@@ -243,8 +243,15 @@ def inflate(journaler, linstor, vdi_uuid, vdi_path, new_size, old_size): + ) + linstor.resize_volume(vdi_uuid, new_size) + ++ result_size = linstor.get_volume_size(vdi_uuid) ++ if result_size < new_size: ++ util.SMlog( ++ 'WARNING: Cannot inflate volume to {}B, result size: {}B' ++ .format(new_size, result_size) ++ ) ++ + if not util.zeroOut( +- vdi_path, new_size - vhdutil.VHD_FOOTER_SIZE, ++ vdi_path, result_size - vhdutil.VHD_FOOTER_SIZE, + vhdutil.VHD_FOOTER_SIZE + ): + raise xs_errors.XenError( +@@ -252,7 +259,7 @@ def inflate(journaler, linstor, vdi_uuid, vdi_path, new_size, old_size): + opterr='Failed to zero out VHD footer {}'.format(vdi_path) + ) + +- LinstorVhdUtil(None, linstor).set_size_phys(vdi_path, new_size, False) ++ LinstorVhdUtil(None, linstor).set_size_phys(vdi_path, result_size, False) + journaler.remove(LinstorJournaler.INFLATE, vdi_uuid) + + +@@ -1399,7 +1406,12 @@ class LinstorSR(SR.SR): + util.SMlog('Cannot deflate missing VDI {}'.format(vdi_uuid)) + return + +- current_size = self._all_volume_info_cache.get(self.uuid).virtual_size ++ assert not self._all_volume_info_cache ++ volume_info = self._linstor.get_volume_info(vdi_uuid) ++ ++ current_size = volume_info.virtual_size ++ assert current_size > 0 ++ + util.zeroOut( + vdi.path, + current_size - vhdutil.VHD_FOOTER_SIZE, +@@ -1695,11 +1707,11 @@ class LinstorVDI(VDI.VDI): + + # 2. Compute size and check space available. + size = vhdutil.validate_and_round_vhd_size(long(size)) +- util.SMlog('LinstorVDI.create: type={}, size={}'.format( +- self.vdi_type, size +- )) +- + volume_size = compute_volume_size(size, self.vdi_type) ++ util.SMlog( ++ 'LinstorVDI.create: type={}, vhd-size={}, volume-size={}' ++ .format(self.vdi_type, size, volume_size) ++ ) + self.sr._ensure_space_available(volume_size) + + # 3. Set sm_config attribute of VDI parent class. +@@ -1917,9 +1929,23 @@ class LinstorVDI(VDI.VDI): + + def resize(self, sr_uuid, vdi_uuid, size): + util.SMlog('LinstorVDI.resize for {}'.format(self.uuid)) ++ if not self.sr._is_master: ++ raise xs_errors.XenError( ++ 'VDISize', ++ opterr='resize on slave not allowed' ++ ) ++ + if self.hidden: + raise xs_errors.XenError('VDIUnavailable', opterr='hidden VDI') + ++ # Compute the virtual VHD and DRBD volume size. ++ size = vhdutil.validate_and_round_vhd_size(long(size)) ++ volume_size = compute_volume_size(size, self.vdi_type) ++ util.SMlog( ++ 'LinstorVDI.resize: type={}, vhd-size={}, volume-size={}' ++ .format(self.vdi_type, size, volume_size) ++ ) ++ + if size < self.size: + util.SMlog( + 'vdi_resize: shrinking not supported: ' +@@ -1927,18 +1953,13 @@ class LinstorVDI(VDI.VDI): + ) + raise xs_errors.XenError('VDISize', opterr='shrinking not allowed') + +- # Compute the virtual VHD size. +- size = vhdutil.validate_and_round_vhd_size(long(size)) +- + if size == self.size: + return VDI.VDI.get_params(self) + +- # Compute the LINSTOR volume size. +- new_volume_size = compute_volume_size(size, self.vdi_type) + if self.vdi_type == vhdutil.VDI_TYPE_RAW: + old_volume_size = self.size + else: +- old_volume_size = self.capacity ++ old_volume_size = self.utilisation + if self.sr._provisioning == 'thin': + # VDI is currently deflated, so keep it deflated. + new_volume_size = old_volume_size diff --git a/SOURCES/0103-fix-linstor-manager-add-a-static-iptables-rule-for-D.patch b/SOURCES/0103-fix-linstor-manager-add-a-static-iptables-rule-for-D.patch new file mode 100644 index 0000000..8736c29 --- /dev/null +++ b/SOURCES/0103-fix-linstor-manager-add-a-static-iptables-rule-for-D.patch @@ -0,0 +1,122 @@ +From 01efc1b1f0655e160932bb3667c0fa74733ec84c Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 17 Jan 2023 11:55:00 +0100 +Subject: [PATCH 103/177] fix(linstor-manager): add a static iptables rule for + DRBD volumes + +Using the XAPI iptables firewall may drop DRBD packets when the connection +tracking subsystem runs out of entries temporarily. + +Instead, use a static rule completely independent of the connection tracking +module, it allow packets to pass even when the connection tracking table +ran full temporarily. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 48 ++++++++++++++++++++++++++++++++--------- + 1 file changed, 38 insertions(+), 10 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 5c4c5c90..6ee435c6 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -40,11 +40,12 @@ LVM_PLUGIN = 'lvm.py' + THIN_POOL = 'thin_pool' + + FIREWALL_PORT_SCRIPT = '/etc/xapi.d/plugins/firewall-port' +-LINSTOR_PORTS = [3366, 3370, 3376, 3377, '7000:8000', 8076, 8077] ++LINSTOR_PORTS = [3366, 3370, 3376, 3377, 8076, 8077] ++DRBD_PORTS = '7000:8000' + + +-def update_port(port, open): +- fn = 'open' if open else 'close' ++def update_linstor_port(port, open_ports): ++ fn = 'open' if open_ports else 'close' + args = ( + FIREWALL_PORT_SCRIPT, fn, str(port), 'tcp' + ) +@@ -55,9 +56,36 @@ def update_port(port, open): + raise Exception('Failed to {} port: {} {}'.format(fn, out, err)) + + +-def update_all_ports(open): ++def has_iptables_rule(rule): ++ (ret, stdout, stderr) = util.doexec(['iptables', '-C'] + rule) ++ return not ret ++ ++ ++def update_drbd_ports(open_ports): ++ # We want to use a static rule regarding DRBD volumes, ++ # so we can't use the XAPI firewall port script, we have to manually ++ # check for existing rules before updating iptables service. ++ rule = ['INPUT', '-p', 'tcp', '--dport', DRBD_PORTS, '-j', 'ACCEPT'] ++ if open_ports == has_iptables_rule(rule): ++ return ++ if open_ports: ++ rule.insert(1, '1') ++ (ret, stdout, stderr) = util.doexec(['iptables', '-I'] + rule) ++ if ret: ++ raise Exception('Failed to add DRBD rule: {}'.format(stderr)) ++ else: ++ (ret, stdout, stderr) = util.doexec(['iptables', '-D'] + rule) ++ if ret: ++ raise Exception('Failed to remove DRBD rule: {}'.format(stderr)) ++ (ret, stdout, stderr) = util.doexec(['service', 'iptables', 'save']) ++ if ret: ++ raise Exception('Failed to save DRBD rule: {}'.format(stderr)) ++ ++ ++def update_all_ports(open_ports): + for port in LINSTOR_PORTS: +- update_port(port, open) ++ update_linstor_port(port, open_ports) ++ update_drbd_ports(open_ports) + + + def update_linstor_satellite_service(start): +@@ -202,7 +230,7 @@ def prepare_sr(session, args): + try: + LinstorSR.activate_lvm_group(args['groupName']) + +- update_all_ports(open=True) ++ update_all_ports(open_ports=True) + # We don't want to enable and start minidrbdcluster daemon during + # SR creation. + update_minidrbdcluster_service(start=False) +@@ -217,7 +245,7 @@ def release_sr(session, args): + try: + update_linstor_satellite_service(start=False) + update_minidrbdcluster_service(start=False) +- update_all_ports(open=False) ++ update_all_ports(open_ports=False) + return str(True) + except Exception as e: + util.SMlog('linstor-manager:release_sr error: {}'.format(e)) +@@ -533,7 +561,7 @@ def add_host(session, args): + + try: + # 4. Enable services. +- update_all_ports(open=True) ++ update_all_ports(open_ports=True) + update_minidrbdcluster_service(start=True) + update_linstor_satellite_service(start=True) + +@@ -664,7 +692,7 @@ def add_host(session, args): + if stop_services and not linstor.has_node(node_name): + update_linstor_satellite_service(start=False) + update_minidrbdcluster_service(start=False) +- update_all_ports(open=False) ++ update_all_ports(open_ports=False) + except Exception: + pass + +@@ -747,7 +775,7 @@ def remove_host(session, args): + try: + update_linstor_satellite_service(start=False) + update_minidrbdcluster_service(start=False) +- update_all_ports(open=False) ++ update_all_ports(open_ports=False) + except Exception as e: + util.SMlog('Error while stopping services: {}'.format(e)) + pass diff --git a/SOURCES/0104-feat-LinstorSR-sync-with-last-http-nbd-transfer-vers.patch b/SOURCES/0104-feat-LinstorSR-sync-with-last-http-nbd-transfer-vers.patch new file mode 100644 index 0000000..21b5963 --- /dev/null +++ b/SOURCES/0104-feat-LinstorSR-sync-with-last-http-nbd-transfer-vers.patch @@ -0,0 +1,119 @@ +From 511280dc71a3766cead7f37138cc3b8f843a187b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 3 Feb 2023 16:38:49 +0100 +Subject: [PATCH 104/177] feat(LinstorSR): sync with last http-nbd-transfer + version + +- Increase auto promote timeout of heartbeat VDI to reduce CPU usage +- Modify server regexes +- Force device size parameter of NBD servers + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 25 +++++++++++++++++++++---- + drivers/linstorvolumemanager.py | 19 +++++++++++++++++++ + 2 files changed, 40 insertions(+), 4 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 72ec9de7..31f45055 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1761,6 +1761,13 @@ class LinstorVDI(VDI.VDI): + METADATA_OF_POOL_TAG: '' + } + self._linstor.set_volume_metadata(self.uuid, volume_metadata) ++ ++ # Set the open timeout to 1min to reduce CPU usage ++ # in http-disk-server when a secondary server tries to open ++ # an already opened volume. ++ if self.ty == 'ha_statefile' or self.ty == 'redo_log': ++ self._linstor.set_auto_promote_timeout(self.uuid, 600) ++ + self._linstor.mark_volume_as_persistent(self.uuid) + except util.CommandException as e: + failed = True +@@ -2595,9 +2602,11 @@ class LinstorVDI(VDI.VDI): + with open(pid_path, 'w') as pid_file: + pid_file.write(str(http_server.pid)) + ++ reg_server_ready = re.compile("Server ready!$") + def is_ready(): + while http_server.poll() is None: +- if http_server.stdout.readline().rstrip() == 'Server ready!': ++ line = http_server.stdout.readline() ++ if reg_server_ready.search(line): + return True + return False + try: +@@ -2630,10 +2639,16 @@ class LinstorVDI(VDI.VDI): + nbd_server = None + + try: ++ # We use a precomputed device size. ++ # So if the XAPI is modified, we must update these values! + if volume_name == 'xcp-persistent-ha-statefile': ++ # See: https://github.com/xapi-project/xen-api/blob/703479fa448a8d7141954bb6e8964d8e25c4ac2e/ocaml/xapi/xha_statefile.ml#L32-L37 + port = '8076' ++ device_size = 4 * 1024 * 1024 + else: ++ # See: https://github.com/xapi-project/xen-api/blob/703479fa448a8d7141954bb6e8964d8e25c4ac2e/ocaml/database/redo_log.ml#L41-L44 + port = '8077' ++ device_size = 256 * 1024 * 1024 + + try: + session = util.timeout_call(5, util.get_localAPI_session) +@@ -2653,7 +2668,9 @@ class LinstorVDI(VDI.VDI): + '--nbd-name', + volume_name, + '--urls', +- ','.join(map(lambda ip: 'http://' + ip + ':' + port, ips)) ++ ','.join(map(lambda ip: 'http://' + ip + ':' + port, ips)), ++ '--device-size', ++ str(device_size) + ] + + util.SMlog('Starting {} using port {}...'.format(arguments[0], port)) +@@ -2670,11 +2687,11 @@ class LinstorVDI(VDI.VDI): + with open(pid_path, 'w') as pid_file: + pid_file.write(str(nbd_server.pid)) + +- reg_nbd_path = re.compile("^NBD `(/dev/nbd[0-9]+)` is now attached.$") ++ reg_nbd_path = re.compile("NBD `(/dev/nbd[0-9]+)` is now attached.$") + def get_nbd_path(): + while nbd_server.poll() is None: + line = nbd_server.stdout.readline() +- match = reg_nbd_path.match(line) ++ match = reg_nbd_path.search(line) + if match: + return match.group(1) + # Use a timeout to never block the smapi if there is a problem. +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 81cce802..e0f39e71 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -851,6 +851,25 @@ class LinstorVolumeManager(object): + ) + return size * 1024 + ++ ++ def set_auto_promote_timeout(self, volume_uuid, timeout): ++ """ ++ Define the blocking time of open calls when a DRBD ++ is already open on another host. ++ :param str volume_uuid: The volume uuid to modify. ++ """ ++ ++ volume_name = self.get_volume_name(volume_uuid) ++ result = self._linstor.resource_dfn_modify(volume_name, { ++ 'DrbdOptions/Resource/auto-promote-timeout': timeout ++ }) ++ error_str = self._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not change the auto promote timeout of `{}`: {}' ++ .format(volume_uuid, error_str) ++ ) ++ + def get_volume_info(self, volume_uuid): + """ + Get the volume info of a particular volume. diff --git a/SOURCES/0105-fix-LinstorSR-don-t-check-VDI-metadata-while-listing.patch b/SOURCES/0105-fix-LinstorSR-don-t-check-VDI-metadata-while-listing.patch new file mode 100644 index 0000000..3446543 --- /dev/null +++ b/SOURCES/0105-fix-LinstorSR-don-t-check-VDI-metadata-while-listing.patch @@ -0,0 +1,35 @@ +From 344d0c58408c89f48fb41cb1bfd70e5bed56781e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 13 Feb 2023 17:24:16 +0100 +Subject: [PATCH 105/177] fix(LinstorSR): don't check VDI metadata while + listing VDIs if it's deleted + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 31f45055..41ece825 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1169,6 +1169,9 @@ class LinstorSR(SR.SR): + if not introduce: + continue + ++ if vdi_uuid.startswith('DELETED_'): ++ continue ++ + volume_metadata = volumes_metadata.get(vdi_uuid) + if not volume_metadata: + util.SMlog( +@@ -1177,9 +1180,6 @@ class LinstorSR(SR.SR): + ) + continue + +- if vdi_uuid.startswith('DELETED_'): +- continue +- + util.SMlog( + 'Trying to introduce VDI {} as it is present in ' + 'LINSTOR and not in XAPI...' diff --git a/SOURCES/0106-fix-LinstorSR-don-t-check-metadata-when-destroying-s.patch b/SOURCES/0106-fix-LinstorSR-don-t-check-metadata-when-destroying-s.patch new file mode 100644 index 0000000..f0b94ad --- /dev/null +++ b/SOURCES/0106-fix-LinstorSR-don-t-check-metadata-when-destroying-s.patch @@ -0,0 +1,32 @@ +From a00ccf6b47da3012b77dcba6cc22e21124434ec9 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 13 Feb 2023 17:27:43 +0100 +Subject: [PATCH 106/177] fix(LinstorSR): don't check metadata when destroying + snap in undo_clone + +Remove useless check in the snap rollback helper when there is an error +during the second `_create_snapshot` call (when VDI.clone command is executed). + +There is no reason to verify the metadata, this code is +present in the LVM driver, but is useless here. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 4 ---- + 1 file changed, 4 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 41ece825..94cf1b77 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1496,10 +1496,6 @@ class LinstorSR(SR.SR): + # Remove the child nodes. + if snap_uuid and snap_uuid in volume_names: + util.SMlog('Destroying snap {}...'.format(snap_uuid)) +- snap_metadata = self._linstor.get_volume_metadata(snap_uuid) +- +- if snap_metadata.get(VDI_TYPE_TAG) != vhdutil.VDI_TYPE_VHD: +- raise util.SMException('Clone {} not VHD'.format(snap_uuid)) + + try: + self._linstor.destroy_volume(snap_uuid) diff --git a/SOURCES/0107-fix-linstorvhdutil-handle-correctly-generic-exceptio.patch b/SOURCES/0107-fix-linstorvhdutil-handle-correctly-generic-exceptio.patch new file mode 100644 index 0000000..4846877 --- /dev/null +++ b/SOURCES/0107-fix-linstorvhdutil-handle-correctly-generic-exceptio.patch @@ -0,0 +1,60 @@ +From 04722ec04e536210e0504bb784e649639d6af306 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 15 Feb 2023 11:34:54 +0100 +Subject: [PATCH 107/177] fix(linstorvhdutil): handle correctly generic + exceptions in _raise_openers_exception + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvhdutil.py | 21 +++++++++++---------- + 1 file changed, 11 insertions(+), 10 deletions(-) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 63d59ab5..05225e88 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -289,25 +289,26 @@ class LinstorVhdUtil: + # -------------------------------------------------------------------------- + + def _raise_openers_exception(self, device_path, e): ++ if isinstance(e, util.CommandException): ++ e_str = 'cmd: `{}`, code: `{}`, reason: `{}`'.format(e.cmd, e.code, e.reason) ++ else: ++ e_str = str(e) ++ + e_with_openers = None + try: + volume_uuid = self._linstor.get_volume_uuid_from_device_path( + device_path + ) +- e_wrapper = util.CommandException( +- e.code, +- e.cmd, +- e.reason + ' (openers: {})'.format( ++ e_wrapper = Exception( ++ e_str + ' (openers: {})'.format( + self._linstor.get_volume_openers(volume_uuid) + ) + ) + except Exception as illformed_e: +- e_wrapper = util.CommandException( +- e.code, +- e.cmd, +- e.reason + ' (unable to get openers: {})'.format(illformed_e) ++ e_wrapper = Exception( ++ e_str + ' (unable to get openers: {})'.format(illformed_e) + ) +- util.SMlog('raise opener exception: {} ({})'.format(e_wrapper, e_wrapper.reason)) ++ util.SMlog('raise opener exception: {}'.format(e_wrapper)) + raise e_wrapper # pylint: disable = E0702 + + def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): +@@ -330,7 +331,7 @@ class LinstorVhdUtil: + return self._call_local_vhd_util(local_method, device_path, *args, **kwargs) + except ErofsLinstorCallException as e: + # Volume is locked on a host, find openers. +- self._raise_openers_exception(device_path, e) ++ self._raise_openers_exception(device_path, e.cmd_err) + + def _call_vhd_util(self, local_method, remote_method, device_path, use_parent, *args, **kwargs): + # Note: `use_parent` exists to know if the VHD parent is used by the local/remote method. diff --git a/SOURCES/0108-fix-minidrbdcluster-robustify-to-unmount-correctly-L.patch b/SOURCES/0108-fix-minidrbdcluster-robustify-to-unmount-correctly-L.patch new file mode 100644 index 0000000..796e273 --- /dev/null +++ b/SOURCES/0108-fix-minidrbdcluster-robustify-to-unmount-correctly-L.patch @@ -0,0 +1,86 @@ +From 3b63e456318beb1746adab95136d142d1ee93f9e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 16 Feb 2023 14:24:07 +0100 +Subject: [PATCH 108/177] fix(minidrbdcluster): robustify to unmount correctly + LINSTOR DB + +There is a small delay during which the database may not be unmounted +because there are still processes using it. So must retry in this case. +It's caused by the termination of the LINSTOR controller. + +Signed-off-by: Ronan Abhamon +--- + Makefile | 1 + + etc/systemd/system/var-lib-linstor.service | 2 +- + scripts/safe-umount | 39 ++++++++++++++++++++++ + 3 files changed, 41 insertions(+), 1 deletion(-) + create mode 100755 scripts/safe-umount + +diff --git a/Makefile b/Makefile +index 42058c82..72d7be3f 100755 +--- a/Makefile ++++ b/Makefile +@@ -241,6 +241,7 @@ install: precheck + mkdir -p $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/fork-log-daemon $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/linstor-kv-tool $(SM_STAGING)$(BIN_DEST) ++ install -m 755 scripts/safe-umount $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/local-device-change $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/check-device-sharing $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/usb_change $(SM_STAGING)$(LIBEXEC) +diff --git a/etc/systemd/system/var-lib-linstor.service b/etc/systemd/system/var-lib-linstor.service +index d230d048..e9deb904 100644 +--- a/etc/systemd/system/var-lib-linstor.service ++++ b/etc/systemd/system/var-lib-linstor.service +@@ -17,5 +17,5 @@ Description=Mount filesystem for the LINSTOR controller + [Service] + Type=oneshot + ExecStart=/bin/mount -w /dev/drbd/by-res/xcp-persistent-database/0 /var/lib/linstor +-ExecStop=/bin/umount /var/lib/linstor ++ExecStop=/opt/xensource/libexec/safe-umount /var/lib/linstor + RemainAfterExit=true +diff --git a/scripts/safe-umount b/scripts/safe-umount +new file mode 100755 +index 00000000..9c1dcc40 +--- /dev/null ++++ b/scripts/safe-umount +@@ -0,0 +1,39 @@ ++#!/usr/bin/env python2 ++ ++import argparse ++import subprocess ++import sys ++import time ++ ++ ++def safe_umount(path): ++ retry_count = 10 ++ not_mounted_str = 'umount: {}: not mounted'.format(path) ++ ++ last_code = 0 ++ while retry_count: ++ proc = subprocess.Popen(['mountpoint', '-q', path]) ++ proc.wait() ++ if proc.returncode: ++ return 0 ++ ++ proc = subprocess.Popen(['umount', path], stderr=subprocess.PIPE) ++ (stdout, stderr) = proc.communicate() ++ if not proc.returncode: ++ return 0 ++ ++ error = stderr.strip() ++ if error == not_mounted_str: ++ return 0 ++ ++ retry_count -= 1 ++ last_code = proc.returncode ++ time.sleep(0.500) ++ return last_code ++ ++ ++if __name__ == '__main__': ++ parser = argparse.ArgumentParser() ++ parser.add_argument('path') ++ args = parser.parse_args() ++ sys.exit(safe_umount(args.path)) diff --git a/SOURCES/0109-fix-minidrbdcluster-handle-correctly-KeyboardInterru.patch b/SOURCES/0109-fix-minidrbdcluster-handle-correctly-KeyboardInterru.patch new file mode 100644 index 0000000..6e1765e --- /dev/null +++ b/SOURCES/0109-fix-minidrbdcluster-handle-correctly-KeyboardInterru.patch @@ -0,0 +1,34 @@ +From af8bb2c311fb2dcf54f158b60b62c4b77073bbe8 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 20 Feb 2023 19:30:18 +0100 +Subject: [PATCH 109/177] fix(minidrbdcluster): handle correctly + KeyboardInterrupt with systemd units + +It's necessary to always add systemd services in the running list before +trying to start a service, because if a KeyboardInterrupt is sent, we can have +a running LINSTOR controller not present in the list, and then we can no longer +unmount /var/lib/linstor because the controller is never stopped... + +Signed-off-by: Ronan Abhamon +--- + scripts/minidrbdcluster | 5 +++-- + 1 file changed, 3 insertions(+), 2 deletions(-) + +diff --git a/scripts/minidrbdcluster b/scripts/minidrbdcluster +index eae7cbfe..03d6b010 100755 +--- a/scripts/minidrbdcluster ++++ b/scripts/minidrbdcluster +@@ -104,10 +104,11 @@ def process(events2, resources, running_services, status): + res_name, may_promote, promotion_score = m.groups() + if res_name in resources and may_promote == 'yes': + for systemd_unit in resources[res_name]['systemd-units']: +- if not ensure_systemd_started(systemd_unit): +- break + if systemd_unit not in running_services: + running_services.append(systemd_unit) ++ if not ensure_systemd_started(systemd_unit): ++ running_services.pop() ++ break + m = PEER_ROLE_RE.match(line) + if m: + res_name, conn_name, role = m.groups() diff --git a/SOURCES/0110-feat-LinstorSR-use-drbd-reactor-instead-of-minidrbdc.patch b/SOURCES/0110-feat-LinstorSR-use-drbd-reactor-instead-of-minidrbdc.patch new file mode 100644 index 0000000..9a60125 --- /dev/null +++ b/SOURCES/0110-feat-LinstorSR-use-drbd-reactor-instead-of-minidrbdc.patch @@ -0,0 +1,618 @@ +From 180f9e933cad1eefb1d9694b4a98d912d70b8697 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 24 Feb 2023 14:28:29 +0100 +Subject: [PATCH 110/177] feat(LinstorSR): use drbd-reactor instead of + minidrbdcluster + +Signed-off-by: Ronan Abhamon +--- + Makefile | 7 -- + drivers/LinstorSR.py | 37 +++--- + drivers/linstor-manager | 67 +++++++++-- + drivers/linstorvolumemanager.py | 4 +- + etc/minidrbdcluster.ini | 14 --- + scripts/minidrbdcluster | 203 -------------------------------- + systemd/minidrbdcluster.service | 19 --- + 7 files changed, 76 insertions(+), 275 deletions(-) + delete mode 100644 etc/minidrbdcluster.ini + delete mode 100755 scripts/minidrbdcluster + delete mode 100644 systemd/minidrbdcluster.service + +diff --git a/Makefile b/Makefile +index 72d7be3f..bc3e97fe 100755 +--- a/Makefile ++++ b/Makefile +@@ -100,7 +100,6 @@ MPATH_CUSTOM_CONF_DIR := /etc/multipath/conf.d/ + MODPROBE_DIR := /etc/modprobe.d/ + EXTENSION_SCRIPT_DEST := /etc/xapi.d/extensions/ + LOGROTATE_DIR := /etc/logrotate.d/ +-MINI_DRBD_CLUSTER_CONF_DIR := /etc/ + + SM_STAGING := $(DESTDIR) + SM_STAMP := $(MY_OBJ_DIR)/.staging_stamp +@@ -156,7 +155,6 @@ install: precheck + mkdir -p $(SM_STAGING)$(MPATH_CUSTOM_CONF_DIR) + mkdir -p $(SM_STAGING)$(MODPROBE_DIR) + mkdir -p $(SM_STAGING)$(LOGROTATE_DIR) +- mkdir -p $(SM_STAGING)$(MINI_DRBD_CLUSTER_CONF_DIR) + mkdir -p $(SM_STAGING)$(DEBUG_DEST) + mkdir -p $(SM_STAGING)$(BIN_DEST) + mkdir -p $(SM_STAGING)$(MASTER_SCRIPT_DEST) +@@ -184,8 +182,6 @@ install: precheck + $(SM_STAGING)/$(SYSTEMD_CONF_DIR)/linstor-satellite.service.d/ + install -m 644 etc/systemd/system/var-lib-linstor.service \ + $(SM_STAGING)/$(SYSTEMD_CONF_DIR) +- install -m 644 etc/minidrbdcluster.ini \ +- $(SM_STAGING)/$(MINI_DRBD_CLUSTER_CONF_DIR) + install -m 644 etc/make-dummy-sr.service \ + $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) + install -m 644 systemd/xs-sm.service \ +@@ -204,8 +200,6 @@ install: precheck + $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) + install -m 644 systemd/linstor-monitor.service \ + $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) +- install -m 644 systemd/minidrbdcluster.service \ +- $(SM_STAGING)/$(SYSTEMD_SERVICE_DIR) + for i in $(UDEV_RULES); do \ + install -m 644 udev/$$i.rules \ + $(SM_STAGING)$(UDEV_RULES_DIR); done +@@ -258,7 +252,6 @@ install: precheck + install -m 755 scripts/xe-getlunidentifier $(SM_STAGING)$(BIN_DEST) + install -m 755 scripts/make-dummy-sr $(SM_STAGING)$(LIBEXEC) + install -m 755 scripts/storage-init $(SM_STAGING)$(LIBEXEC) +- install -m 755 scripts/minidrbdcluster $(SM_STAGING)$(LIBEXEC) + + .PHONY: clean + clean: +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 94cf1b77..a3da28e7 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -461,6 +461,10 @@ class LinstorSR(SR.SR): + return method(self, *args, **kwargs) + + def load(self, *args, **kwargs): ++ # Activate all LVMs to make drbd-reactor happy. ++ if self.srcmd.cmd == 'sr_attach': ++ activate_lvm_group(self._group_name) ++ + if not self._has_session: + if self.srcmd.cmd in ( + 'vdi_attach_from_config', +@@ -707,7 +711,7 @@ class LinstorSR(SR.SR): + ) + + # Ensure ports are opened and LINSTOR satellites +- # are activated. In the same time the minidrbdcluster instances ++ # are activated. In the same time the drbd-reactor instances + # must be stopped. + self._prepare_sr_on_all_hosts(self._group_name, enabled=True) + +@@ -730,9 +734,9 @@ class LinstorSR(SR.SR): + + try: + util.SMlog( +- "Finishing SR creation, enable minidrbdcluster on all hosts..." ++ "Finishing SR creation, enable drbd-reactor on all hosts..." + ) +- self._update_minidrbdcluster_on_all_hosts(enabled=True) ++ self._update_drbd_reactor_on_all_hosts(enabled=True) + except Exception as e: + try: + self._linstor.destroy() +@@ -777,7 +781,7 @@ class LinstorSR(SR.SR): + ) + + try: +- self._update_minidrbdcluster_on_all_hosts( ++ self._update_drbd_reactor_on_all_hosts( + controller_node_name=node_name, enabled=False + ) + +@@ -789,12 +793,12 @@ class LinstorSR(SR.SR): + ) + except Exception as e: + try: +- self._update_minidrbdcluster_on_all_hosts( ++ self._update_drbd_reactor_on_all_hosts( + controller_node_name=node_name, enabled=True + ) + except Exception as e2: + util.SMlog( +- 'Failed to restart minidrbdcluster after destroy fail: {}' ++ 'Failed to restart drbd-reactor after destroy fail: {}' + .format(e2) + ) + util.SMlog('Failed to delete LINSTOR SR: {}'.format(e)) +@@ -838,7 +842,6 @@ class LinstorSR(SR.SR): + 'SRUnavailable', + opterr='no such group: {}'.format(self._group_name) + ) +- activate_lvm_group(self._group_name) + + @_locked_load + def detach(self, uuid): +@@ -963,15 +966,15 @@ class LinstorSR(SR.SR): + for slave in util.get_all_slaves(self.session): + self._prepare_sr(slave, group_name, enabled) + +- def _update_minidrbdcluster(self, host, enabled): ++ def _update_drbd_reactor(self, host, enabled): + self._exec_manager_command( + host, +- 'updateMinidrbdcluster', ++ 'updateDrbdReactor', + {'enabled': str(enabled)}, + 'SRUnavailable' + ) + +- def _update_minidrbdcluster_on_all_hosts( ++ def _update_drbd_reactor_on_all_hosts( + self, enabled, controller_node_name=None + ): + if controller_node_name == 'localhost': +@@ -999,27 +1002,27 @@ class LinstorSR(SR.SR): + )) + + if enabled and controller_host: +- util.SMlog('{} minidrbdcluster on controller host `{}`...'.format( ++ util.SMlog('{} drbd-reactor on controller host `{}`...'.format( + action_name, controller_node_name + )) + # If enabled is true, we try to start the controller on the desired + # node name first. +- self._update_minidrbdcluster(controller_host, enabled) ++ self._update_drbd_reactor(controller_host, enabled) + + for host_ref, hostname in secondary_hosts: +- util.SMlog('{} minidrbdcluster on host {}...'.format( ++ util.SMlog('{} drbd-reactor on host {}...'.format( + action_name, hostname + )) +- self._update_minidrbdcluster(host_ref, enabled) ++ self._update_drbd_reactor(host_ref, enabled) + + if not enabled and controller_host: +- util.SMlog('{} minidrbdcluster on controller host `{}`...'.format( ++ util.SMlog('{} drbd-reactor on controller host `{}`...'.format( + action_name, controller_node_name + )) +- # If enabled is false, we disable the minidrbdcluster service of ++ # If enabled is false, we disable the drbd-reactor service of + # the controller host last. Why? Otherwise the linstor-controller + # of other nodes can be started, and we don't want that. +- self._update_minidrbdcluster(controller_host, enabled) ++ self._update_drbd_reactor(controller_host, enabled) + + # -------------------------------------------------------------------------- + # Metadata. +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 6ee435c6..7e34b5f6 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -22,6 +22,7 @@ sys.path[0] = '/opt/xensource/sm/' + + import base64 + import distutils.util ++import os + import socket + import XenAPI + import XenAPIPlugin +@@ -43,6 +44,19 @@ FIREWALL_PORT_SCRIPT = '/etc/xapi.d/plugins/firewall-port' + LINSTOR_PORTS = [3366, 3370, 3376, 3377, 8076, 8077] + DRBD_PORTS = '7000:8000' + ++DRBD_REACTOR_CONF = '/etc/drbd-reactor.d/sm-linstor.toml' ++ ++DRBD_REACTOR_CONF_CONTENT = """[[promoter]] ++ ++[promoter.resources.xcp-persistent-database] ++start = [ "var-lib-linstor.service", "linstor-controller.service" ] ++""" ++ ++DRBD_REACTOR_DEPS = [ ++ '/run/systemd/system/linstor-controller.service.d/reactor.conf', ++ '/run/systemd/system/var-lib-linstor.service.d/reactor.conf' ++] ++ + + def update_linstor_port(port, open_ports): + fn = 'open' if open_ports else 'close' +@@ -101,8 +115,35 @@ def update_linstor_satellite_service(start): + util.enable_and_start_service(service, True) + + +-def update_minidrbdcluster_service(start): +- util.enable_and_start_service('minidrbdcluster', start) ++def update_drbd_reactor_service(start): ++ if start: ++ util.atomicFileWrite(DRBD_REACTOR_CONF, None, DRBD_REACTOR_CONF_CONTENT) ++ else: ++ try: ++ os.remove(DRBD_REACTOR_CONF) ++ except Exception: ++ pass ++ ++ util.stop_service('drbd-reactor') ++ ++ try: ++ util.stop_service('drbd-promote@xcp\x2dpersistent\x2ddatabase.service') ++ except Exception as e: ++ if str(e).rstrip().endswith(' not loaded.'): ++ pass ++ raise e ++ ++ util.stop_service('linstor-controller') ++ util.stop_service('var-lib-linstor.service') ++ ++ for dep in DRBD_REACTOR_DEPS: ++ try: ++ os.remove(dep) ++ except Exception: ++ pass ++ ++ util.doexec(['systemctl', 'daemon-reload']) ++ util.enable_and_start_service('drbd-reactor', start) + + + def exec_create_sr(session, name, description, disks, volume_group, redundancy, provisioning, force): +@@ -231,9 +272,9 @@ def prepare_sr(session, args): + LinstorSR.activate_lvm_group(args['groupName']) + + update_all_ports(open_ports=True) +- # We don't want to enable and start minidrbdcluster daemon during ++ # We don't want to enable and start drbd-reactor daemon during + # SR creation. +- update_minidrbdcluster_service(start=False) ++ update_drbd_reactor_service(start=False) + update_linstor_satellite_service(start=True) + return str(True) + except Exception as e: +@@ -244,7 +285,7 @@ def prepare_sr(session, args): + def release_sr(session, args): + try: + update_linstor_satellite_service(start=False) +- update_minidrbdcluster_service(start=False) ++ update_drbd_reactor_service(start=False) + update_all_ports(open_ports=False) + return str(True) + except Exception as e: +@@ -252,14 +293,14 @@ def release_sr(session, args): + return str(False) + + +-def update_minidrbdcluster(session, args): ++def update_drbd_reactor(session, args): + try: + enabled = distutils.util.strtobool(args['enabled']) +- update_minidrbdcluster_service(start=enabled) ++ update_drbd_reactor_service(start=enabled) + return str(True) + except Exception as e: + util.SMlog( +- 'linstor-manager:update_minidrbdcluster error: {}'.format(e) ++ 'linstor-manager:update_drbd_reactor error: {}'.format(e) + ) + return str(False) + +@@ -308,7 +349,7 @@ def destroy(session, args): + try: + group_name = args['groupName'] + +- # When destroy is called, there are no running minidrbdcluster daemons. ++ # When destroy is called, there are no running drbd-reactor daemons. + # So the controllers are stopped too, we must start an instance. + util.restart_service('var-lib-linstor.service') + util.restart_service('linstor-controller') +@@ -562,7 +603,7 @@ def add_host(session, args): + try: + # 4. Enable services. + update_all_ports(open_ports=True) +- update_minidrbdcluster_service(start=True) ++ update_drbd_reactor_service(start=True) + update_linstor_satellite_service(start=True) + + # 5. Try to create local node. +@@ -691,7 +732,7 @@ def add_host(session, args): + # If we failed to remove the node, we don't stop services. + if stop_services and not linstor.has_node(node_name): + update_linstor_satellite_service(start=False) +- update_minidrbdcluster_service(start=False) ++ update_drbd_reactor_service(start=False) + update_all_ports(open_ports=False) + except Exception: + pass +@@ -774,7 +815,7 @@ def remove_host(session, args): + # 3. Stop services. + try: + update_linstor_satellite_service(start=False) +- update_minidrbdcluster_service(start=False) ++ update_drbd_reactor_service(start=False) + update_all_ports(open_ports=False) + except Exception as e: + util.SMlog('Error while stopping services: {}'.format(e)) +@@ -1005,7 +1046,7 @@ if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, + 'releaseSr': release_sr, +- 'updateMinidrbdcluster': update_minidrbdcluster, ++ 'updateDrbdReactor': update_drbd_reactor, + 'attach': attach, + 'detach': detach, + 'destroy': destroy, +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index e0f39e71..4662043c 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1622,7 +1622,7 @@ class LinstorVolumeManager(object): + ) + finally: + # Controller must be stopped and volume unmounted because +- # it is the role of the minidrbdcluster daemon to do the right ++ # it is the role of the drbd-reactor daemon to do the right + # actions. + cls._start_controller(start=False) + cls._mount_volume( +@@ -2625,7 +2625,7 @@ class LinstorVolumeManager(object): + ) + + # We must modify the quorum. Otherwise we can't use correctly the +- # minidrbdcluster daemon. ++ # drbd-reactor daemon. + if auto_quorum: + result = lin.resource_dfn_modify(DATABASE_VOLUME_NAME, { + 'DrbdOptions/auto-quorum': 'disabled', +diff --git a/etc/minidrbdcluster.ini b/etc/minidrbdcluster.ini +deleted file mode 100644 +index 9e523427..00000000 +--- a/etc/minidrbdcluster.ini ++++ /dev/null +@@ -1,14 +0,0 @@ +-# minidrbdcluster keeps a service running on one of the nodes. +-# Quorum must be enabled in the DRBD resource! +-# +-# The section names are the names of DRBD resources. Within a +-# section name the systemd-units to activate on one of the nodes. +- +-[xcp-persistent-database] +-systemd-units=var-lib-linstor.service,linstor-controller.service +- +-[xcp-persistent-ha-statefile] +-systemd-units= +- +-[xcp-persistent-redo-log] +-systemd-units= +diff --git a/scripts/minidrbdcluster b/scripts/minidrbdcluster +deleted file mode 100755 +index 03d6b010..00000000 +--- a/scripts/minidrbdcluster ++++ /dev/null +@@ -1,203 +0,0 @@ +-#! /usr/bin/env python2 +- +-import configparser +-import re +-import signal +-import subprocess +- +-DRBDADM_OPEN_FAILED_RE = re.compile( +- 'open\\((.*)\\) failed: No such file or directory' +-) +-MAY_PROMOT_RE = re.compile( +- '(?:exists|change) resource name:((?:\\w|-)+) ' +- '(?:(?:\\w|-)+\\:(?:\\w|-)+ )*may_promote:(yes|no) promotion_score:(\\d+)' +-) +-PEER_ROLE_RE = re.compile( +- '(?:exists|change) connection name:((?:\\w|-)+) peer-node-id:(?:\\d+) ' +- 'conn-name:((?:\\w|-)+) (?:(?:\\w|-)+\\:(?:\\w|-)+ )*role:(Primary|Secondary|Unknown)' +-) +-HAVE_QUORUM_RE = re.compile( +- '(?:exists|change) device name:((?:\\w|-)+) ' +- '(?:(?:\\w|-)+\\:(?:\\w|-)+ )*quorum:(yes|no)' +-) +- +- +-class SigHupException(Exception): +- pass +- +- +-def sig_handler(sig, frame): +- raise SigHupException( +- 'Received signal ' + str(sig) + +- ' on line ' + str(frame.f_lineno) + +- ' in ' + frame.f_code.co_filename +- ) +- +- +-def preexec_subprocess(): +- signal.signal(signal.SIGINT, signal.SIG_IGN) +- +- +-def exec_subprocess(args): +- proc = subprocess.Popen(args, preexec_fn=preexec_subprocess) +- raise_sigint = False +- while True: +- try: +- proc.wait() +- break +- except KeyboardInterrupt: +- raise_sigint = True +- except: # noqa: E722 +- pass +- +- if raise_sigint: +- raise KeyboardInterrupt +- +- return proc.returncode +- +- +-def call_systemd(operation, service): +- verbose = operation in ('start', 'stop') +- if verbose: +- print('Trying to %s %s' % (operation, service)) +- ret = exec_subprocess(['systemctl', operation, service]) +- if verbose: +- print('%s for %s %s' % ( +- 'success' if ret == 0 else 'failure', operation, service +- )) +- return ret == 0 +- +- +-def ensure_systemd_started(service): +- if not exec_subprocess(['systemctl', 'is-active', '--quiet', service]): +- return True # Already active. +- +- return call_systemd('start', service) +- +- +-def show_status(services, status): +- print('status:') +- for systemd_unit in services: +- call_systemd('status', systemd_unit) +- for res_name in status: +- print('%s is %s' % (res_name, status[res_name])) +- +- +-def stop_services(services): +- for systemd_unit in reversed(services): +- call_systemd('stop', systemd_unit) +- +- +-def get_systemd_units(systemd_units_str): +- systemd_units = [] +- for systemd_unit in systemd_units_str.split(','): +- systemd_unit = systemd_unit.strip() +- if systemd_unit: +- systemd_units.append(systemd_unit) +- return systemd_units +- +- +-def process(events2, resources, running_services, status): +- line = events2.stdout.readline() +- m = MAY_PROMOT_RE.match(line) +- if m: +- res_name, may_promote, promotion_score = m.groups() +- if res_name in resources and may_promote == 'yes': +- for systemd_unit in resources[res_name]['systemd-units']: +- if systemd_unit not in running_services: +- running_services.append(systemd_unit) +- if not ensure_systemd_started(systemd_unit): +- running_services.pop() +- break +- m = PEER_ROLE_RE.match(line) +- if m: +- res_name, conn_name, role = m.groups() +- if res_name in status: +- status[res_name][conn_name] = role +- m = HAVE_QUORUM_RE.match(line) +- if m: +- res_name, have_quorum = m.groups() +- if res_name in resources and have_quorum == 'no': +- systemd_units = resources[res_name]['systemd-units'] +- to_stop = [x for x in systemd_units if x in running_services] +- if to_stop: +- print('Lost quorum on %s' % (res_name)) +- for systemd_unit in reversed(to_stop): +- r = call_systemd('stop', systemd_unit) +- if r: +- running_services.remove(systemd_unit) +- +- +-def active_drbd_volume(res_name): +- retry = True +- args = ['drbdadm', 'adjust', res_name] +- while True: +- proc = subprocess.Popen(args, stderr=subprocess.PIPE) +- (stdout, stderr) = proc.communicate() +- if not proc.returncode: +- return # Success. \o/ +- +- if not retry: +- break +- +- m = DRBDADM_OPEN_FAILED_RE.match(stderr) +- if m and subprocess.call(['lvchange', '-ay', m.groups()[0]]) == 0: +- retry = False +- else: +- break +- +- print('Failed to execute `{}`: {}'.format(args, stderr)) +- +- +-def main(): +- # 1. Load minidrbdcluster config. +- config = configparser.ConfigParser() +- config.read('/etc/minidrbdcluster.ini') +- resources = config._sections +- if not resources: +- raise Exception( +- 'No resources to watch, maybe /etc/minidrbdcluster.ini missing' +- ) +- print('Managing DRBD resources: %s' % (' '.join(resources))) +- +- # 2. Prepare resources. +- status = dict() +- all_services = [] # Contains common services between each DRBD volumes. +- for res_name, resource in resources.iteritems(): +- status[res_name] = dict() +- active_drbd_volume(res_name) +- systemd_units = get_systemd_units(resource['systemd-units']) +- resource['systemd-units'] = systemd_units +- +- for systemd_unit in systemd_units: +- if systemd_unit not in all_services: +- all_services.append(systemd_unit) +- +- # 3. Ensure all services are stopped. +- stop_services(all_services) +- +- # 4. Run! +- signal.signal(signal.SIGHUP, sig_handler) +- +- running_services = [] +- +- print('Starting process...') +- events2 = subprocess.Popen( +- ['drbdsetup', 'events2'], stdout=subprocess.PIPE +- ) +- run = True +- while run: +- try: +- process(events2, resources, running_services, status) +- except KeyboardInterrupt: +- run = False +- except SigHupException: +- show_status(running_services, status) +- except Exception: +- print('Unhandled exception: %s' % str(e)) +- +- print('Exiting...') +- stop_services(running_services) +- +-if __name__ == '__main__': +- main() +diff --git a/systemd/minidrbdcluster.service b/systemd/minidrbdcluster.service +deleted file mode 100644 +index 1ddf91f3..00000000 +--- a/systemd/minidrbdcluster.service ++++ /dev/null +@@ -1,19 +0,0 @@ +-[Unit] +-Description=Minimalistic high-availability cluster resource manager +-Before=xs-sm.service +-Wants=network-online.target +-After=network-online.target +- +-[Service] +-Type=simple +-Environment=PYTHONUNBUFFERED=1 +-ExecStart=/opt/xensource/libexec/minidrbdcluster +-KillMode=process +-KillSignal=SIGINT +-SendSIGKILL=no +-StandardOutput=journal +-StandardError=journal +-SyslogIdentifier=minidrbdcluster +- +-[Install] +-WantedBy=multi-user.target diff --git a/SOURCES/0111-fix-LinstorSR-ensure-vhdutil-calls-are-correctly-exe.patch b/SOURCES/0111-fix-LinstorSR-ensure-vhdutil-calls-are-correctly-exe.patch new file mode 100644 index 0000000..7ba0109 --- /dev/null +++ b/SOURCES/0111-fix-LinstorSR-ensure-vhdutil-calls-are-correctly-exe.patch @@ -0,0 +1,144 @@ +From b49b91f7a56cc4e673fbc6edcc05bc40a14d1bea Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 1 Mar 2023 10:56:43 +0100 +Subject: [PATCH 111/177] fix(LinstorSR): ensure vhdutil calls are correctly + executed on pools with > 3 hosts + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 20 +++++++------------- + drivers/linstorvhdutil.py | 19 ++++++++++++++++--- + drivers/linstorvolumemanager.py | 8 ++++---- + 3 files changed, 27 insertions(+), 20 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index a3da28e7..10e0f543 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1884,9 +1884,12 @@ class LinstorVDI(VDI.VDI): + return self._attach_using_http_nbd() + + if not util.pathexists(self.path): +- raise xs_errors.XenError( +- 'VDIUnavailable', opterr='Could not find: {}'.format(self.path) +- ) ++ # Ensure we have a path... ++ self._linstor.get_device_path(vdi_uuid) ++ if not util.pathexists(self.path): ++ raise xs_errors.XenError( ++ 'VDIUnavailable', opterr='Could not find: {}'.format(self.path) ++ ) + + self.attached = True + return VDI.VDI.attach(self, self.sr.uuid, self.uuid) +@@ -2137,16 +2140,7 @@ class LinstorVDI(VDI.VDI): + self.size = volume_info.virtual_size + self.parent = '' + else: +- try: +- vhd_info = self.sr._vhdutil.get_vhd_info(self.uuid) +- except util.CommandException as e: +- if e.code != errno.ENOENT: +- raise +- # Path doesn't exist. Probably a diskless without local path. +- # Force creation and retry. +- self._linstor.get_device_path(self.uuid) +- vhd_info = self.sr._vhdutil.get_vhd_info(self.uuid) +- ++ vhd_info = self.sr._vhdutil.get_vhd_info(self.uuid) + self.hidden = vhd_info.hidden + self.size = vhd_info.sizeVirt + self.parent = vhd_info.parentUuid +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 05225e88..c1b817d7 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -47,7 +47,7 @@ def call_vhd_util_on_host(session, host_ref, method, device_path, args): + return response + + +-class ErofsLinstorCallException(Exception): ++class LinstorCallException(Exception): + def __init__(self, cmd_err): + self.cmd_err = cmd_err + +@@ -55,6 +55,14 @@ class ErofsLinstorCallException(Exception): + return str(self.cmd_err) + + ++class ErofsLinstorCallException(LinstorCallException): ++ pass ++ ++ ++class NoPathLinstorCallException(LinstorCallException): ++ pass ++ ++ + def linstorhostcall(local_method, remote_method): + def decorated(response_parser): + def wrapper(*args, **kwargs): +@@ -70,12 +78,12 @@ def linstorhostcall(local_method, remote_method): + + # Try to read locally if the device is not in use or if the device + # is up to date and not diskless. +- (node_names, in_use) = \ ++ (node_names, in_use_by) = \ + self._linstor.find_up_to_date_diskful_nodes(vdi_uuid) + + local_e = None + try: +- if not in_use or socket.gethostname() in node_names: ++ if not in_use_by or socket.gethostname() in node_names: + return self._call_local_vhd_util(local_method, device_path, *args[2:], **kwargs) + except ErofsLinstorCallException as e: + local_e = e.cmd_err +@@ -88,6 +96,9 @@ def linstorhostcall(local_method, remote_method): + ) + ) + ++ if in_use_by: ++ node_names = {in_use_by} ++ + # B. Execute the plugin on master or slave. + remote_args = { + 'devicePath': device_path, +@@ -319,6 +330,8 @@ class LinstorVhdUtil: + except util.CommandException as e: + if e.code == errno.EROFS or e.code == EMEDIUMTYPE: + raise ErofsLinstorCallException(e) # Break retry calls. ++ if e.code == errno.ENOENT: ++ raise NoPathLinstorCallException(e) + raise e + # Retry only locally if it's not an EROFS exception. + return util.retry(local_call, 5, 2, exceptions=[util.CommandException]) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 4662043c..5ab83c41 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1410,12 +1410,12 @@ class LinstorVolumeManager(object): + The disk must be up to data to be used. + :param str volume_uuid: The volume to use. + :return: The available nodes. +- :rtype: tuple(set(str), bool) ++ :rtype: tuple(set(str), str) + """ + + volume_name = self.get_volume_name(volume_uuid) + +- in_use = False ++ in_use_by = None + node_names = set() + + resource_states = filter( +@@ -1428,9 +1428,9 @@ class LinstorVolumeManager(object): + if volume_state.disk_state == 'UpToDate': + node_names.add(resource_state.node_name) + if resource_state.in_use: +- in_use = True ++ in_use_by = resource_state.node_name + +- return (node_names, in_use) ++ return (node_names, in_use_by) + + def invalidate_resource_cache(self): + """ diff --git a/SOURCES/0112-fix-LinstorSR-replace-bad-param-in-detach_thin-impl.patch b/SOURCES/0112-fix-LinstorSR-replace-bad-param-in-detach_thin-impl.patch new file mode 100644 index 0000000..446ca15 --- /dev/null +++ b/SOURCES/0112-fix-LinstorSR-replace-bad-param-in-detach_thin-impl.patch @@ -0,0 +1,25 @@ +From 099f7a633435610b5e09c487ef0d42473d8fd9a9 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 9 Mar 2023 17:06:59 +0100 +Subject: [PATCH 112/177] fix(LinstorSR): replace bad param in detach_thin impl + +To get the physical size, the volume UUID must be used, not the path. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 10e0f543..c42f07d6 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -215,7 +215,7 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + new_volume_size = LinstorVolumeManager.round_up_volume_size( + # TODO: Replace pylint comment with this feature when possible: + # https://github.com/PyCQA/pylint/pull/2926 +- LinstorVhdUtil(session, linstor).get_size_phys(device_path) # pylint: disable = E1120 ++ LinstorVhdUtil(session, linstor).get_size_phys(vdi_uuid) # pylint: disable = E1120 + ) + + volume_info = linstor.get_volume_info(vdi_uuid) diff --git a/SOURCES/0113-fix-linstorvolumemanager-remove-usage-of-realpath.patch b/SOURCES/0113-fix-linstorvolumemanager-remove-usage-of-realpath.patch new file mode 100644 index 0000000..7a2a0be --- /dev/null +++ b/SOURCES/0113-fix-linstorvolumemanager-remove-usage-of-realpath.patch @@ -0,0 +1,53 @@ +From 1d84de36cf45bbaba8fa0ea993bba09e1c18616d Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 10 Mar 2023 18:11:10 +0100 +Subject: [PATCH 113/177] fix(linstorvolumemanager): remove usage of realpath + +Because a diskless DRBD path not always exist, get_volume_name_from_device_path can fail. +It's easy to reproduce using > 4 hosts and with a call to linstorvhdutil.get_vhd_info: +This problem can occur if the parent of a VHD is not on the same machine and if this parent doesn't have a DRBD path locally. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 23 ++++++----------------- + 1 file changed, 6 insertions(+), 17 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 5ab83c41..8befb33f 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -914,28 +914,17 @@ class LinstorVolumeManager(object): + + def get_volume_name_from_device_path(self, device_path): + """ +- Get the volume name of a device_path on the current host. ++ Get the volume name of a device_path. + :param str device_path: The dev path to find the volume name. +- :return: The volume name of the local device path. ++ :return: The volume name of the device path. + :rtype: str + """ + +- node_name = socket.gethostname() +- +- resources = filter( +- lambda resource: resource.node_name == node_name, +- self._get_resource_cache().resources +- ) +- +- real_device_path = os.path.realpath(device_path) +- for resource in resources: +- if resource.volumes[0].device_path == real_device_path: +- return resource.name ++ assert device_path.startswith(DRBD_BY_RES_PATH) + +- raise LinstorVolumeManagerError( +- 'Unable to find volume name from dev path `{}`' +- .format(device_path) +- ) ++ res_name_end = device_path.find('/', len(DRBD_BY_RES_PATH)) ++ assert res_name_end != -1 ++ return device_path[len(DRBD_BY_RES_PATH):res_name_end] + + def update_volume_uuid(self, volume_uuid, new_volume_uuid, force=False): + """ diff --git a/SOURCES/0114-fix-linstorvhdutil-avoid-parent-path-resolution.patch b/SOURCES/0114-fix-linstorvhdutil-avoid-parent-path-resolution.patch new file mode 100644 index 0000000..bbc1c4e --- /dev/null +++ b/SOURCES/0114-fix-linstorvhdutil-avoid-parent-path-resolution.patch @@ -0,0 +1,108 @@ +From e7550ad1ca7033adb3bb6dbc3c6a79fa2386d90b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 16 Mar 2023 18:54:46 +0100 +Subject: [PATCH 114/177] fix(linstorvhdutil): avoid parent path resolution + +When many hosts are used (>= 4), we can fail to get +VHD info (with parent option) because the local parent VDI +path can be absent (no DRBD diskless path). So it's necessary +to deactivate parent resolution: +- vhdutil has been patched to support that +- vhdutil returns a relative path now when "-u" option is used + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 2 +- + drivers/linstorvhdutil.py | 9 ++++++--- + drivers/linstorvolumemanager.py | 13 ++++++++++--- + drivers/vhdutil.py | 5 ++++- + 4 files changed, 21 insertions(+), 8 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 7e34b5f6..9e96aaca 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -400,7 +400,7 @@ def get_vhd_info(session, args): + ) + + vhd_info = vhdutil.getVHDInfo( +- device_path, extract_uuid, include_parent ++ device_path, extract_uuid, include_parent, False + ) + return json.dumps(vhd_info.__dict__) + except Exception as e: +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index c1b817d7..8b6985d9 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -143,8 +143,8 @@ class LinstorVhdUtil: + + def check(self, vdi_uuid, ignore_missing_footer=False, fast=False): + kwargs = { +- 'ignoreMissingFooter': str(ignore_missing_footer), +- 'fast': str(fast) ++ 'ignoreMissingFooter': ignore_missing_footer, ++ 'fast': fast + } + return self._check(vdi_uuid, **kwargs) # pylint: disable = E1123 + +@@ -153,7 +153,10 @@ class LinstorVhdUtil: + return distutils.util.strtobool(response) + + def get_vhd_info(self, vdi_uuid, include_parent=True): +- kwargs = {'includeParent': str(include_parent)} ++ kwargs = { ++ 'includeParent': include_parent, ++ 'resolveParent': False ++ } + # TODO: Replace pylint comment with this feature when possible: + # https://github.com/PyCQA/pylint/pull/2926 + return self._get_vhd_info(vdi_uuid, self._extract_uuid, **kwargs) # pylint: disable = E1123 +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 8befb33f..91db3d80 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -920,11 +920,18 @@ class LinstorVolumeManager(object): + :rtype: str + """ + +- assert device_path.startswith(DRBD_BY_RES_PATH) ++ # Assume that we have a path like this: ++ # - "/dev/drbd/by-res/xcp-volume-/0" ++ # - "../xcp-volume-/0" ++ if device_path.startswith(DRBD_BY_RES_PATH): ++ prefix_len = len(DRBD_BY_RES_PATH) ++ else: ++ assert device_path.startswith('../') ++ prefix_len = 3 + +- res_name_end = device_path.find('/', len(DRBD_BY_RES_PATH)) ++ res_name_end = device_path.find('/', prefix_len) + assert res_name_end != -1 +- return device_path[len(DRBD_BY_RES_PATH):res_name_end] ++ return device_path[prefix_len:res_name_end] + + def update_volume_uuid(self, volume_uuid, new_volume_uuid, force=False): + """ +diff --git a/drivers/vhdutil.py b/drivers/vhdutil.py +index d75edb11..48337f87 100755 +--- a/drivers/vhdutil.py ++++ b/drivers/vhdutil.py +@@ -100,13 +100,16 @@ def fullSizeVHD(virtual_size): + def ioretry(cmd, errlist=[errno.EIO, errno.EAGAIN]): + return util.ioretry(lambda: util.pread2(cmd), errlist) + +-def getVHDInfo(path, extractUuidFunction, includeParent = True): ++def getVHDInfo(path, extractUuidFunction, includeParent=True, resolveParent=True): + """Get the VHD info. The parent info may optionally be omitted: vhd-util + tries to verify the parent by opening it, which results in error if the VHD + resides on an inactive LV""" + opts = "-vsf" + if includeParent: + opts += "p" ++ if not resolveParent: ++ opts += "u" ++ + cmd = [VHD_UTIL, "query", OPT_LOG_ERR, opts, "-n", path] + ret = ioretry(cmd) + fields = ret.strip().split('\n') diff --git a/SOURCES/0115-fix-LinstorSR-create-parent-path-during-attach.patch b/SOURCES/0115-fix-LinstorSR-create-parent-path-during-attach.patch new file mode 100644 index 0000000..f2f3337 --- /dev/null +++ b/SOURCES/0115-fix-LinstorSR-create-parent-path-during-attach.patch @@ -0,0 +1,38 @@ +From 03a0c875d8ee453743237f36e9dcd9715f4a07d2 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 17 Mar 2023 12:06:08 +0100 +Subject: [PATCH 115/177] fix(LinstorSR): create parent path during attach + +It's necessary to force DRBD diskless path creation when +a VDI is attached. Otherwise the attach can fail on pool with +>= 4 hosts. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 11 ++++++----- + 1 file changed, 6 insertions(+), 5 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index c42f07d6..48feec7a 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1883,13 +1883,14 @@ class LinstorVDI(VDI.VDI): + ): + return self._attach_using_http_nbd() + +- if not util.pathexists(self.path): +- # Ensure we have a path... +- self._linstor.get_device_path(vdi_uuid) +- if not util.pathexists(self.path): ++ # Ensure we have a path... ++ while vdi_uuid: ++ path = self._linstor.get_device_path(vdi_uuid) ++ if not util.pathexists(path): + raise xs_errors.XenError( +- 'VDIUnavailable', opterr='Could not find: {}'.format(self.path) ++ 'VDIUnavailable', opterr='Could not find: {}'.format(path) + ) ++ vdi_uuid = self.sr._vhdutil.get_vhd_info(vdi_uuid).parentUuid + + self.attached = True + return VDI.VDI.attach(self, self.sr.uuid, self.uuid) diff --git a/SOURCES/0116-fix-LinstorSR-retry-if-we-can-t-build-volume-cache.patch b/SOURCES/0116-fix-LinstorSR-retry-if-we-can-t-build-volume-cache.patch new file mode 100644 index 0000000..54aefd2 --- /dev/null +++ b/SOURCES/0116-fix-LinstorSR-retry-if-we-can-t-build-volume-cache.patch @@ -0,0 +1,30 @@ +From 601a07072f2ce0bb275c31a0daa5d803d9cfc885 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 3 Apr 2023 10:03:57 +0200 +Subject: [PATCH 116/177] fix(LinstorSR): retry if we can't build volume cache + +Otherwise after SR creation, the master PBD can be unplugged. +See: https://xcp-ng.org/forum/post/60726 + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 6 +++++- + 1 file changed, 5 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 48feec7a..324033a0 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1553,7 +1553,11 @@ class LinstorSR(SR.SR): + def _create_linstor_cache(self): + self._all_volume_metadata_cache = \ + self._linstor.get_volumes_with_metadata() +- self._all_volume_info_cache = self._linstor.get_volumes_with_info() ++ self._all_volume_info_cache = util.retry( ++ self._linstor.get_volumes_with_info, ++ maxretry=10, ++ period=1 ++ ) + + def _destroy_linstor_cache(self): + self._all_volume_info_cache = None diff --git a/SOURCES/0117-fix-linstorvolumemanager-reduce-peer-slots-param-to-.patch b/SOURCES/0117-fix-linstorvolumemanager-reduce-peer-slots-param-to-.patch new file mode 100644 index 0000000..371b903 --- /dev/null +++ b/SOURCES/0117-fix-linstorvolumemanager-reduce-peer-slots-param-to-.patch @@ -0,0 +1,54 @@ +From 2d862e93882ff9d63aa75c39d671825aeeefeba7 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 25 Apr 2023 10:46:00 +0200 +Subject: [PATCH 117/177] fix(linstorvolumemanager): reduce peer-slots param to + 3 + +Because we use 3 backing disks at most, it's useless to increase the default linstor limit (8). +Diskless is a resource that does not count in the peer-slots param. + +Note: this change is important to reduce the RAM usage, see => +https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#s-meta-data-size +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 91db3d80..6f20c02c 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2089,7 +2089,7 @@ class LinstorVolumeManager(object): + volume_uuid, + self._group_name + ) +- self._increase_volume_peer_slots(self._linstor, volume_name) ++ self._configure_volume_peer_slots(self._linstor, volume_name) + + def clean(): + try: +@@ -2475,12 +2475,12 @@ class LinstorVolumeManager(object): + ) + + @classmethod +- def _increase_volume_peer_slots(cls, lin, volume_name): +- result = lin.resource_dfn_modify(volume_name, {}, peer_slots=31) ++ def _configure_volume_peer_slots(cls, lin, volume_name): ++ result = lin.resource_dfn_modify(volume_name, {}, peer_slots=3) + error_str = cls._get_error_str(result) + if error_str: + raise LinstorVolumeManagerError( +- 'Could not increase volume peer slots of {}: {}' ++ 'Could not configure volume peer slots of {}: {}' + .format(volume_name, error_str) + ) + +@@ -2581,7 +2581,7 @@ class LinstorVolumeManager(object): + vlm_sizes=['{}B'.format(size)], + definitions_only=True + ), DATABASE_VOLUME_NAME, group_name) +- cls._increase_volume_peer_slots(lin, DATABASE_VOLUME_NAME) ++ cls._configure_volume_peer_slots(lin, DATABASE_VOLUME_NAME) + + # Create real resources on the first nodes. + resources = [] diff --git a/SOURCES/0118-fix-LinstorSR-attach-a-valid-XAPI-session-is_open-is.patch b/SOURCES/0118-fix-LinstorSR-attach-a-valid-XAPI-session-is_open-is.patch new file mode 100644 index 0000000..fd807a8 --- /dev/null +++ b/SOURCES/0118-fix-LinstorSR-attach-a-valid-XAPI-session-is_open-is.patch @@ -0,0 +1,28 @@ +From e13b45dc17a75702f4a6999e9bc28105ee0e48fb Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 25 Apr 2023 11:20:55 +0200 +Subject: [PATCH 118/177] fix(LinstorSR): attach a valid XAPI session is_open + is called + +Signed-off-by: Ronan Abhamon +--- + drivers/on_slave.py | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/drivers/on_slave.py b/drivers/on_slave.py +index bbef4f7f..a1976607 100755 +--- a/drivers/on_slave.py ++++ b/drivers/on_slave.py +@@ -125,6 +125,12 @@ def _is_open(session, args): + + driver = SR.driver(srType) + sr = driver(cmd, sr_uuid) ++ ++ # session_ref param is required to have a valid session when SR object is created. ++ # It's not the case here, so attach the current session object to make LinstorSR happy. ++ if srType == 'linstor': ++ sr.session = session ++ + vdi = sr.vdi(vdiUuid) + tapdisk = blktap2.Tapdisk.find_by_path(vdi.path) + util.SMlog("Tapdisk for %s: %s" % (vdi.path, tapdisk)) diff --git a/SOURCES/0119-fix-LinstorSR-ensure-we-always-have-a-DRBD-path-to-s.patch b/SOURCES/0119-fix-LinstorSR-ensure-we-always-have-a-DRBD-path-to-s.patch new file mode 100644 index 0000000..efa18e3 --- /dev/null +++ b/SOURCES/0119-fix-LinstorSR-ensure-we-always-have-a-DRBD-path-to-s.patch @@ -0,0 +1,25 @@ +From 05bc50d10da120a065fff476a0a0532058d8192e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 28 Apr 2023 10:43:27 +0200 +Subject: [PATCH 119/177] fix(LinstorSR): ensure we always have a DRBD path to + snap + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 324033a0..8c0b007d 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -2392,6 +2392,9 @@ class LinstorVDI(VDI.VDI): + elif depth >= vhdutil.MAX_CHAIN_SIZE: + raise xs_errors.XenError('SnapshotChainTooLong') + ++ # Ensure we have a valid path if we don't have a local diskful. ++ self.sr._linstor.get_device_path(self.uuid) ++ + volume_path = self.path + if not util.pathexists(volume_path): + raise xs_errors.XenError( diff --git a/SOURCES/0120-fix-LinstorSR-remove-hosts-ips-param.patch b/SOURCES/0120-fix-LinstorSR-remove-hosts-ips-param.patch new file mode 100644 index 0000000..f4aef61 --- /dev/null +++ b/SOURCES/0120-fix-LinstorSR-remove-hosts-ips-param.patch @@ -0,0 +1,165 @@ +From c7030966d57e33747b6dde463f35e9d7d79e6fb5 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 30 May 2023 11:19:13 +0200 +Subject: [PATCH 120/177] fix(LinstorSR): remove hosts/ips param + +--- + drivers/LinstorSR.py | 47 +++++---------------------------- + drivers/linstorvolumemanager.py | 11 +++----- + 2 files changed, 11 insertions(+), 47 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 8c0b007d..7a9cbac6 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -100,8 +100,6 @@ CAPABILITIES = [ + + CONFIGURATION = [ + ['group-name', 'LVM group name'], +- ['hosts', 'host names to use'], +- ['ips', 'ips to use (optional, defaults to management networks)'], + ['redundancy', 'replication count'], + ['provisioning', '"thin" or "thick" are accepted (optional, defaults to thin)'], + ['monitor-db-quorum', 'disable controller when only one host is online (optional, defaults to true)'] +@@ -353,7 +351,6 @@ def activate_lvm_group(group_name): + # Usage example: + # xe sr-create type=linstor name-label=linstor-sr + # host-uuid=d2deba7a-c5ad-4de1-9a20-5c8df3343e93 +-# device-config:hosts=node-linstor1,node-linstor2,node-linstor3 + # device-config:group-name=vg_loop device-config:redundancy=2 + + +@@ -385,8 +382,6 @@ class LinstorSR(SR.SR): + ) + + # Check parameters. +- if 'hosts' not in self.dconf or not self.dconf['hosts']: +- raise xs_errors.XenError('LinstorConfigHostsMissing') + if 'group-name' not in self.dconf or not self.dconf['group-name']: + raise xs_errors.XenError('LinstorConfigGroupNameMissing') + if 'redundancy' not in self.dconf or not self.dconf['redundancy']: +@@ -431,12 +426,6 @@ class LinstorSR(SR.SR): + self.lock = Lock(vhdutil.LOCK_TYPE_SR, self.uuid) + self.sr_vditype = SR.DEFAULT_TAP + +- self._hosts = list(set(self.dconf['hosts'].split(','))) +- if 'ips' not in self.dconf or not self.dconf['ips']: +- self._ips = None +- else: +- self._ips = self.dconf['ips'].split(',') +- + if self.cmd == 'sr_create': + self._redundancy = int(self.dconf['redundancy']) or 1 + self._linstor = None # Ensure that LINSTOR attribute exists. +@@ -647,7 +636,8 @@ class LinstorSR(SR.SR): + def create(self, uuid, size): + util.SMlog('LinstorSR.create for {}'.format(self.uuid)) + +- if self._redundancy > len(self._hosts): ++ host_adresses = util.get_host_addresses(self.session) ++ if self._redundancy > len(host_adresses): + raise xs_errors.XenError( + 'LinstorSRCreate', + opterr='Redundancy greater than host count' +@@ -676,39 +666,17 @@ class LinstorSR(SR.SR): + ) + + online_hosts = util.get_online_hosts(self.session) +- if len(online_hosts) < len(self._hosts): ++ if len(online_hosts) < len(host_adresses): + raise xs_errors.XenError( + 'LinstorSRCreate', + opterr='Not enough online hosts' + ) + + ips = {} +- if not self._ips: +- for host in online_hosts: +- record = self.session.xenapi.host.get_record(host) +- hostname = record['hostname'] +- if hostname in self._hosts: +- ips[hostname] = record['address'] +- elif len(self._ips) != len(self._hosts): +- raise xs_errors.XenError( +- 'LinstorSRCreate', +- opterr='ips must be equal to host count' +- ) +- else: +- for host in online_hosts: +- record = self.session.xenapi.host.get_record(host) +- hostname = record['hostname'] +- try: +- index = self._hosts.index(hostname) +- ips[hostname] = self._ips[index] +- except ValueError as e: +- pass +- +- if len(ips) != len(self._hosts): +- raise xs_errors.XenError( +- 'LinstorSRCreate', +- opterr='Not enough online hosts' +- ) ++ for host_ref in online_hosts: ++ record = self.session.xenapi.host.get_record(host_ref) ++ hostname = record['hostname'] ++ ips[hostname] = record['address'] + + # Ensure ports are opened and LINSTOR satellites + # are activated. In the same time the drbd-reactor instances +@@ -720,7 +688,6 @@ class LinstorSR(SR.SR): + try: + self._linstor = LinstorVolumeManager.create_sr( + self._group_name, +- self._hosts, + ips, + self._redundancy, + thin_provisioning=self._provisioning == 'thin', +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 6f20c02c..464ab2ce 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1588,14 +1588,13 @@ class LinstorVolumeManager(object): + + @classmethod + def create_sr( +- cls, group_name, node_names, ips, redundancy, ++ cls, group_name, ips, redundancy, + thin_provisioning, auto_quorum, + logger=default_logger.__func__ + ): + """ + Create a new SR on the given nodes. + :param str group_name: The SR group_name to use. +- :param list[str] node_names: String list of nodes. + :param set(str) ips: Node ips. + :param int redundancy: How many copy of volumes should we store? + :param bool thin_provisioning: Use thin or thick provisioning. +@@ -1609,7 +1608,6 @@ class LinstorVolumeManager(object): + cls._start_controller(start=True) + sr = cls._create_sr( + group_name, +- node_names, + ips, + redundancy, + thin_provisioning, +@@ -1630,7 +1628,7 @@ class LinstorVolumeManager(object): + + @classmethod + def _create_sr( +- cls, group_name, node_names, ips, redundancy, ++ cls, group_name, ips, redundancy, + thin_provisioning, auto_quorum, + logger=default_logger.__func__ + ): +@@ -1639,9 +1637,8 @@ class LinstorVolumeManager(object): + + lin = cls._create_linstor_instance(uri, keep_uri_unmodified=True) + +- for node_name in node_names: +- ip = ips[node_name] +- ++ node_names = ips.keys() ++ for node_name, ip in ips.iteritems(): + while True: + # Try to create node. + result = lin.node_create( diff --git a/SOURCES/0121-fix-LinstorSR-compute-correctly-SR-size-using-pool-c.patch b/SOURCES/0121-fix-LinstorSR-compute-correctly-SR-size-using-pool-c.patch new file mode 100644 index 0000000..0039908 --- /dev/null +++ b/SOURCES/0121-fix-LinstorSR-compute-correctly-SR-size-using-pool-c.patch @@ -0,0 +1,90 @@ +From 54fa0adaf9bd385ca6e3f7b2eab5616f10b458be Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 1 Jun 2023 17:40:37 +0200 +Subject: [PATCH 121/177] fix(LinstorSR): compute correctly SR size using pool + count + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 4 +-- + drivers/linstorvolumemanager.py | 45 +++++++++++++++++---------------- + 2 files changed, 25 insertions(+), 24 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 7a9cbac6..f6c43569 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1065,8 +1065,8 @@ class LinstorSR(SR.SR): + def _update_physical_size(self): + # We use the size of the smallest disk, this is an approximation that + # ensures the displayed physical size is reachable by the user. +- self.physical_size = \ +- self._linstor.min_physical_size * len(self._hosts) / \ ++ (min_physical_size, pool_count) = self._linstor.get_min_physical_size() ++ self.physical_size = min_physical_size * pool_count / \ + self._linstor.redundancy + + self.physical_utilisation = self._linstor.allocated_volume_size +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 464ab2ce..ee637ae2 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -492,28 +492,6 @@ class LinstorVolumeManager(object): + """ + return self._compute_size('free_capacity') + +- @property +- def min_physical_size(self): +- """ +- Give the minimum physical size of the SR. +- I.e. the size of the smallest disk. +- :return: The physical min size. +- :rtype: int +- """ +- size = None +- for pool in self._get_storage_pools(force=True): +- space = pool.free_space +- if space: +- current_size = space.total_capacity +- if current_size < 0: +- raise LinstorVolumeManagerError( +- 'Failed to get pool total_capacity attr of `{}`' +- .format(pool.node_name) +- ) +- if size is None or current_size < size: +- size = current_size +- return (size or 0) * 1024 +- + @property + def allocated_volume_size(self): + """ +@@ -554,6 +532,29 @@ class LinstorVolumeManager(object): + + return total_size * 1024 + ++ def get_min_physical_size(self): ++ """ ++ Give the minimum physical size of the SR. ++ I.e. the size of the smallest disk + the number of pools. ++ :return: The physical min size. ++ :rtype: tuple(int, int) ++ """ ++ size = None ++ pool_count = 0 ++ for pool in self._get_storage_pools(force=True): ++ space = pool.free_space ++ if space: ++ pool_count += 1 ++ current_size = space.total_capacity ++ if current_size < 0: ++ raise LinstorVolumeManagerError( ++ 'Failed to get pool total_capacity attr of `{}`' ++ .format(pool.node_name) ++ ) ++ if size is None or current_size < size: ++ size = current_size ++ return (pool_count, (size or 0) * 1024) ++ + @property + def metadata(self): + """ diff --git a/SOURCES/0122-fix-blktap2-ensure-we-can-import-this-module-when-LI.patch b/SOURCES/0122-fix-blktap2-ensure-we-can-import-this-module-when-LI.patch new file mode 100644 index 0000000..becc395 --- /dev/null +++ b/SOURCES/0122-fix-blktap2-ensure-we-can-import-this-module-when-LI.patch @@ -0,0 +1,45 @@ +From 3cf1299c59cab4cf4f92a7a98db1757b0c876996 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 6 Jun 2023 11:50:54 +0200 +Subject: [PATCH 122/177] fix(blktap2): ensure we can import this module when + LINSTOR is not installed + +Signed-off-by: Ronan Abhamon +--- + drivers/blktap2.py | 9 +++++++-- + 1 file changed, 7 insertions(+), 2 deletions(-) + +diff --git a/drivers/blktap2.py b/drivers/blktap2.py +index 370f7fb8..fd7c643d 100755 +--- a/drivers/blktap2.py ++++ b/drivers/blktap2.py +@@ -36,7 +36,6 @@ import json + import xs_errors + import XenAPI + import scsiutil +-from linstorvolumemanager import log_drbd_openers + from syslog import openlog, syslog + from stat import * # S_ISBLK(), ... + import nfs +@@ -51,6 +50,12 @@ from xmlrpclib import ServerProxy, Transport + from socket import socket, AF_UNIX, SOCK_STREAM + from httplib import HTTP, HTTPConnection + ++try: ++ from linstorvolumemanager import log_drbd_openers ++ LINSTOR_AVAILABLE = True ++except ImportError: ++ LINSTOR_AVAILABLE = False ++ + PLUGIN_TAP_PAUSE = "tapdisk-pause" + + SOCKPATH = "/var/xapi/xcp-rrdd" +@@ -832,7 +837,7 @@ class Tapdisk(object): + retry_open += 1 + time.sleep(1) + continue +- if err == errno.EROFS: ++ if LINSTOR_AVAILABLE and err == errno.EROFS: + log_drbd_openers(path) + raise + try: diff --git a/SOURCES/0123-fix-LinstorSR-ensure-volume-cache-can-be-recreated.patch b/SOURCES/0123-fix-LinstorSR-ensure-volume-cache-can-be-recreated.patch new file mode 100644 index 0000000..4cf95c0 --- /dev/null +++ b/SOURCES/0123-fix-LinstorSR-ensure-volume-cache-can-be-recreated.patch @@ -0,0 +1,109 @@ +From b06882d0cdb391ca92cd08146b935648cbb6ef6e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 21 Jun 2023 14:10:18 +0200 +Subject: [PATCH 123/177] fix(LinstorSR): ensure volume cache can be recreated + +After SR creation we may fail to load volumes with this exception: +"Failed to get usable size of..." and so we can't plug the master PBD. + +Regardless of the retry timeout, the only solution to fetch the usable +size of the DB is to recreate the connection to the LINSTOR API. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 64 +++++++++++++++++++++++++++----------------- + 1 file changed, 40 insertions(+), 24 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index f6c43569..0bccc167 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -523,28 +523,7 @@ class LinstorSR(SR.SR): + + if self.srcmd.cmd != 'sr_create' and self.srcmd.cmd != 'sr_detach': + try: +- controller_uri = get_controller_uri() +- +- self._journaler = LinstorJournaler( +- controller_uri, self._group_name, logger=util.SMlog +- ) +- +- # Try to open SR if exists. +- # We can repair only if we are on the master AND if +- # we are trying to execute an exclusive operation. +- # Otherwise we could try to delete a VDI being created or +- # during a snapshot. An exclusive op is the guarantee that +- # the SR is locked. +- self._linstor = LinstorVolumeManager( +- controller_uri, +- self._group_name, +- repair=( +- self._is_master and +- self.srcmd.cmd in self.ops_exclusive +- ), +- logger=util.SMlog +- ) +- self._vhdutil = LinstorVhdUtil(self.session, self._linstor) ++ self._reconnect() + except Exception as e: + raise xs_errors.XenError('SRUnavailable', opterr=str(e)) + +@@ -1518,12 +1497,25 @@ class LinstorSR(SR.SR): + # -------------------------------------------------------------------------- + + def _create_linstor_cache(self): ++ # TODO: use a nonlocal with python3. ++ class context: ++ reconnect = False ++ ++ def create_cache(): ++ try: ++ if context.reconnect: ++ self._reconnect() ++ return self._linstor.get_volumes_with_info() ++ except Exception as e: ++ context.reconnect = True ++ raise e ++ + self._all_volume_metadata_cache = \ + self._linstor.get_volumes_with_metadata() + self._all_volume_info_cache = util.retry( +- self._linstor.get_volumes_with_info, ++ create_cache, + maxretry=10, +- period=1 ++ period=3 + ) + + def _destroy_linstor_cache(self): +@@ -1534,6 +1526,30 @@ class LinstorSR(SR.SR): + # Misc. + # -------------------------------------------------------------------------- + ++ def _reconnect(self): ++ controller_uri = get_controller_uri() ++ ++ self._journaler = LinstorJournaler( ++ controller_uri, self._group_name, logger=util.SMlog ++ ) ++ ++ # Try to open SR if exists. ++ # We can repair only if we are on the master AND if ++ # we are trying to execute an exclusive operation. ++ # Otherwise we could try to delete a VDI being created or ++ # during a snapshot. An exclusive op is the guarantee that ++ # the SR is locked. ++ self._linstor = LinstorVolumeManager( ++ controller_uri, ++ self._group_name, ++ repair=( ++ self._is_master and ++ self.srcmd.cmd in self.ops_exclusive ++ ), ++ logger=util.SMlog ++ ) ++ self._vhdutil = LinstorVhdUtil(self.session, self._linstor) ++ + def _ensure_space_available(self, amount_needed): + space_available = self._linstor.max_volume_size_allowed + if (space_available < amount_needed): diff --git a/SOURCES/0124-fix-linstor-manager-remove-dead-useless-code-in-add-.patch b/SOURCES/0124-fix-linstor-manager-remove-dead-useless-code-in-add-.patch new file mode 100644 index 0000000..3424fe3 --- /dev/null +++ b/SOURCES/0124-fix-linstor-manager-remove-dead-useless-code-in-add-.patch @@ -0,0 +1,206 @@ +From 085905977ebf04aea99cc2301aaea15fd1f8ee4f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 20 Jul 2023 10:46:33 +0200 +Subject: [PATCH 124/177] fix(linstor-manager): remove dead/useless code in + add/remove_host helpers + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 128 +++++----------------------------------- + 1 file changed, 15 insertions(+), 113 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 9e96aaca..45201eed 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -555,7 +555,7 @@ def has_controller_running(session, args): + def add_host(session, args): + group_name = args['groupName'] + +- # 1. Find SR and PBDs. ++ # 1. Find all LINSTOR SRs and PBDs. + srs = dict() + for sr_ref, sr in session.xenapi.SR.get_all_records().items(): + if sr.get('type') == 'linstor': +@@ -597,7 +597,6 @@ def add_host(session, args): + node_name = socket.gethostname() + has_node = linstor.has_node(node_name) + +- pbd_id = 0 + new_pbd_ref = None + + try: +@@ -610,66 +609,30 @@ def add_host(session, args): + if not has_node: + linstor.create_node(node_name, util.get_this_host_address(session)) + +- # 6. Recreate PBDs. +- # Use the redundancy given by Linstor instead of smapi config. +- redundancy = linstor.redundancy +- default_device_config = None ++ # 6. Try to create PBD. + this_host = util.get_this_host_ref(session) + create_new_pbd = True + + assert pbds +- pbds = pbds.items() +- for pbd_ref, pbd in pbds: +- device_config = pbd['device_config'] +- +- hosts = filter( +- lambda host: len(host.strip()), +- device_config.get('hosts', []).split(',') +- ) +- hosts.append(node_name) +- hosts = ','.join(list(set(hosts))) ++ for pbd in pbds.values(): ++ if pbd['host'] == this_host: ++ create_new_pbd = False ++ break + ++ device_config = pbd['device_config'] + # Should be the same on all hosts. + provisioning = device_config['provisioning'] + +- if not default_device_config: +- default_device_config = { +- 'group-name': group_name, +- 'redundancy': redundancy, +- 'hosts': hosts, +- 'provisioning': provisioning +- } +- +- if pbd['currently_attached']: +- session.xenapi.PBD.unplug(pbd_ref) +- session.xenapi.PBD.destroy(pbd_ref) +- pbd_id += 1 +- +- host = pbd['host'] +- if host == this_host: +- create_new_pbd = False +- +- pbd_ref = session.xenapi.PBD.create({ +- 'host': host, +- 'SR': sr_ref, +- 'device_config': { +- 'group-name': group_name, +- 'redundancy': redundancy, +- 'hosts': hosts, +- 'provisioning': provisioning +- } +- }) +- try: +- session.xenapi.PBD.plug(pbd_ref) +- except Exception as e: +- util.SMlog('Failed to replug PBD: {}'.format(e)) +- + # 7. Create new PBD. + if create_new_pbd: + new_pbd_ref = session.xenapi.PBD.create({ + 'host': this_host, + 'SR': sr_ref, +- 'device_config': default_device_config ++ 'device_config': { ++ 'group-name': group_name, ++ 'redundancy': linstor.redundancy, ++ 'provisioning': provisioning ++ } + }) + try: + session.xenapi.PBD.plug(new_pbd_ref) +@@ -685,38 +648,6 @@ def add_host(session, args): + except Exception: + pass + +- for pbd_ref, pbd in pbds[:pbd_id]: +- try: +- session.xenapi.PBD.unplug(pbd_ref) +- except Exception: +- pass +- +- try: +- session.xenapi.PBD.destroy(pbd_ref) +- except Exception: +- pass +- +- try: +- device_config = pbd['device_config'] +- session.xenapi.PBD.create({ +- 'host': host, +- 'SR': sr_ref, +- 'device_config': { +- 'group-name': group_name, +- 'redundancy': redundancy, +- 'hosts': device_config['hosts'], +- 'provisioning': device_config['provisioning'] +- } +- }) +- except Exception as pbd_error: +- util.SMlog('Failed to recreate PBD: {}'.format(pbd_error)) +- pass +- +- try: +- session.xenapi.PBD.plug(pbd_ref) +- except Exception: +- pass +- + if new_pbd_ref: + try: + session.xenapi.PBD.unplug(new_pbd_ref) +@@ -743,7 +674,7 @@ def add_host(session, args): + def remove_host(session, args): + group_name = args['groupName'] + +- # 1. Find SRs and PBDs. ++ # 1. Find all LINSTOR SRs and PBDs. + srs = dict() + for sr_ref, sr in session.xenapi.SR.get_all_records().items(): + if sr.get('type') == 'linstor': +@@ -772,45 +703,16 @@ def remove_host(session, args): + if linstor.has_node(node_name): + raise Exception('Failed to remove node! Unknown error.') + +- redundancy = linstor.redundancy + this_host = util.get_this_host_ref(session) + +- # 3. Update PBDs. ++ # 3. Remove PBD. + for pbd_ref, pbd in pbds.items(): + host = pbd['host'] + if host == this_host: + if pbd['currently_attached']: + session.xenapi.PBD.unplug(pbd_ref) + session.xenapi.PBD.destroy(pbd_ref) +- continue +- +- device_config = pbd['device_config'] +- hosts = device_config.get('hosts', []).split(',') +- try: +- hosts.remove(node_name) +- except Exception as e: +- continue +- hosts = ','.join(list(set(hosts))) +- +- if pbd['currently_attached']: +- session.xenapi.PBD.unplug(pbd_ref) +- session.xenapi.PBD.destroy(pbd_ref) +- +- pbd_ref = session.xenapi.PBD.create({ +- 'host': host, +- 'SR': pbd['SR'], +- 'device_config': { +- 'group-name': group_name, +- 'redundancy': redundancy, +- 'hosts': hosts, +- 'provisioning': device_config['provisioning'] +- } +- }) +- +- try: +- session.xenapi.PBD.plug(pbd_ref) +- except Exception as e: +- util.SMlog('Failed to replug PBD: {}'.format(e)) ++ break + + # 3. Stop services. + try: diff --git a/SOURCES/0125-fix-LinstorSR-Ensure-we-always-have-a-device-path-du.patch b/SOURCES/0125-fix-LinstorSR-Ensure-we-always-have-a-device-path-du.patch new file mode 100644 index 0000000..3cac9a0 --- /dev/null +++ b/SOURCES/0125-fix-LinstorSR-Ensure-we-always-have-a-device-path-du.patch @@ -0,0 +1,56 @@ +From 6dc4f983d955ca094c0bfb580cfda4dbb194ca3d Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 1 Aug 2023 15:16:17 +0200 +Subject: [PATCH 125/177] fix(LinstorSR): Ensure we always have a device path + during leaf-coalesce calls + +So we must not verify that we have a valid DRBD path in the load step, +it can fail on many hosts, instead we must create a diskless path only +during the real coalesce. + +Note: I removed this assert: `assert virtual_size >= volume_size`, +it seems that it's not always true, I suppose the volume size can be +greater than expected due to a bigger allocation in the LVM or DRBD layer. + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 12 +++++------- + 1 file changed, 5 insertions(+), 7 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index f6c4346b..19d03d96 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -1394,11 +1394,6 @@ class LinstorVDI(VDI): + + self.fileName = self.sr._linstor.get_volume_name(self.uuid) + self.path = self.sr._linstor.build_device_path(self.fileName) +- if not util.pathexists(self.path): +- raise util.SMException( +- '{} of {} not found' +- .format(self.fileName, self.uuid) +- ) + + if not info: + try: +@@ -3025,8 +3020,6 @@ class LinstorSR(SR): + parent.sizeVirt + meta_overhead + bitmap_overhead + ) + volume_size = self._linstor.get_volume_size(parent.uuid) +- +- assert virtual_size >= volume_size + return virtual_size - volume_size + + def _hasValidDevicePath(self, uuid): +@@ -3047,6 +3040,11 @@ class LinstorSR(SR): + finally: + self.unlock() + ++ def _prepareCoalesceLeaf(self, vdi): ++ # Move diskless path if necessary. We must have an access ++ # to modify locally the volume. ++ self._linstor.get_device_path(vdi.uuid) ++ + def _handleInterruptedCoalesceLeaf(self): + entries = self.journaler.get_all(VDI.JRN_LEAF) + for uuid, parentUuid in entries.iteritems(): diff --git a/SOURCES/0126-fix-LinstorSR-always-use-lock.acquire-during-attach-.patch b/SOURCES/0126-fix-LinstorSR-always-use-lock.acquire-during-attach-.patch new file mode 100644 index 0000000..ed9d595 --- /dev/null +++ b/SOURCES/0126-fix-LinstorSR-always-use-lock.acquire-during-attach-.patch @@ -0,0 +1,56 @@ +From a4de2c2acab46a3d45ffea3be0818669ed559320 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 16 Aug 2023 12:04:01 +0200 +Subject: [PATCH 126/177] fix(LinstorSR): always use lock.acquire() during + attach/detach + +We can't use a retry range on the lock because we can trigger a bad situation +in the detach step... When the GC has a lock on the SR and we try to acquire +the same lock in a detach call, and if we can't get this lock after 20 seconds, +the consequences are very bad: +- Many tapdisk instances of the same VDI can be created on two hosts +- The VDI info are not updated correctly +- And this issue is not immediately visible + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 12 ++---------- + 1 file changed, 2 insertions(+), 10 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 0bccc167..98919a48 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -142,14 +142,6 @@ def compute_volume_size(virtual_size, image_type): + return LinstorVolumeManager.round_up_volume_size(virtual_size) + + +-def try_lock(lock): +- for i in range(20): +- if lock.acquireNoblock(): +- return +- time.sleep(1) +- raise util.SRBusyException() +- +- + def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + volume_metadata = linstor.get_volume_metadata(vdi_uuid) + image_type = volume_metadata.get(VDI_TYPE_TAG) +@@ -158,7 +150,7 @@ def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + + lock = Lock(vhdutil.LOCK_TYPE_SR, sr_uuid) + try: +- try_lock(lock) ++ lock.acquire() + + device_path = linstor.get_device_path(vdi_uuid) + +@@ -191,7 +183,7 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + + lock = Lock(vhdutil.LOCK_TYPE_SR, sr_uuid) + try: +- try_lock(lock) ++ lock.acquire() + + vdi_ref = session.xenapi.VDI.get_by_uuid(vdi_uuid) + vbds = session.xenapi.VBD.get_all_records_where( diff --git a/SOURCES/0127-fix-LinstorSR-mare-sure-hostnames-are-unique-at-SR-c.patch b/SOURCES/0127-fix-LinstorSR-mare-sure-hostnames-are-unique-at-SR-c.patch new file mode 100644 index 0000000..e91d273 --- /dev/null +++ b/SOURCES/0127-fix-LinstorSR-mare-sure-hostnames-are-unique-at-SR-c.patch @@ -0,0 +1,28 @@ +From a7c3c1602e8210ebd74741d5c8ec2f1ef474918b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 17 Aug 2023 14:52:13 +0200 +Subject: [PATCH 127/177] fix(LinstorSR): mare sure hostnames are unique at SR + creation + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 98919a48..e512487c 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -649,6 +649,12 @@ class LinstorSR(SR.SR): + hostname = record['hostname'] + ips[hostname] = record['address'] + ++ if len(ips) != len(online_hosts): ++ raise xs_errors.XenError( ++ 'LinstorSRCreate', ++ opterr='Multiple hosts with same hostname' ++ ) ++ + # Ensure ports are opened and LINSTOR satellites + # are activated. In the same time the drbd-reactor instances + # must be stopped. diff --git a/SOURCES/0128-fix-LinstorSR-ensure-we-can-attach-non-special-stati.patch b/SOURCES/0128-fix-LinstorSR-ensure-we-can-attach-non-special-stati.patch new file mode 100644 index 0000000..e9af586 --- /dev/null +++ b/SOURCES/0128-fix-LinstorSR-ensure-we-can-attach-non-special-stati.patch @@ -0,0 +1,25 @@ +From 54bca8660c18628ec36c1835b9f14aec80c187a8 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 18 Aug 2023 11:06:56 +0200 +Subject: [PATCH 128/177] fix(LinstorSR): ensure we can attach non-special + static VDIs + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index e512487c..17083703 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -461,6 +461,9 @@ class LinstorSR(SR.SR): + logger=util.SMlog, + attempt_count=attempt_count + ) ++ # Only required if we are attaching from config using a non-special VDI. ++ # I.e. not an HA volume. ++ self._vhdutil = LinstorVhdUtil(self.session, self._linstor) + + controller_uri = get_controller_uri() + if controller_uri: diff --git a/SOURCES/0129-fix-LinstorSR-ensure-we-can-detach-when-deflate-call.patch b/SOURCES/0129-fix-LinstorSR-ensure-we-can-detach-when-deflate-call.patch new file mode 100644 index 0000000..7a58884 --- /dev/null +++ b/SOURCES/0129-fix-LinstorSR-ensure-we-can-detach-when-deflate-call.patch @@ -0,0 +1,82 @@ +From 9a295973c226d411ab4276b5bee8a241dc7c555e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 31 Aug 2023 18:00:27 +0200 +Subject: [PATCH 129/177] fix(LinstorSR): ensure we can detach when deflate + call is not possible + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 45 +++++++++++++++++++++++++++++--------------- + 1 file changed, 30 insertions(+), 15 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 17083703..a6ca8840 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -175,7 +175,7 @@ def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + lock.release() + + +-def detach_thin(session, linstor, sr_uuid, vdi_uuid): ++def detach_thin_impl(session, linstor, sr_uuid, vdi_uuid): + volume_metadata = linstor.get_volume_metadata(vdi_uuid) + image_type = volume_metadata.get(VDI_TYPE_TAG) + if image_type == vhdutil.VDI_TYPE_RAW: +@@ -185,21 +185,26 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + try: + lock.acquire() + +- vdi_ref = session.xenapi.VDI.get_by_uuid(vdi_uuid) +- vbds = session.xenapi.VBD.get_all_records_where( +- 'field "VDI" = "{}"'.format(vdi_ref) +- ) ++ def check_vbd_count(): ++ vdi_ref = session.xenapi.VDI.get_by_uuid(vdi_uuid) ++ vbds = session.xenapi.VBD.get_all_records_where( ++ 'field "VDI" = "{}"'.format(vdi_ref) ++ ) + +- num_plugged = 0 +- for vbd_rec in vbds.values(): +- if vbd_rec['currently_attached']: +- num_plugged += 1 +- if num_plugged > 1: +- raise xs_errors.XenError( +- 'VDIUnavailable', +- opterr='Cannot deflate VDI {}, already used by ' +- 'at least 2 VBDs'.format(vdi_uuid) +- ) ++ num_plugged = 0 ++ for vbd_rec in vbds.values(): ++ if vbd_rec['currently_attached']: ++ num_plugged += 1 ++ if num_plugged > 1: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Cannot deflate VDI {}, already used by ' ++ 'at least 2 VBDs'.format(vdi_uuid) ++ ) ++ ++ # We can have multiple VBDs attached to a VDI during a VM-template clone. ++ # So we use a timeout to ensure that we can detach the volume properly. ++ util.retry(check_vbd_count, maxretry=10, period=1) + + device_path = linstor.get_device_path(vdi_uuid) + new_volume_size = LinstorVolumeManager.round_up_volume_size( +@@ -217,6 +222,16 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + lock.release() + + ++def detach_thin(session, linstor, sr_uuid, vdi_uuid): ++ # This function must always return without errors. ++ # Otherwise it could cause errors in the XAPI regarding the state of the VDI. ++ # It's why we use this `try` block. ++ try: ++ detach_thin_impl(session, linstor, sr_uuid, vdi_uuid) ++ except Exception as e: ++ util.SMlog('Failed to detach properly VDI {}: {}'.format(vdi_uuid, e)) ++ ++ + def inflate(journaler, linstor, vdi_uuid, vdi_path, new_size, old_size): + # Only inflate if the LINSTOR volume capacity is not enough. + new_size = LinstorVolumeManager.round_up_volume_size(new_size) diff --git a/SOURCES/0130-fix-LinstorSR-assume-VDI-is-always-a-VHD-when-the-in.patch b/SOURCES/0130-fix-LinstorSR-assume-VDI-is-always-a-VHD-when-the-in.patch new file mode 100644 index 0000000..610ce8f --- /dev/null +++ b/SOURCES/0130-fix-LinstorSR-assume-VDI-is-always-a-VHD-when-the-in.patch @@ -0,0 +1,31 @@ +From 0065166fc2de0bb1368eb590d6a12762cabdda04 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 28 Sep 2023 16:00:02 +0200 +Subject: [PATCH 130/177] fix(LinstorSR): assume VDI is always a VHD when the + info is missing during cleanup + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 8 ++++---- + 1 file changed, 4 insertions(+), 4 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 19d03d96..5353e9aa 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -2995,11 +2995,11 @@ class LinstorSR(SR): + not list(volumes_metadata[vdi_uuid].items()): + continue # Ignore it, probably deleted. + +- vdi_type = volumes_metadata[vdi_uuid][VDI_TYPE_TAG] +- if vdi_type == vhdutil.VDI_TYPE_VHD: +- info = self._vhdutil.get_vhd_info(vdi_uuid) +- else: ++ vdi_type = volumes_metadata[vdi_uuid].get(VDI_TYPE_TAG) ++ if vdi_type == vhdutil.VDI_TYPE_RAW: + info = None ++ else: ++ info = self._vhdutil.get_vhd_info(vdi_uuid) + except Exception as e: + Util.log( + ' [VDI {}: failed to load VDI info]: {}' diff --git a/SOURCES/0131-fix-LinstorSR-remove-SR-lock-during-thin-attach-deta.patch b/SOURCES/0131-fix-LinstorSR-remove-SR-lock-during-thin-attach-deta.patch new file mode 100644 index 0000000..ce3ebf7 --- /dev/null +++ b/SOURCES/0131-fix-LinstorSR-remove-SR-lock-during-thin-attach-deta.patch @@ -0,0 +1,149 @@ +From f894c7a0cadbed19314bb93e04bdc4a7a6bcb62b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 2 Oct 2023 16:48:49 +0200 +Subject: [PATCH 131/177] fix(LinstorSR): remove SR lock during thin + attach/detach + +This lock is normally useless and can create a dead lock when thin mode is activated: +- A task try to deactivate a volume during a VM shutdown on a slave (so a VDI A is locked). + Then a new task is created on the master host, we try to get the SR lock on the master. +- In parallel a tap-pause is asked from the master to the slave, the master SR lock is now locked. + The tap-pause request is received on the slave, but we can't lock VDI A because it's already + locked. + +So to resume: a dead lock is only possible if we try to shutdown a VM with a particular VDI +and if we try to snapshot it in the same time. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 102 +++++++++++++++++++------------------------ + 1 file changed, 45 insertions(+), 57 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index a6ca8840..ed41e77a 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -148,32 +148,26 @@ def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + if image_type == vhdutil.VDI_TYPE_RAW: + return + +- lock = Lock(vhdutil.LOCK_TYPE_SR, sr_uuid) +- try: +- lock.acquire() ++ device_path = linstor.get_device_path(vdi_uuid) ++ ++ # If the virtual VHD size is lower than the LINSTOR volume size, ++ # there is nothing to do. ++ vhd_size = compute_volume_size( ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 ++ LinstorVhdUtil(session, linstor).get_size_virt(vdi_uuid), # pylint: disable = E1120 ++ image_type ++ ) + +- device_path = linstor.get_device_path(vdi_uuid) ++ volume_info = linstor.get_volume_info(vdi_uuid) ++ volume_size = volume_info.virtual_size + +- # If the virtual VHD size is lower than the LINSTOR volume size, +- # there is nothing to do. +- vhd_size = compute_volume_size( +- # TODO: Replace pylint comment with this feature when possible: +- # https://github.com/PyCQA/pylint/pull/2926 +- LinstorVhdUtil(session, linstor).get_size_virt(vdi_uuid), # pylint: disable = E1120 +- image_type ++ if vhd_size > volume_size: ++ inflate( ++ journaler, linstor, vdi_uuid, device_path, ++ vhd_size, volume_size + ) + +- volume_info = linstor.get_volume_info(vdi_uuid) +- volume_size = volume_info.virtual_size +- +- if vhd_size > volume_size: +- inflate( +- journaler, linstor, vdi_uuid, device_path, +- vhd_size, volume_size +- ) +- finally: +- lock.release() +- + + def detach_thin_impl(session, linstor, sr_uuid, vdi_uuid): + volume_metadata = linstor.get_volume_metadata(vdi_uuid) +@@ -181,45 +175,39 @@ def detach_thin_impl(session, linstor, sr_uuid, vdi_uuid): + if image_type == vhdutil.VDI_TYPE_RAW: + return + +- lock = Lock(vhdutil.LOCK_TYPE_SR, sr_uuid) +- try: +- lock.acquire() +- +- def check_vbd_count(): +- vdi_ref = session.xenapi.VDI.get_by_uuid(vdi_uuid) +- vbds = session.xenapi.VBD.get_all_records_where( +- 'field "VDI" = "{}"'.format(vdi_ref) +- ) ++ def check_vbd_count(): ++ vdi_ref = session.xenapi.VDI.get_by_uuid(vdi_uuid) ++ vbds = session.xenapi.VBD.get_all_records_where( ++ 'field "VDI" = "{}"'.format(vdi_ref) ++ ) + +- num_plugged = 0 +- for vbd_rec in vbds.values(): +- if vbd_rec['currently_attached']: +- num_plugged += 1 +- if num_plugged > 1: +- raise xs_errors.XenError( +- 'VDIUnavailable', +- opterr='Cannot deflate VDI {}, already used by ' +- 'at least 2 VBDs'.format(vdi_uuid) +- ) ++ num_plugged = 0 ++ for vbd_rec in vbds.values(): ++ if vbd_rec['currently_attached']: ++ num_plugged += 1 ++ if num_plugged > 1: ++ raise xs_errors.XenError( ++ 'VDIUnavailable', ++ opterr='Cannot deflate VDI {}, already used by ' ++ 'at least 2 VBDs'.format(vdi_uuid) ++ ) + +- # We can have multiple VBDs attached to a VDI during a VM-template clone. +- # So we use a timeout to ensure that we can detach the volume properly. +- util.retry(check_vbd_count, maxretry=10, period=1) ++ # We can have multiple VBDs attached to a VDI during a VM-template clone. ++ # So we use a timeout to ensure that we can detach the volume properly. ++ util.retry(check_vbd_count, maxretry=10, period=1) + +- device_path = linstor.get_device_path(vdi_uuid) +- new_volume_size = LinstorVolumeManager.round_up_volume_size( +- # TODO: Replace pylint comment with this feature when possible: +- # https://github.com/PyCQA/pylint/pull/2926 +- LinstorVhdUtil(session, linstor).get_size_phys(vdi_uuid) # pylint: disable = E1120 +- ) ++ device_path = linstor.get_device_path(vdi_uuid) ++ new_volume_size = LinstorVolumeManager.round_up_volume_size( ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 ++ LinstorVhdUtil(session, linstor).get_size_phys(vdi_uuid) # pylint: disable = E1120 ++ ) + +- volume_info = linstor.get_volume_info(vdi_uuid) +- old_volume_size = volume_info.virtual_size +- deflate( +- linstor, vdi_uuid, device_path, new_volume_size, old_volume_size +- ) +- finally: +- lock.release() ++ volume_info = linstor.get_volume_info(vdi_uuid) ++ old_volume_size = volume_info.virtual_size ++ deflate( ++ linstor, vdi_uuid, device_path, new_volume_size, old_volume_size ++ ) + + + def detach_thin(session, linstor, sr_uuid, vdi_uuid): diff --git a/SOURCES/0132-fix-LinstorSR-ensure-database-is-mounted-during-scan.patch b/SOURCES/0132-fix-LinstorSR-ensure-database-is-mounted-during-scan.patch new file mode 100644 index 0000000..89966d4 --- /dev/null +++ b/SOURCES/0132-fix-LinstorSR-ensure-database-is-mounted-during-scan.patch @@ -0,0 +1,66 @@ +From 30b45e72a11b9e72a9edcdd22d71a15b8d132e63 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 3 Oct 2023 18:42:42 +0200 +Subject: [PATCH 132/177] fix(LinstorSR): ensure database is mounted during + scan + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 15 +++++++++++++++ + drivers/linstorvolumemanager.py | 10 +++++++++- + 2 files changed, 24 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index ed41e77a..ed5998e8 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -824,6 +824,21 @@ class LinstorSR(SR.SR): + if self.vdis[vdi_uuid].deleted: + del self.vdis[vdi_uuid] + ++ # Security to prevent VDIs from being forgotten if the controller ++ # is started without a shared and mounted /var/lib/linstor path. ++ try: ++ self._linstor.get_database_path() ++ except Exception: ++ # Failed to get database path, ensure we don't have ++ # VDIs in the XAPI database... ++ if self.session.xenapi.SR.get_VDIs( ++ self.session.xenapi.SR.get_by_uuid(self.uuid) ++ ): ++ raise xs_errors.XenError( ++ 'SRUnavailable', ++ opterr='Database is not mounted' ++ ) ++ + # Update the database before the restart of the GC to avoid + # bad sync in the process if new VDIs have been introduced. + ret = super(LinstorSR, self).scan(self.uuid) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index ee637ae2..f1f3bce7 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -884,7 +884,7 @@ class LinstorVolumeManager(object): + + def get_device_path(self, volume_uuid): + """ +- Get the dev path of a volume. ++ Get the dev path of a volume, create a diskless if necessary. + :param str volume_uuid: The volume uuid to get the dev path. + :return: The current device path of the volume. + :rtype: str +@@ -1587,6 +1587,14 @@ class LinstorVolumeManager(object): + + return resources + ++ def get_database_path(self): ++ """ ++ Get the database path. ++ :return: The current database path. ++ :rtype: str ++ """ ++ return self._request_database_path(self._linstor) ++ + @classmethod + def create_sr( + cls, group_name, ips, redundancy, diff --git a/SOURCES/0133-fix-LinstorSR-restart-drbd-reactor-in-case-of-failur.patch b/SOURCES/0133-fix-LinstorSR-restart-drbd-reactor-in-case-of-failur.patch new file mode 100644 index 0000000..503effa --- /dev/null +++ b/SOURCES/0133-fix-LinstorSR-restart-drbd-reactor-in-case-of-failur.patch @@ -0,0 +1,50 @@ +From 6603e355de8b19f851669fa3d58a42662ea4e0eb Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 4 Oct 2023 14:30:36 +0200 +Subject: [PATCH 133/177] fix(LinstorSR): restart drbd-reactor in case of + failure + +Otherwise we can have all hosts unusable after a massive reboot: +all the drbd-reactor services are marked as failed, and no controller +is started. + +Signed-off-by: Ronan Abhamon +--- + Makefile | 3 +++ + etc/systemd/system/drbd-reactor.service.d/override.conf | 6 ++++++ + 2 files changed, 9 insertions(+) + create mode 100644 etc/systemd/system/drbd-reactor.service.d/override.conf + +diff --git a/Makefile b/Makefile +index bc3e97fe..7f7740cb 100755 +--- a/Makefile ++++ b/Makefile +@@ -149,6 +149,7 @@ install: precheck + mkdir -p $(SM_STAGING)$(UDEV_SCRIPTS_DIR) + mkdir -p $(SM_STAGING)$(INIT_DIR) + mkdir -p $(SM_STAGING)$(SYSTEMD_CONF_DIR) ++ mkdir -p $(SM_STAGING)$(SYSTEMD_CONF_DIR)/drbd-reactor.service.d + mkdir -p $(SM_STAGING)$(SYSTEMD_CONF_DIR)/linstor-satellite.service.d + mkdir -p $(SM_STAGING)$(SYSTEMD_SERVICE_DIR) + mkdir -p $(SM_STAGING)$(MPATH_CONF_DIR) +@@ -178,6 +179,8 @@ install: precheck + $(SM_STAGING)/$(SM_DEST) + install -m 644 etc/logrotate.d/$(SMLOG_CONF) \ + $(SM_STAGING)/$(LOGROTATE_DIR) ++ install -m 644 etc/systemd/system/drbd-reactor.service.d/override.conf \ ++ $(SM_STAGING)/$(SYSTEMD_CONF_DIR)/drbd-reactor.service.d/ + install -m 644 etc/systemd/system/linstor-satellite.service.d/override.conf \ + $(SM_STAGING)/$(SYSTEMD_CONF_DIR)/linstor-satellite.service.d/ + install -m 644 etc/systemd/system/var-lib-linstor.service \ +diff --git a/etc/systemd/system/drbd-reactor.service.d/override.conf b/etc/systemd/system/drbd-reactor.service.d/override.conf +new file mode 100644 +index 00000000..2f99a46a +--- /dev/null ++++ b/etc/systemd/system/drbd-reactor.service.d/override.conf +@@ -0,0 +1,6 @@ ++[Service] ++StartLimitInterval=60 ++StartLimitBurst=10 ++ ++Restart=always ++RestartSec=2 diff --git a/SOURCES/0134-fix-linstorvolumemanager-retry-in-case-of-failure-du.patch b/SOURCES/0134-fix-linstorvolumemanager-retry-in-case-of-failure-du.patch new file mode 100644 index 0000000..31f2a66 --- /dev/null +++ b/SOURCES/0134-fix-linstorvolumemanager-retry-in-case-of-failure-du.patch @@ -0,0 +1,30 @@ +From ac881521fa06183435435e47933c6e7f034dfed8 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 9 Oct 2023 10:37:32 +0200 +Subject: [PATCH 134/177] fix(linstorvolumemanager): retry in case of failure + during mkfs call on database + +The device is not always ready after creation. +So we must retry the mkfs call in case of failure. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index f1f3bce7..23e80d91 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2666,7 +2666,10 @@ class LinstorVolumeManager(object): + ) + + try: +- util.pread2([DATABASE_MKFS, expected_device_path]) ++ util.retry( ++ lambda: util.pread2([DATABASE_MKFS, expected_device_path]), ++ maxretry=5 ++ ) + except Exception as e: + raise LinstorVolumeManagerError( + 'Failed to execute {} on database volume: {}' diff --git a/SOURCES/0135-fix-linstorvolumemanager-avoid-diskless-creation-whe.patch b/SOURCES/0135-fix-linstorvolumemanager-avoid-diskless-creation-whe.patch new file mode 100644 index 0000000..b55b86f --- /dev/null +++ b/SOURCES/0135-fix-linstorvolumemanager-avoid-diskless-creation-whe.patch @@ -0,0 +1,143 @@ +From 69ddfe0cf317308e2097ef4cacb081b9c4c60c40 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 26 Sep 2023 11:48:38 +0200 +Subject: [PATCH 135/177] fix(linstorvolumemanager): avoid diskless creation + when a new resource is added + +Like said in this discussion https://github.com/xcp-ng/sm/pull/34 : +"Instead of using diskless_on_remaing, only place a resource on demand on a node. +If a cluster would ever exceed 20 nodes, having diskless on all of them +might get problematic, as all of them are part of the quorum voting." + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 51 ++++++--------------------------- + 1 file changed, 9 insertions(+), 42 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 23e80d91..49ca83c0 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -614,8 +614,7 @@ class LinstorVolumeManager(object): + return volume_uuid in self._volumes + + def create_volume( +- self, volume_uuid, size, persistent=True, volume_name=None, +- no_diskless=False ++ self, volume_uuid, size, persistent=True, volume_name=None + ): + """ + Create a new volume on the SR. +@@ -625,8 +624,6 @@ class LinstorVolumeManager(object): + on the next constructor call LinstorSR(...). + :param str volume_name: If set, this name is used in the LINSTOR + database instead of a generated name. +- :param bool no_diskless: If set, the default group redundancy is not +- used, instead the volume is created on all nodes. + :return: The current device path of the volume. + :rtype: str + """ +@@ -635,8 +632,7 @@ class LinstorVolumeManager(object): + if not volume_name: + volume_name = self.build_volume_name(util.gen_uuid()) + volume_properties = self._create_volume_with_properties( +- volume_uuid, volume_name, size, place_resources=True, +- no_diskless=no_diskless ++ volume_uuid, volume_name, size, place_resources=True + ) + + # Volume created! Now try to find the device path. +@@ -1295,8 +1291,7 @@ class LinstorVolumeManager(object): + # Note: placed outside try/except block because we create only definition first. + # There is no reason to call `clean` before the real resource creation. + volume_properties = self._create_volume_with_properties( +- clone_uuid, clone_volume_name, size, +- place_resources=False ++ clone_uuid, clone_volume_name, size, place_resources=False + ) + + # After this point, `clean` can be called for any fail because the clone UUID +@@ -1758,7 +1753,7 @@ class LinstorVolumeManager(object): + name=group_name, + place_count=redundancy, + storage_pool=group_name, +- diskless_on_remaining=True ++ diskless_on_remaining=False + ) + error_str = cls._get_error_str(result) + if error_str: +@@ -2062,28 +2057,11 @@ class LinstorVolumeManager(object): + return self._storage_pools + + def _create_volume( +- self, volume_uuid, volume_name, size, place_resources, +- no_diskless=False ++ self, volume_uuid, volume_name, size, place_resources + ): +- if no_diskless and not place_resources: +- raise LinstorVolumeManagerError( +- 'Could not create volume `{}` from SR `{}`: it\'s impossible ' +- .format(volume_uuid, self._group_name) + +- 'to force no diskless without placing resources' +- ) +- + size = self.round_up_volume_size(size) + self._mark_resource_cache_as_dirty() + +- resources = [] +- if no_diskless: +- for node_name in self._get_node_names(): +- resources.append(linstor.ResourceData( +- node_name=node_name, +- rsc_name=volume_name, +- storage_pool=self._group_name +- )) +- + def create_definition(): + self._check_volume_creation_errors( + self._linstor.resource_group_spawn( +@@ -2109,23 +2087,13 @@ class LinstorVolumeManager(object): + def create(): + try: + create_definition() +- if no_diskless: +- # Create a physical resource on each node. +- result = self._linstor.resource_create(resources) +- error_str = self._get_error_str(result) +- if error_str: +- raise LinstorVolumeManagerError( +- 'Could not create volume `{}` from SR `{}`: {}'.format( +- volume_uuid, self._group_name, error_str +- ) +- ) +- elif place_resources: ++ if place_resources: + # Basic case when we use the default redundancy of the group. + self._check_volume_creation_errors( + self._linstor.resource_auto_place( + rsc_name=volume_name, + place_count=self._redundancy, +- diskless_on_remaining=not no_diskless ++ diskless_on_remaining=False + ), + volume_uuid, + self._group_name +@@ -2141,8 +2109,7 @@ class LinstorVolumeManager(object): + util.retry(create, maxretry=5) + + def _create_volume_with_properties( +- self, volume_uuid, volume_name, size, place_resources, +- no_diskless=False ++ self, volume_uuid, volume_name, size, place_resources + ): + if self.check_volume_exists(volume_uuid): + raise LinstorVolumeManagerError( +@@ -2171,7 +2138,7 @@ class LinstorVolumeManager(object): + volume_properties[self.PROP_VOLUME_NAME] = volume_name + + self._create_volume( +- volume_uuid, volume_name, size, place_resources, no_diskless ++ volume_uuid, volume_name, size, place_resources + ) + + assert volume_properties.namespace == \ diff --git a/SOURCES/0136-fix-LinstorSR-remove-diskless-after-VDI.detach-calls.patch b/SOURCES/0136-fix-LinstorSR-remove-diskless-after-VDI.detach-calls.patch new file mode 100644 index 0000000..8daf637 --- /dev/null +++ b/SOURCES/0136-fix-LinstorSR-remove-diskless-after-VDI.detach-calls.patch @@ -0,0 +1,88 @@ +From 7e6f37fb95ad0974b7e4be3e336b348517fa0946 Mon Sep 17 00:00:00 2001 +From: Rene Peinthor +Date: Tue, 25 Jul 2023 11:19:39 +0200 +Subject: [PATCH 136/177] fix(LinstorSR): remove diskless after VDI.detach + calls + +Signed-off-by: Rene Peinthor +Co-authored-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 20 ++++++++++++++++++++ + drivers/linstorvolumemanager.py | 26 +++++++++++++++++++++++--- + 2 files changed, 43 insertions(+), 3 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index ed5998e8..7b9a8e57 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1943,6 +1943,26 @@ class LinstorVDI(VDI.VDI): + .format(e) + ) + ++ # We remove only on slaves because the volume can be used by the GC. ++ if self.sr._is_master: ++ return ++ ++ while vdi_uuid: ++ try: ++ path = self._linstor.build_device_path(self._linstor.get_volume_name(vdi_uuid)) ++ parent_vdi_uuid = self.sr._vhdutil.get_vhd_info(vdi_uuid).parentUuid ++ except Exception: ++ break ++ ++ if util.pathexists(path): ++ try: ++ self._linstor.remove_volume_if_diskless(vdi_uuid) ++ except Exception as e: ++ # Ensure we can always detach properly. ++ # I don't want to corrupt the XAPI info. ++ util.SMlog('Failed to clean VDI {} during detach: {}'.format(vdi_uuid, e)) ++ vdi_uuid = parent_vdi_uuid ++ + def resize(self, sr_uuid, vdi_uuid, size): + util.SMlog('LinstorVDI.resize for {}'.format(self.uuid)) + if not self.sr._is_master: +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 49ca83c0..0f6fbcf3 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -781,6 +781,28 @@ class LinstorVolumeManager(object): + if waiting: + self._logger('No volume locked now!') + ++ def remove_volume_if_diskless(self, volume_uuid): ++ """ ++ Remove disless path from local node. ++ :param str volume_uuid: The volume uuid to remove. ++ """ ++ ++ self._ensure_volume_exists(volume_uuid) ++ ++ volume_properties = self._get_volume_properties(volume_uuid) ++ volume_name = volume_properties.get(self.PROP_VOLUME_NAME) ++ ++ node_name = socket.gethostname() ++ result = self._linstor.resource_delete_if_diskless( ++ node_name=node_name, rsc_name=volume_name ++ ) ++ if not linstor.Linstor.all_api_responses_no_error(result): ++ raise LinstorVolumeManagerError( ++ 'Unable to delete diskless path of `{}` on node `{}`: {}' ++ .format(volume_name, node_name, ', '.join( ++ [str(x) for x in result])) ++ ) ++ + def introduce_volume(self, volume_uuid): + pass # TODO: Implement me. + +@@ -2459,9 +2481,7 @@ class LinstorVolumeManager(object): + + @classmethod + def _activate_device_path(cls, lin, node_name, volume_name): +- result = lin.resource_create([ +- linstor.ResourceData(node_name, volume_name, diskless=True) +- ]) ++ result = lin.resource_make_available(node_name, volume_name, diskful=False) + if linstor.Linstor.all_api_responses_no_error(result): + return + errors = linstor.Linstor.filter_api_call_response_errors(result) diff --git a/SOURCES/0137-fix-LinstorSR-robustify-_load_vdi_info-in-cleanup.py.patch b/SOURCES/0137-fix-LinstorSR-robustify-_load_vdi_info-in-cleanup.py.patch new file mode 100644 index 0000000..e6446d0 --- /dev/null +++ b/SOURCES/0137-fix-LinstorSR-robustify-_load_vdi_info-in-cleanup.py.patch @@ -0,0 +1,96 @@ +From 995c6b7cf9ecb9baee34e05586320ccb4339651b Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 11 Oct 2023 12:39:56 +0200 +Subject: [PATCH 137/177] fix(LinstorSR): robustify _load_vdi_info in + cleanup.py + +After a failed snapshot like that: +``` +Sep 18 12:02:47 xcp1 SM: [909] ['/usr/bin/vhd-util', 'snapshot', '--debug', '-n', '/dev/drbd/by-res/xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d/0', '-p', '/dev/drbd/by-res/xcp-volume-432905e3-57ba-41c7-8517-3331e1907c69/0', '-S', '2097152'] +Sep 18 12:02:49 xcp1 SM: [909] FAILED in util.pread: (rc 30) stdout: '', stderr: '/dev/drbd/by-res/xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d/0: failed to create: -30 +Sep 18 12:02:50 xcp1 SM: [909] raise opener exception: cmd: `['/usr/bin/vhd-util', 'snapshot', '--debug', '-n', '/dev/drbd/by-res/xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d/0', '-p', '/dev/drbd/by-res/xcp-volume-432905e3-57ba-41c7-8517-3331e1907c69/0', '-S', '2097152']`, code: `30`, reason: `/dev/drbd/by-res/xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d/0: failed to create: -30` (openers: {'xcp3': {}, 'xcp2': {}, 'xcp1': {}}) +Sep 18 12:02:50 xcp1 SM: [909] ***** Failed to snapshot!: EXCEPTION , cmd: `['/usr/bin/vhd-util', 'snapshot', '--debug', '-n', '/dev/drbd/by-res/xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d/0', '-p', '/dev/drbd/by-res/xcp-volume-432905e3-57ba-41c7-8517-3331e1907c69/0', '-S', '2097152']`, code: `30`, reason: `/dev/drbd/by-res/xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d/0: failed to create: -30` (openers: {'xcp3': {}, 'xcp2': {}, 'xcp1': {}}) +Sep 18 12:02:50 xcp1 SM: [909] Cannot destroy VDI 4e1ac2a2-3d57-408f-92a8-ccc03882511f during undo clone: Cannot destroy volume `4e1ac2a2-3d57-408f-92a8-ccc03882511f`: Could not destroy resource `xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d` from SR `xcp-sr-linstor_group_thin_device`: Resource 'xcp-volume-bbf05b73-bbad-438e-a194-3470558c8a8d' on node 'xcp2' is still in use. +Sep 18 12:02:50 xcp1 SM: [909] Trying to update volume UUID 4e1ac2a2-3d57-408f-92a8-ccc03882511f to DELETED_4e1ac2a2-3d57-408f-92a8-ccc03882511f... +``` + +The remaining volume can be empty, so we must ignore all volumes with the `DELETED_` prefix. +These are not valid VHDs after all. +And also we must be sure to never set the RAW flag on corrupted VHDs during cleanup. + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 46 ++++++++++++++++++++++++++++++++++++++++++---- + 1 file changed, 42 insertions(+), 4 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 5353e9aa..376da3ec 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -2990,16 +2990,31 @@ class LinstorSR(SR): + all_volume_info = self._linstor.get_volumes_with_info() + volumes_metadata = self._linstor.get_volumes_with_metadata() + for vdi_uuid, volume_info in all_volume_info.items(): ++ deleted = False + try: + if not volume_info.name and \ + not list(volumes_metadata[vdi_uuid].items()): + continue # Ignore it, probably deleted. + + vdi_type = volumes_metadata[vdi_uuid].get(VDI_TYPE_TAG) +- if vdi_type == vhdutil.VDI_TYPE_RAW: +- info = None +- else: ++ if vdi_uuid.startswith('DELETED_'): ++ # Assume it's really a RAW volume of a failed snap without VHD header/footer. ++ deleted = True ++ elif vdi_type == vhdutil.VDI_TYPE_VHD: + info = self._vhdutil.get_vhd_info(vdi_uuid) ++ else: ++ # Ensure it's not a VHD... ++ try: ++ info = self._vhdutil.get_vhd_info(vdi_uuid) ++ except: ++ try: ++ self._vhdutil.force_repair( ++ self._linstor.get_device_path(vdi_uuid) ++ ) ++ info = self._vhdutil.get_vhd_info(vdi_uuid) ++ except: ++ info = None ++ + except Exception as e: + Util.log( + ' [VDI {}: failed to load VDI info]: {}' +@@ -3007,7 +3022,30 @@ class LinstorSR(SR): + ) + info = vhdutil.VHDInfo(vdi_uuid) + info.error = 1 +- all_vdi_info[vdi_uuid] = info ++ ++ if not deleted: ++ all_vdi_info[vdi_uuid] = info ++ continue ++ ++ # We must remove this VDI now without adding it in the VDI list. ++ # Otherwise `Relinking` calls and other actions can be launched on it. ++ # We don't want that... ++ assert deleted ++ assert vdi_uuid.startswith('DELETED_') ++ Util.log('Deleting bad VDI {}'.format(vdi_uuid)) ++ ++ self.lock() ++ try: ++ self._linstor.destroy_volume(vdi_uuid) ++ try: ++ self.forgetVDI(vdi_uuid) ++ except: ++ pass ++ except Exception as e: ++ Util.log('Cannot delete bad VDI: {}'.format(e)) ++ finally: ++ self.unlock() ++ + return all_vdi_info + + # TODO: Maybe implement _liveLeafCoalesce/_prepareCoalesceLeaf/ diff --git a/SOURCES/0138-fix-LinstorSR-ensure-detach-never-fails-on-plugin-fa.patch b/SOURCES/0138-fix-LinstorSR-ensure-detach-never-fails-on-plugin-fa.patch new file mode 100644 index 0000000..51f0ecb --- /dev/null +++ b/SOURCES/0138-fix-LinstorSR-ensure-detach-never-fails-on-plugin-fa.patch @@ -0,0 +1,29 @@ +From 64d36a80ff1e19ee7b4a8816f537b1e90ccda32f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 23 Oct 2023 14:31:27 +0200 +Subject: [PATCH 138/177] fix(LinstorSR): ensure detach never fails on plugin + failure + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 7b9a8e57..d0fc4219 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -2241,7 +2241,12 @@ class LinstorVDI(VDI.VDI): + 'srUuid': self.sr.uuid, + 'vdiUuid': self.uuid + } +- self.sr._exec_manager_command(master, fn, args, 'VDIUnavailable') ++ ++ try: ++ self.sr._exec_manager_command(master, fn, args, 'VDIUnavailable') ++ except Exception: ++ if fn != 'detach': ++ raise + + # Reload size attrs after inflate or deflate! + self._load_this() diff --git a/SOURCES/0139-fix-LinstorSR-ensure-we-coalesce-only-volumes-with-a.patch b/SOURCES/0139-fix-LinstorSR-ensure-we-coalesce-only-volumes-with-a.patch new file mode 100644 index 0000000..b7211db --- /dev/null +++ b/SOURCES/0139-fix-LinstorSR-ensure-we-coalesce-only-volumes-with-a.patch @@ -0,0 +1,137 @@ +From c9ab4bc2198754e8eacc2fe4699eb5e6e5e4728e Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 23 Oct 2023 15:52:23 +0200 +Subject: [PATCH 139/177] fix(LinstorSR): ensure we coalesce only volumes with + a valid size + +--- + drivers/cleanup.py | 14 ++++++++++++++ + drivers/linstor-manager | 15 +++++++++++++++ + drivers/linstorvhdutil.py | 21 ++++++++++++++++++--- + 3 files changed, 47 insertions(+), 3 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 376da3ec..4e4620f3 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -1441,6 +1441,20 @@ class LinstorVDI(VDI): + return super(LinstorVDI, self).pause(failfast) + + def coalesce(self): ++ # Note: We raise `SMException` here to skip the current coalesce in case of failure. ++ # Using another exception we can't execute the next coalesce calls. ++ try: ++ drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) ++ except Exception as e: ++ raise util.SMException( ++ 'VDI {} could not be coalesced because the DRBD block size cannot be read: {}' ++ .format(self.uuid, e)) ++ ++ if self._sizeVHD > drbd_size: ++ raise util.SMException( ++ 'VDI {} could not be coalesced because VHD phys size > DRBD block size ({} > {})' ++ .format(self.uuid, self._sizeVHD, drbd_size)) ++ + self.sr._vhdutil.force_coalesce(self.path) + + def getParent(self): +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 45201eed..9e5e1d6e 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -485,6 +485,18 @@ def get_block_bitmap(session, args): + raise + + ++def get_drbd_size(session, args): ++ try: ++ device_path = args['devicePath'] ++ (ret, stdout, stderr) = util.doexec(['blockdev', '--getsize64', device_path]) ++ if ret == 0: ++ return stdout.strip() ++ raise Exception('Failed to get DRBD size: {}'.format(stderr)) ++ except Exception: ++ util.SMlog('linstor-manager:get_drbd_size error: {}'.format(stderr)) ++ raise ++ ++ + def set_parent(session, args): + try: + device_path = args['devicePath'] +@@ -969,6 +981,9 @@ if __name__ == '__main__': + 'getKeyHash': get_key_hash, + 'getBlockBitmap': get_block_bitmap, + ++ # Small helper to get the DRBD blockdev size. ++ 'getDrbdSize': get_drbd_size, ++ + # Called by cleanup.py to coalesce when a primary + # is opened on a non-local host. + 'setParent': set_parent, +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 8b6985d9..5f3ae084 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -38,7 +38,7 @@ def call_vhd_util_on_host(session, host_ref, method, device_path, args): + util.SMlog('call-plugin ({} with {}) exception: {}'.format( + method, args, e + )) +- raise ++ raise util.SMException(str(e)) + + util.SMlog('call-plugin ({} with {}) returned: {}'.format( + method, args, response +@@ -47,7 +47,7 @@ def call_vhd_util_on_host(session, host_ref, method, device_path, args): + return response + + +-class LinstorCallException(Exception): ++class LinstorCallException(util.SMException): + def __init__(self, cmd_err): + self.cmd_err = cmd_err + +@@ -207,6 +207,16 @@ class LinstorVhdUtil: + def get_block_bitmap(self, vdi_uuid, response): + return base64.b64decode(response) + ++ @linstorhostcall('_get_drbd_size', 'getDrbdSize') ++ def get_drbd_size(self, vdi_uuid, response): ++ return int(response) ++ ++ def _get_drbd_size(self, path): ++ (ret, stdout, stderr) = util.doexec(['blockdev', '--getsize64', path]) ++ if ret == 0: ++ return int(stdout.strip()) ++ raise util.SMException('Failed to get DRBD size: {}'.format(stderr)) ++ + # -------------------------------------------------------------------------- + # Setters: only used locally. + # -------------------------------------------------------------------------- +@@ -308,7 +318,6 @@ class LinstorVhdUtil: + else: + e_str = str(e) + +- e_with_openers = None + try: + volume_uuid = self._linstor.get_volume_uuid_from_device_path( + device_path +@@ -326,6 +335,9 @@ class LinstorVhdUtil: + raise e_wrapper # pylint: disable = E0702 + + def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): ++ if isinstance(local_method, str): ++ local_method = getattr(self, local_method) ++ + try: + def local_call(): + try: +@@ -355,6 +367,9 @@ class LinstorVhdUtil: + # another host using the DRBD opener list. In the other case, if the parent is required, + # we must check where this last one is open instead of the child. + ++ if isinstance(local_method, str): ++ local_method = getattr(self, local_method) ++ + # A. Try to write locally... + try: + return self._call_local_vhd_util(local_method, device_path, *args, **kwargs) diff --git a/SOURCES/0140-fix-LinstorSR-don-t-try-to-repair-persistent-volumes.patch b/SOURCES/0140-fix-LinstorSR-don-t-try-to-repair-persistent-volumes.patch new file mode 100644 index 0000000..d58554b --- /dev/null +++ b/SOURCES/0140-fix-LinstorSR-don-t-try-to-repair-persistent-volumes.patch @@ -0,0 +1,196 @@ +From e615d0cc2f5dad974bec0de819fa3bf0a4b5d07a Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 24 Oct 2023 11:57:40 +0200 +Subject: [PATCH 140/177] fix(LinstorSR): don't try to repair persistent + volumes in GC + +Use constants to simplify maintenance. +--- + drivers/LinstorSR.py | 21 ++++++++----- + drivers/cleanup.py | 54 ++++++++++++++++----------------- + drivers/linstorvolumemanager.py | 4 ++- + 3 files changed, 43 insertions(+), 36 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index d0fc4219..d0ce079a 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -23,6 +23,7 @@ try: + from linstorvolumemanager import get_controller_node_name + from linstorvolumemanager import LinstorVolumeManager + from linstorvolumemanager import LinstorVolumeManagerError ++ from linstorvolumemanager import PERSISTENT_PREFIX + + LINSTOR_AVAILABLE = True + except ImportError: +@@ -76,6 +77,10 @@ TRACE_PERFS = False + # Enable/Disable VHD key hash support. + USE_KEY_HASH = False + ++# Special volumes. ++HA_VOLUME_NAME = PERSISTENT_PREFIX + 'ha-statefile' ++REDO_LOG_VOLUME_NAME = PERSISTENT_PREFIX + 'redo-log' ++ + # ============================================================================== + + # TODO: Supports 'VDI_INTRODUCE', 'VDI_RESET_ON_BOOT/2', 'SR_TRIM', +@@ -1727,9 +1732,9 @@ class LinstorVDI(VDI.VDI): + try: + volume_name = None + if self.ty == 'ha_statefile': +- volume_name = 'xcp-persistent-ha-statefile' ++ volume_name = HA_VOLUME_NAME + elif self.ty == 'redo_log': +- volume_name = 'xcp-persistent-redo-log' ++ volume_name = REDO_LOG_VOLUME_NAME + + self._linstor.create_volume( + self.uuid, volume_size, persistent=False, +@@ -2085,7 +2090,7 @@ class LinstorVDI(VDI.VDI): + # instead. + volume_name = self._linstor.get_volume_name(self.uuid) + if not USE_HTTP_NBD_SERVERS or volume_name not in [ +- 'xcp-persistent-ha-statefile', 'xcp-persistent-redo-log' ++ HA_VOLUME_NAME, REDO_LOG_VOLUME_NAME + ]: + if not self.path or not util.pathexists(self.path): + available = False +@@ -2587,7 +2592,7 @@ class LinstorVDI(VDI.VDI): + http_server = None + + try: +- if volume_name == 'xcp-persistent-ha-statefile': ++ if volume_name == HA_VOLUME_NAME: + port = '8076' + else: + port = '8077' +@@ -2669,7 +2674,7 @@ class LinstorVDI(VDI.VDI): + try: + # We use a precomputed device size. + # So if the XAPI is modified, we must update these values! +- if volume_name == 'xcp-persistent-ha-statefile': ++ if volume_name == HA_VOLUME_NAME: + # See: https://github.com/xapi-project/xen-api/blob/703479fa448a8d7141954bb6e8964d8e25c4ac2e/ocaml/xapi/xha_statefile.ml#L32-L37 + port = '8076' + device_size = 4 * 1024 * 1024 +@@ -2793,7 +2798,7 @@ class LinstorVDI(VDI.VDI): + def _check_http_nbd_volume_name(self): + volume_name = self.path[14:] + if volume_name not in [ +- 'xcp-persistent-ha-statefile', 'xcp-persistent-redo-log' ++ HA_VOLUME_NAME, REDO_LOG_VOLUME_NAME + ]: + raise xs_errors.XenError( + 'VDIUnavailable', +@@ -2862,8 +2867,8 @@ class LinstorVDI(VDI.VDI): + http_service = None + if drbd_path: + assert(drbd_path in ( +- '/dev/drbd/by-res/xcp-persistent-ha-statefile/0', +- '/dev/drbd/by-res/xcp-persistent-redo-log/0' ++ '/dev/drbd/by-res/{}/0'.format(HA_VOLUME_NAME), ++ '/dev/drbd/by-res/{}/0'.format(REDO_LOG_VOLUME_NAME) + )) + self._start_persistent_http_server(volume_name) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 4e4620f3..1f898225 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -55,6 +55,7 @@ try: + from linstorvolumemanager import get_controller_uri + from linstorvolumemanager import LinstorVolumeManager + from linstorvolumemanager import LinstorVolumeManagerError ++ from linstorvolumemanager import PERSISTENT_PREFIX as LINSTOR_PERSISTENT_PREFIX + + LINSTOR_AVAILABLE = True + except ImportError: +@@ -3004,16 +3005,36 @@ class LinstorSR(SR): + all_volume_info = self._linstor.get_volumes_with_info() + volumes_metadata = self._linstor.get_volumes_with_metadata() + for vdi_uuid, volume_info in all_volume_info.items(): +- deleted = False + try: +- if not volume_info.name and \ +- not list(volumes_metadata[vdi_uuid].items()): ++ volume_metadata = volumes_metadata[vdi_uuid] ++ if not volume_info.name and not list(volume_metadata.items()): + continue # Ignore it, probably deleted. + +- vdi_type = volumes_metadata[vdi_uuid].get(VDI_TYPE_TAG) + if vdi_uuid.startswith('DELETED_'): + # Assume it's really a RAW volume of a failed snap without VHD header/footer. +- deleted = True ++ # We must remove this VDI now without adding it in the VDI list. ++ # Otherwise `Relinking` calls and other actions can be launched on it. ++ # We don't want that... ++ Util.log('Deleting bad VDI {}'.format(vdi_uuid)) ++ ++ self.lock() ++ try: ++ self._linstor.destroy_volume(vdi_uuid) ++ try: ++ self.forgetVDI(vdi_uuid) ++ except: ++ pass ++ except Exception as e: ++ Util.log('Cannot delete bad VDI: {}'.format(e)) ++ finally: ++ self.unlock() ++ continue ++ ++ vdi_type = volume_metadata.get(VDI_TYPE_TAG) ++ volume_name = self._linstor.get_volume_name(vdi_uuid) ++ if volume_name.startswith(LINSTOR_PERSISTENT_PREFIX): ++ # Always RAW! ++ info = None + elif vdi_type == vhdutil.VDI_TYPE_VHD: + info = self._vhdutil.get_vhd_info(vdi_uuid) + else: +@@ -3037,28 +3058,7 @@ class LinstorSR(SR): + info = vhdutil.VHDInfo(vdi_uuid) + info.error = 1 + +- if not deleted: +- all_vdi_info[vdi_uuid] = info +- continue +- +- # We must remove this VDI now without adding it in the VDI list. +- # Otherwise `Relinking` calls and other actions can be launched on it. +- # We don't want that... +- assert deleted +- assert vdi_uuid.startswith('DELETED_') +- Util.log('Deleting bad VDI {}'.format(vdi_uuid)) +- +- self.lock() +- try: +- self._linstor.destroy_volume(vdi_uuid) +- try: +- self.forgetVDI(vdi_uuid) +- except: +- pass +- except Exception as e: +- Util.log('Cannot delete bad VDI: {}'.format(e)) +- finally: +- self.unlock() ++ all_vdi_info[vdi_uuid] = info + + return all_vdi_info + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 0f6fbcf3..47ca4e87 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -30,9 +30,11 @@ import time + import util + import uuid + ++# Persistent prefix to add to RAW persistent volumes. ++PERSISTENT_PREFIX = 'xcp-persistent-' + + # Contains the data of the "/var/lib/linstor" directory. +-DATABASE_VOLUME_NAME = 'xcp-persistent-database' ++DATABASE_VOLUME_NAME = PERSISTENT_PREFIX + 'database' + DATABASE_SIZE = 1 << 30 # 1GB. + DATABASE_PATH = '/var/lib/linstor' + DATABASE_MKFS = 'mkfs.ext4' diff --git a/SOURCES/0141-fix-linstorvhdutil-format-correctly-message-if-vhd-u.patch b/SOURCES/0141-fix-linstorvhdutil-format-correctly-message-if-vhd-u.patch new file mode 100644 index 0000000..edabecf --- /dev/null +++ b/SOURCES/0141-fix-linstorvhdutil-format-correctly-message-if-vhd-u.patch @@ -0,0 +1,25 @@ +From ed396144e6293ac060fe8bbfde48363ed0313468 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 14 Nov 2023 18:21:53 +0100 +Subject: [PATCH 141/177] fix(linstorvhdutil): format correctly message if + vhd-util cannot be run + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvhdutil.py | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 5f3ae084..3ce7ab9e 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -441,7 +441,7 @@ class LinstorVhdUtil: + + raise xs_errors.XenError( + 'VDIUnavailable', +- opterr='No valid host found to run vhd-util command `{}` (path=`{}`, openers=`{}`): {}' +- .format(remote_method, device_path, openers, e) ++ opterr='No valid host found to run vhd-util command `{}` (path=`{}`, openers=`{}`)' ++ .format(remote_method, device_path, openers) + ) + return util.retry(remote_call, 5, 2) diff --git a/SOURCES/0142-fix-LinstorSR-wait-during-attach-to-open-DRBD-path.patch b/SOURCES/0142-fix-LinstorSR-wait-during-attach-to-open-DRBD-path.patch new file mode 100644 index 0000000..2e92d27 --- /dev/null +++ b/SOURCES/0142-fix-LinstorSR-wait-during-attach-to-open-DRBD-path.patch @@ -0,0 +1,43 @@ +From 7d872f79b8857c2db490cf0c77a5c1c647af59e2 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 24 Oct 2023 23:07:23 +0200 +Subject: [PATCH 142/177] fix(LinstorSR): wait during attach to open DRBD path + +ENODATA and other errors like EROFS can be raised when +a new DRBD path is created on the fly. So ensure to block before tapdisk starts. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 19 +++++++++++++++++++ + 1 file changed, 19 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index d0ce079a..3063abf4 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1901,6 +1901,25 @@ class LinstorVDI(VDI.VDI): + raise xs_errors.XenError( + 'VDIUnavailable', opterr='Could not find: {}'.format(path) + ) ++ ++ # Diskless path can be created on the fly, ensure we can open it. ++ def check_volume_usable(): ++ while True: ++ try: ++ with open(path, 'r+'): ++ pass ++ except IOError as e: ++ if e.errno == errno.ENODATA: ++ time.sleep(2) ++ continue ++ if e.errno == errno.EROFS: ++ util.SMlog('Volume not attachable because RO. Openers: {}'.format( ++ self.sr._linstor.get_volume_openers(vdi_uuid) ++ )) ++ raise ++ break ++ util.retry(check_volume_usable, 15, 2) ++ + vdi_uuid = self.sr._vhdutil.get_vhd_info(vdi_uuid).parentUuid + + self.attached = True diff --git a/SOURCES/0143-fix-LinstorSR-support-different-volume-sizes-in-clea.patch b/SOURCES/0143-fix-LinstorSR-support-different-volume-sizes-in-clea.patch new file mode 100644 index 0000000..4237fc8 --- /dev/null +++ b/SOURCES/0143-fix-LinstorSR-support-different-volume-sizes-in-clea.patch @@ -0,0 +1,514 @@ +From d1e3ace3c5c16ba428c4151482f4065d09930621 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 14 Nov 2023 18:08:26 +0100 +Subject: [PATCH 143/177] fix(LinstorSR): support different volume sizes in + cleanup.py + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 97 +++++------------------------ + drivers/cleanup.py | 128 +++++++++++++++++++++++++++++++------- + drivers/linstorvhdutil.py | 75 ++++++++++++++++++++++ + 3 files changed, 194 insertions(+), 106 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 3063abf4..6802f494 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -134,19 +134,6 @@ OPS_EXCLUSIVE = [ + # ============================================================================== + + +-def compute_volume_size(virtual_size, image_type): +- if image_type == vhdutil.VDI_TYPE_VHD: +- # All LINSTOR VDIs have the metadata area preallocated for +- # the maximum possible virtual size (for fast online VDI.resize). +- meta_overhead = vhdutil.calcOverheadEmpty(LinstorVDI.MAX_SIZE) +- bitmap_overhead = vhdutil.calcOverheadBitmap(virtual_size) +- virtual_size += meta_overhead + bitmap_overhead +- elif image_type != vhdutil.VDI_TYPE_RAW: +- raise Exception('Invalid image type: {}'.format(image_type)) +- +- return LinstorVolumeManager.round_up_volume_size(virtual_size) +- +- + def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + volume_metadata = linstor.get_volume_metadata(vdi_uuid) + image_type = volume_metadata.get(VDI_TYPE_TAG) +@@ -157,7 +144,7 @@ def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + + # If the virtual VHD size is lower than the LINSTOR volume size, + # there is nothing to do. +- vhd_size = compute_volume_size( ++ vhd_size = LinstorVhdUtil.compute_volume_size( + # TODO: Replace pylint comment with this feature when possible: + # https://github.com/PyCQA/pylint/pull/2926 + LinstorVhdUtil(session, linstor).get_size_virt(vdi_uuid), # pylint: disable = E1120 +@@ -168,9 +155,8 @@ def attach_thin(session, journaler, linstor, sr_uuid, vdi_uuid): + volume_size = volume_info.virtual_size + + if vhd_size > volume_size: +- inflate( +- journaler, linstor, vdi_uuid, device_path, +- vhd_size, volume_size ++ LinstorVhdUtil(session, linstor).inflate( ++ journaler, vdi_uuid, device_path, vhd_size, volume_size + ) + + +@@ -202,17 +188,16 @@ def detach_thin_impl(session, linstor, sr_uuid, vdi_uuid): + util.retry(check_vbd_count, maxretry=10, period=1) + + device_path = linstor.get_device_path(vdi_uuid) ++ vhdutil_inst = LinstorVhdUtil(session, linstor) + new_volume_size = LinstorVolumeManager.round_up_volume_size( + # TODO: Replace pylint comment with this feature when possible: + # https://github.com/PyCQA/pylint/pull/2926 +- LinstorVhdUtil(session, linstor).get_size_phys(vdi_uuid) # pylint: disable = E1120 ++ vhdutil_inst.get_size_phys(vdi_uuid) # pylint: disable = E1120 + ) + + volume_info = linstor.get_volume_info(vdi_uuid) + old_volume_size = volume_info.virtual_size +- deflate( +- linstor, vdi_uuid, device_path, new_volume_size, old_volume_size +- ) ++ vhdutil_inst.deflate(vdi_uuid, device_path, new_volume_size, old_volume_size) + + + def detach_thin(session, linstor, sr_uuid, vdi_uuid): +@@ -225,56 +210,6 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + util.SMlog('Failed to detach properly VDI {}: {}'.format(vdi_uuid, e)) + + +-def inflate(journaler, linstor, vdi_uuid, vdi_path, new_size, old_size): +- # Only inflate if the LINSTOR volume capacity is not enough. +- new_size = LinstorVolumeManager.round_up_volume_size(new_size) +- if new_size <= old_size: +- return +- +- util.SMlog( +- 'Inflate {} (size={}, previous={})' +- .format(vdi_uuid, new_size, old_size) +- ) +- +- journaler.create( +- LinstorJournaler.INFLATE, vdi_uuid, old_size +- ) +- linstor.resize_volume(vdi_uuid, new_size) +- +- result_size = linstor.get_volume_size(vdi_uuid) +- if result_size < new_size: +- util.SMlog( +- 'WARNING: Cannot inflate volume to {}B, result size: {}B' +- .format(new_size, result_size) +- ) +- +- if not util.zeroOut( +- vdi_path, result_size - vhdutil.VHD_FOOTER_SIZE, +- vhdutil.VHD_FOOTER_SIZE +- ): +- raise xs_errors.XenError( +- 'EIO', +- opterr='Failed to zero out VHD footer {}'.format(vdi_path) +- ) +- +- LinstorVhdUtil(None, linstor).set_size_phys(vdi_path, result_size, False) +- journaler.remove(LinstorJournaler.INFLATE, vdi_uuid) +- +- +-def deflate(linstor, vdi_uuid, vdi_path, new_size, old_size): +- new_size = LinstorVolumeManager.round_up_volume_size(new_size) +- if new_size >= old_size: +- return +- +- util.SMlog( +- 'Deflate {} (new size={}, previous={})' +- .format(vdi_uuid, new_size, old_size) +- ) +- +- LinstorVhdUtil(None, linstor).set_size_phys(vdi_path, new_size) +- # TODO: Change the LINSTOR volume size using linstor.resize_volume. +- +- + IPS_XHA_CACHE = None + + +@@ -1390,7 +1325,7 @@ class LinstorSR(SR.SR): + current_size - vhdutil.VHD_FOOTER_SIZE, + vhdutil.VHD_FOOTER_SIZE + ) +- deflate(self._linstor, vdi_uuid, vdi.path, old_size, current_size) ++ self._vhdutil.deflate(vdi_uuid, vdi.path, old_size, current_size) + + def _handle_interrupted_clone( + self, vdi_uuid, clone_info, force_undo=False +@@ -1502,9 +1437,9 @@ class LinstorSR(SR.SR): + # Inflate to the right size. + if base_type == vhdutil.VDI_TYPE_VHD: + vdi = self.vdi(vdi_uuid) +- volume_size = compute_volume_size(vdi.size, vdi.vdi_type) +- inflate( +- self._journaler, self._linstor, vdi_uuid, vdi.path, ++ volume_size = LinstorVhdUtil.compute_volume_size(vdi.size, vdi.vdi_type) ++ self._vhdutil.inflate( ++ self._journaler, vdi_uuid, vdi.path, + volume_size, vdi.capacity + ) + self.vdis[vdi_uuid] = vdi +@@ -1619,8 +1554,6 @@ class LinstorVDI(VDI.VDI): + TYPE_RAW = 'raw' + TYPE_VHD = 'vhd' + +- MAX_SIZE = 2 * 1024 * 1024 * 1024 * 1024 # Max VHD size. +- + # Metadata size given to the "S" param of vhd-util create. + # "-S size (MB) for metadata preallocation". + # Increase the performance when resize is called. +@@ -1717,7 +1650,7 @@ class LinstorVDI(VDI.VDI): + + # 2. Compute size and check space available. + size = vhdutil.validate_and_round_vhd_size(long(size)) +- volume_size = compute_volume_size(size, self.vdi_type) ++ volume_size = LinstorVhdUtil.compute_volume_size(size, self.vdi_type) + util.SMlog( + 'LinstorVDI.create: type={}, vhd-size={}, volume-size={}' + .format(self.vdi_type, size, volume_size) +@@ -1869,7 +1802,7 @@ class LinstorVDI(VDI.VDI): + if ( + self.vdi_type == vhdutil.VDI_TYPE_RAW or + not writable or +- self.capacity >= compute_volume_size(self.size, self.vdi_type) ++ self.capacity >= LinstorVhdUtil.compute_volume_size(self.size, self.vdi_type) + ): + need_inflate = False + +@@ -1938,7 +1871,7 @@ class LinstorVDI(VDI.VDI): + + # The VDI is already deflated if the VHD image size + metadata is + # equal to the LINSTOR volume size. +- volume_size = compute_volume_size(self.size, self.vdi_type) ++ volume_size = LinstorVhdUtil.compute_volume_size(self.size, self.vdi_type) + already_deflated = self.capacity <= volume_size + + if already_deflated: +@@ -2000,7 +1933,7 @@ class LinstorVDI(VDI.VDI): + + # Compute the virtual VHD and DRBD volume size. + size = vhdutil.validate_and_round_vhd_size(long(size)) +- volume_size = compute_volume_size(size, self.vdi_type) ++ volume_size = LinstorVhdUtil.compute_volume_size(size, self.vdi_type) + util.SMlog( + 'LinstorVDI.resize: type={}, vhd-size={}, volume-size={}' + .format(self.vdi_type, size, volume_size) +@@ -2033,7 +1966,7 @@ class LinstorVDI(VDI.VDI): + self._linstor.resize(self.uuid, new_volume_size) + else: + if new_volume_size != old_volume_size: +- inflate( ++ self.sr._vhdutil.inflate( + self.sr._journaler, self._linstor, self.uuid, self.path, + new_volume_size, old_volume_size + ) +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 1f898225..d2fb884f 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -1383,8 +1383,6 @@ class LVHDVDI(VDI): + class LinstorVDI(VDI): + """Object representing a VDI in a LINSTOR SR""" + +- MAX_SIZE = 2 * 1024 * 1024 * 1024 * 1024 # Max VHD size. +- + VOLUME_LOCK_TIMEOUT = 30 + + def load(self, info=None): +@@ -1408,8 +1406,41 @@ class LinstorVDI(VDI): + self.parentUuid = info.parentUuid + self.sizeVirt = info.sizeVirt + self._sizeVHD = info.sizePhys ++ self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) + self.hidden = info.hidden + self.scanError = False ++ self.vdi_type = vhdutil.VDI_TYPE_VHD ++ ++ def getDriverName(self): ++ if self.raw: ++ return self.DRIVER_NAME_RAW ++ return self.DRIVER_NAME_VHD ++ ++ def inflate(self, size): ++ if self.raw: ++ return ++ self.sr.lock() ++ try: ++ self.sr._vhdutil.inflate(self.sr.journaler, self.uuid, self.path, size, self.drbd_size) ++ finally: ++ self.sr.unlock() ++ self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) ++ self._sizeVHD = self.sr._vhdutil.get_size_phys(self.uuid) ++ ++ def deflate(self): ++ if self.raw: ++ return ++ self.sr.lock() ++ try: ++ self.sr._vhdutil.deflate(self.uuid, self.path, self._sizeVHD, self.drbd_size) ++ finally: ++ self.sr.unlock() ++ self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) ++ self._sizeVHD = self.sr._vhdutil.get_size_phys(self.uuid) ++ ++ def inflateFully(self): ++ if not self.raw: ++ self.inflate(LinstorVhdUtil.compute_volume_size(self.sizeVirt, self.vdi_type)) + + def rename(self, uuid): + Util.log('Renaming {} -> {} (path={})'.format( +@@ -1432,7 +1463,7 @@ class LinstorVDI(VDI): + VDI.delete(self) + + def validate(self, fast=False): +- if not self.sr._vhdutil.check(self.uuid, fast=fast): ++ if not self.raw and not self.sr._vhdutil.check(self.uuid, fast=fast): + raise util.SMException('VHD {} corrupted'.format(self)) + + def pause(self, failfast=False): +@@ -1444,18 +1475,6 @@ class LinstorVDI(VDI): + def coalesce(self): + # Note: We raise `SMException` here to skip the current coalesce in case of failure. + # Using another exception we can't execute the next coalesce calls. +- try: +- drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) +- except Exception as e: +- raise util.SMException( +- 'VDI {} could not be coalesced because the DRBD block size cannot be read: {}' +- .format(self.uuid, e)) +- +- if self._sizeVHD > drbd_size: +- raise util.SMException( +- 'VDI {} could not be coalesced because VHD phys size > DRBD block size ({} > {})' +- .format(self.uuid, self._sizeVHD, drbd_size)) +- + self.sr._vhdutil.force_coalesce(self.path) + + def getParent(self): +@@ -1505,6 +1524,22 @@ class LinstorVDI(VDI): + Util.log("Failed to update %s with vhd-parent field %s" % \ + (self.uuid, self.parentUuid)) + ++ def _doCoalesce(self): ++ try: ++ self._activateChain() ++ self.parent.validate() ++ self._inflateParentForCoalesce() ++ VDI._doCoalesce(self) ++ finally: ++ self.parent._sizeVHD = self.sr._vhdutil.get_size_phys(self.parent.uuid) ++ self.parent.deflate() ++ ++ def _activateChain(self): ++ vdi = self ++ while vdi: ++ p = self.sr._linstor.get_device_path(vdi.uuid) ++ vdi = vdi.parent ++ + def _setHidden(self, hidden=True): + HIDDEN_TAG = 'hidden' + +@@ -1516,9 +1551,52 @@ class LinstorVDI(VDI): + else: + VDI._setHidden(self, hidden) + ++ def _setSizeVirt(self, size): ++ jfile = self.uuid + '-jvhd' ++ self.sr._linstor.create_volume( ++ jfile, vhdutil.MAX_VHD_JOURNAL_SIZE, persistent=False, volume_name=jfile ++ ) ++ try: ++ self.inflate(LinstorVhdUtil.compute_volume_size(size, self.vdi_type)) ++ self.sr._vhdutil.set_size_virt(size, jfile) ++ finally: ++ try: ++ self.sr._linstor.destroy_volume(jfile) ++ except Exception: ++ # We can ignore it, in any case this volume is not persistent. ++ pass ++ + def _queryVHDBlocks(self): + return self.sr._vhdutil.get_block_bitmap(self.uuid) + ++ def _inflateParentForCoalesce(self): ++ if self.parent.raw: ++ return ++ inc = self._calcExtraSpaceForCoalescing() ++ if inc > 0: ++ self.parent.inflate(self.parent.drbd_size + inc) ++ ++ def _calcExtraSpaceForCoalescing(self): ++ if self.parent.raw: ++ return 0 ++ size_coalesced = LinstorVhdUtil.compute_volume_size( ++ self._getCoalescedSizeData(), self.vdi_type ++ ) ++ Util.log("Coalesced size = %s" % Util.num2str(size_coalesced)) ++ return size_coalesced - self.parent.drbd_size ++ ++ def _calcExtraSpaceForLeafCoalescing(self): ++ assert self.drbd_size > 0 ++ assert self._sizeVHD > 0 ++ deflate_diff = self.drbd_size - LinstorVolumeManager.round_up_volume_size(self._sizeVHD) ++ assert deflate_diff >= 0 ++ return self._calcExtraSpaceForCoalescing() - deflate_diff ++ ++ def _calcExtraSpaceForSnapshotCoalescing(self): ++ assert self._sizeVHD > 0 ++ return self._calcExtraSpaceForCoalescing() + \ ++ LinstorVolumeManager.round_up_volume_size(self._sizeVHD) ++ + ################################################################################ + # + # SR +@@ -3062,17 +3140,19 @@ class LinstorSR(SR): + + return all_vdi_info + +- # TODO: Maybe implement _liveLeafCoalesce/_prepareCoalesceLeaf/ +- # _finishCoalesceLeaf/_updateSlavesOnResize like LVM plugin. ++ def _prepareCoalesceLeaf(self, vdi): ++ vdi._activateChain() ++ vdi.deflate() ++ vdi._inflateParentForCoalesce() ++ ++ def _finishCoalesceLeaf(self, parent): ++ if not parent.isSnapshot() or parent.isAttachedRW(): ++ parent.inflateFully() ++ else: ++ parent.deflate() + + def _calcExtraSpaceNeeded(self, child, parent): +- meta_overhead = vhdutil.calcOverheadEmpty(LinstorVDI.MAX_SIZE) +- bitmap_overhead = vhdutil.calcOverheadBitmap(parent.sizeVirt) +- virtual_size = LinstorVolumeManager.round_up_volume_size( +- parent.sizeVirt + meta_overhead + bitmap_overhead +- ) +- volume_size = self._linstor.get_volume_size(parent.uuid) +- return virtual_size - volume_size ++ return self._linstor.compute_volume_size(parent.sizeVirt, parent.vdi_type) - parent.drbd_size + + def _hasValidDevicePath(self, uuid): + try: +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 3ce7ab9e..239a10ad 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -14,6 +14,8 @@ + # You should have received a copy of the GNU General Public License + # along with this program. If not, see . + ++from linstorjournaler import LinstorJournaler ++from linstorvolumemanager import LinstorVolumeManager + import base64 + import distutils.util + import errno +@@ -133,6 +135,8 @@ def linstormodifier(): + + + class LinstorVhdUtil: ++ MAX_SIZE = 2 * 1024 * 1024 * 1024 * 1024 # Max VHD size. ++ + def __init__(self, session, linstor): + self._session = session + self._linstor = linstor +@@ -225,6 +229,10 @@ class LinstorVhdUtil: + def create(self, path, size, static, msize=0): + return self._call_local_vhd_util_or_fail(vhdutil.create, path, size, static, msize) + ++ @linstormodifier() ++ def set_size_virt(self, path, size, jfile): ++ return self._call_local_vhd_util_or_fail(vhdutil.setSizeVirt, path, size, jfile) ++ + @linstormodifier() + def set_size_virt_fast(self, path, size): + return self._call_local_vhd_util_or_fail(vhdutil.setSizeVirtFast, path, size) +@@ -253,6 +261,56 @@ class LinstorVhdUtil: + def snapshot(self, path, parent, parentRaw, msize=0, checkEmpty=True): + return self._call_local_vhd_util_or_fail(vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) + ++ def inflate(self, journaler, vdi_uuid, vdi_path, new_size, old_size): ++ # Only inflate if the LINSTOR volume capacity is not enough. ++ new_size = LinstorVolumeManager.round_up_volume_size(new_size) ++ if new_size <= old_size: ++ return ++ ++ util.SMlog( ++ 'Inflate {} (size={}, previous={})' ++ .format(vdi_uuid, new_size, old_size) ++ ) ++ ++ journaler.create( ++ LinstorJournaler.INFLATE, vdi_uuid, old_size ++ ) ++ self._linstor.resize_volume(vdi_uuid, new_size) ++ ++ # TODO: Replace pylint comment with this feature when possible: ++ # https://github.com/PyCQA/pylint/pull/2926 ++ result_size = self.get_drbd_size(vdi_uuid) # pylint: disable = E1120 ++ if result_size < new_size: ++ util.SMlog( ++ 'WARNING: Cannot inflate volume to {}B, result size: {}B' ++ .format(new_size, result_size) ++ ) ++ ++ if not util.zeroOut( ++ vdi_path, result_size - vhdutil.VHD_FOOTER_SIZE, ++ vhdutil.VHD_FOOTER_SIZE ++ ): ++ raise xs_errors.XenError( ++ 'EIO', ++ opterr='Failed to zero out VHD footer {}'.format(vdi_path) ++ ) ++ ++ self.set_size_phys(vdi_path, result_size, False) ++ journaler.remove(LinstorJournaler.INFLATE, vdi_uuid) ++ ++ def deflate(self, vdi_uuid, vdi_path, new_size, old_size): ++ new_size = LinstorVolumeManager.round_up_volume_size(new_size) ++ if new_size >= old_size: ++ return ++ ++ util.SMlog( ++ 'Deflate {} (new size={}, previous={})' ++ .format(vdi_uuid, new_size, old_size) ++ ) ++ ++ self.set_size_phys(vdi_path, new_size) ++ # TODO: Change the LINSTOR volume size using linstor.resize_volume. ++ + # -------------------------------------------------------------------------- + # Remote setters: write locally and try on another host in case of failure. + # -------------------------------------------------------------------------- +@@ -273,6 +331,23 @@ class LinstorVhdUtil: + def force_repair(self, path): + return self._call_vhd_util(vhdutil.repair, 'repair', path, use_parent=False) + ++ # -------------------------------------------------------------------------- ++ # Static helpers. ++ # -------------------------------------------------------------------------- ++ ++ @classmethod ++ def compute_volume_size(cls, virtual_size, image_type): ++ if image_type == vhdutil.VDI_TYPE_VHD: ++ # All LINSTOR VDIs have the metadata area preallocated for ++ # the maximum possible virtual size (for fast online VDI.resize). ++ meta_overhead = vhdutil.calcOverheadEmpty(cls.MAX_SIZE) ++ bitmap_overhead = vhdutil.calcOverheadBitmap(virtual_size) ++ virtual_size += meta_overhead + bitmap_overhead ++ elif image_type != vhdutil.VDI_TYPE_RAW: ++ raise Exception('Invalid image type: {}'.format(image_type)) ++ ++ return LinstorVolumeManager.round_up_volume_size(virtual_size) ++ + # -------------------------------------------------------------------------- + # Helpers. + # -------------------------------------------------------------------------- diff --git a/SOURCES/0144-fix-LinstorSR-remove-useless-IPS_XHA_CACHE-var.patch b/SOURCES/0144-fix-LinstorSR-remove-useless-IPS_XHA_CACHE-var.patch new file mode 100644 index 0000000..c147a46 --- /dev/null +++ b/SOURCES/0144-fix-LinstorSR-remove-useless-IPS_XHA_CACHE-var.patch @@ -0,0 +1,28 @@ +From 79d4085610ef1ca85107eccd5a6e79f91d460027 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 20 Nov 2023 10:52:27 +0100 +Subject: [PATCH 144/177] fix(LinstorSR): remove useless IPS_XHA_CACHE var + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 6 ------ + 1 file changed, 6 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 6802f494..c570925f 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -210,13 +210,7 @@ def detach_thin(session, linstor, sr_uuid, vdi_uuid): + util.SMlog('Failed to detach properly VDI {}: {}'.format(vdi_uuid, e)) + + +-IPS_XHA_CACHE = None +- +- + def get_ips_from_xha_config_file(): +- if IPS_XHA_CACHE: +- return IPS_XHA_CACHE +- + ips = dict() + host_id = None + try: diff --git a/SOURCES/0145-fix-LinstorSR-ensure-we-can-deflate-on-any-host-afte.patch b/SOURCES/0145-fix-LinstorSR-ensure-we-can-deflate-on-any-host-afte.patch new file mode 100644 index 0000000..d10edce --- /dev/null +++ b/SOURCES/0145-fix-LinstorSR-ensure-we-can-deflate-on-any-host-afte.patch @@ -0,0 +1,315 @@ +From 5cd04079c9bca752f6f78532ad1ae0bf962c3902 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 21 Nov 2023 13:50:24 +0100 +Subject: [PATCH 145/177] fix(LinstorSR): ensure we can deflate on any host + after a journal rollback + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 10 +---- + drivers/cleanup.py | 2 +- + drivers/linstor-manager | 24 ++++++++++++ + drivers/linstorvhdutil.py | 82 +++++++++++++++++++++++---------------- + 4 files changed, 76 insertions(+), 42 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index c570925f..1d1b51a8 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -197,7 +197,7 @@ def detach_thin_impl(session, linstor, sr_uuid, vdi_uuid): + + volume_info = linstor.get_volume_info(vdi_uuid) + old_volume_size = volume_info.virtual_size +- vhdutil_inst.deflate(vdi_uuid, device_path, new_volume_size, old_volume_size) ++ vhdutil_inst.deflate(device_path, new_volume_size, old_volume_size) + + + def detach_thin(session, linstor, sr_uuid, vdi_uuid): +@@ -1313,13 +1313,7 @@ class LinstorSR(SR.SR): + + current_size = volume_info.virtual_size + assert current_size > 0 +- +- util.zeroOut( +- vdi.path, +- current_size - vhdutil.VHD_FOOTER_SIZE, +- vhdutil.VHD_FOOTER_SIZE +- ) +- self._vhdutil.deflate(vdi_uuid, vdi.path, old_size, current_size) ++ self._vhdutil.force_deflate(vdi.path, old_size, current_size, zeroize=True) + + def _handle_interrupted_clone( + self, vdi_uuid, clone_info, force_undo=False +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index d2fb884f..3c2810cd 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -1432,7 +1432,7 @@ class LinstorVDI(VDI): + return + self.sr.lock() + try: +- self.sr._vhdutil.deflate(self.uuid, self.path, self._sizeVHD, self.drbd_size) ++ self.sr._vhdutil.force_deflate(self.path, self._sizeVHD, self.drbd_size, zeroize=False) + finally: + self.sr.unlock() + self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 9e5e1d6e..7726c1b7 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -28,6 +28,7 @@ import XenAPI + import XenAPIPlugin + + from linstorjournaler import LinstorJournaler ++from linstorvhdutil import LinstorVhdUtil + from linstorvolumemanager import get_controller_uri, get_local_volume_openers, LinstorVolumeManager + from lock import Lock + import json +@@ -528,6 +529,26 @@ def repair(session, args): + raise + + ++def deflate(session, args): ++ try: ++ device_path = args['devicePath'] ++ new_size = int(args['newSize']) ++ old_size = int(args['oldSize']) ++ zeroize = distutils.util.strtobool(args['zeroize']) ++ group_name = args['groupName'] ++ ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ LinstorVhdUtil(session, linstor).deflate(device_path, new_size, old_size, zeroize) ++ return '' ++ except Exception as e: ++ util.SMlog('linstor-manager:deflate error: {}'.format(e)) ++ raise ++ ++ + def lock_vdi(session, args): + lock = None + try: +@@ -990,6 +1011,9 @@ if __name__ == '__main__': + 'coalesce': coalesce, + 'repair': repair, + ++ # Misc writters. ++ 'deflate': deflate, ++ + 'lockVdi': lock_vdi, + 'hasControllerRunning': has_controller_running, + 'addHost': add_host, +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 239a10ad..23d8b6a0 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -31,7 +31,7 @@ MANAGER_PLUGIN = 'linstor-manager' + EMEDIUMTYPE = 124 + + +-def call_vhd_util_on_host(session, host_ref, method, device_path, args): ++def call_remote_method(session, host_ref, method, device_path, args): + try: + response = session.xenapi.host.call_plugin( + host_ref, MANAGER_PLUGIN, method, args +@@ -86,7 +86,7 @@ def linstorhostcall(local_method, remote_method): + local_e = None + try: + if not in_use_by or socket.gethostname() in node_names: +- return self._call_local_vhd_util(local_method, device_path, *args[2:], **kwargs) ++ return self._call_local_method(local_method, device_path, *args[2:], **kwargs) + except ErofsLinstorCallException as e: + local_e = e.cmd_err + except Exception as e: +@@ -112,7 +112,7 @@ def linstorhostcall(local_method, remote_method): + try: + def remote_call(): + host_ref = self._get_readonly_host(vdi_uuid, device_path, node_names) +- return call_vhd_util_on_host(self._session, host_ref, remote_method, device_path, remote_args) ++ return call_remote_method(self._session, host_ref, remote_method, device_path, remote_args) + response = util.retry(remote_call, 5, 2) + except Exception as remote_e: + self._raise_openers_exception(device_path, local_e or remote_e) +@@ -227,39 +227,39 @@ class LinstorVhdUtil: + + @linstormodifier() + def create(self, path, size, static, msize=0): +- return self._call_local_vhd_util_or_fail(vhdutil.create, path, size, static, msize) ++ return self._call_local_method_or_fail(vhdutil.create, path, size, static, msize) + + @linstormodifier() + def set_size_virt(self, path, size, jfile): +- return self._call_local_vhd_util_or_fail(vhdutil.setSizeVirt, path, size, jfile) ++ return self._call_local_method_or_fail(vhdutil.setSizeVirt, path, size, jfile) + + @linstormodifier() + def set_size_virt_fast(self, path, size): +- return self._call_local_vhd_util_or_fail(vhdutil.setSizeVirtFast, path, size) ++ return self._call_local_method_or_fail(vhdutil.setSizeVirtFast, path, size) + + @linstormodifier() + def set_size_phys(self, path, size, debug=True): +- return self._call_local_vhd_util_or_fail(vhdutil.setSizePhys, path, size, debug) ++ return self._call_local_method_or_fail(vhdutil.setSizePhys, path, size, debug) + + @linstormodifier() + def set_parent(self, path, parentPath, parentRaw=False): +- return self._call_local_vhd_util_or_fail(vhdutil.setParent, path, parentPath, parentRaw) ++ return self._call_local_method_or_fail(vhdutil.setParent, path, parentPath, parentRaw) + + @linstormodifier() + def set_hidden(self, path, hidden=True): +- return self._call_local_vhd_util_or_fail(vhdutil.setHidden, path, hidden) ++ return self._call_local_method_or_fail(vhdutil.setHidden, path, hidden) + + @linstormodifier() + def set_key(self, path, key_hash): +- return self._call_local_vhd_util_or_fail(vhdutil.setKey, path, key_hash) ++ return self._call_local_method_or_fail(vhdutil.setKey, path, key_hash) + + @linstormodifier() + def kill_data(self, path): +- return self._call_local_vhd_util_or_fail(vhdutil.killData, path) ++ return self._call_local_method_or_fail(vhdutil.killData, path) + + @linstormodifier() + def snapshot(self, path, parent, parentRaw, msize=0, checkEmpty=True): +- return self._call_local_vhd_util_or_fail(vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) ++ return self._call_local_method_or_fail(vhdutil.snapshot, path, parent, parentRaw, msize, checkEmpty) + + def inflate(self, journaler, vdi_uuid, vdi_path, new_size, old_size): + # Only inflate if the LINSTOR volume capacity is not enough. +@@ -269,7 +269,7 @@ class LinstorVhdUtil: + + util.SMlog( + 'Inflate {} (size={}, previous={})' +- .format(vdi_uuid, new_size, old_size) ++ .format(vdi_path, new_size, old_size) + ) + + journaler.create( +@@ -286,26 +286,22 @@ class LinstorVhdUtil: + .format(new_size, result_size) + ) + +- if not util.zeroOut( +- vdi_path, result_size - vhdutil.VHD_FOOTER_SIZE, +- vhdutil.VHD_FOOTER_SIZE +- ): +- raise xs_errors.XenError( +- 'EIO', +- opterr='Failed to zero out VHD footer {}'.format(vdi_path) +- ) +- ++ self._zeroize(vdi_path, result_size - vhdutil.VHD_FOOTER_SIZE) + self.set_size_phys(vdi_path, result_size, False) + journaler.remove(LinstorJournaler.INFLATE, vdi_uuid) + +- def deflate(self, vdi_uuid, vdi_path, new_size, old_size): ++ def deflate(self, vdi_path, new_size, old_size, zeroize=False): ++ if zeroize: ++ assert old_size > vhdutil.VHD_FOOTER_SIZE ++ self._zeroize(vdi_path, old_size - vhdutil.VHD_FOOTER_SIZE) ++ + new_size = LinstorVolumeManager.round_up_volume_size(new_size) + if new_size >= old_size: + return + + util.SMlog( + 'Deflate {} (new size={}, previous={})' +- .format(vdi_uuid, new_size, old_size) ++ .format(vdi_path, new_size, old_size) + ) + + self.set_size_phys(vdi_path, new_size) +@@ -321,15 +317,27 @@ class LinstorVhdUtil: + 'parentPath': str(parentPath), + 'parentRaw': parentRaw + } +- return self._call_vhd_util(vhdutil.setParent, 'setParent', path, use_parent=False, **kwargs) ++ return self._call_method(vhdutil.setParent, 'setParent', path, use_parent=False, **kwargs) + + @linstormodifier() + def force_coalesce(self, path): +- return self._call_vhd_util(vhdutil.coalesce, 'coalesce', path, use_parent=True) ++ return self._call_method(vhdutil.coalesce, 'coalesce', path, use_parent=True) + + @linstormodifier() + def force_repair(self, path): +- return self._call_vhd_util(vhdutil.repair, 'repair', path, use_parent=False) ++ return self._call_method(vhdutil.repair, 'repair', path, use_parent=False) ++ ++ @linstormodifier() ++ def force_deflate(self, path, newSize, oldSize, zeroize): ++ kwargs = { ++ 'newSize': newSize, ++ 'oldSize': oldSize, ++ 'zeroize': zeroize ++ } ++ return self._call_method('_force_deflate', 'deflate', path, use_parent=False, **kwargs) ++ ++ def _force_deflate(self, path, newSize, oldSize, zeroize): ++ self.deflate(path, newSize, oldSize, zeroize) + + # -------------------------------------------------------------------------- + # Static helpers. +@@ -409,7 +417,7 @@ class LinstorVhdUtil: + util.SMlog('raise opener exception: {}'.format(e_wrapper)) + raise e_wrapper # pylint: disable = E0702 + +- def _call_local_vhd_util(self, local_method, device_path, *args, **kwargs): ++ def _call_local_method(self, local_method, device_path, *args, **kwargs): + if isinstance(local_method, str): + local_method = getattr(self, local_method) + +@@ -429,14 +437,14 @@ class LinstorVhdUtil: + util.SMlog('failed to execute locally vhd-util (sys {})'.format(e.code)) + raise e + +- def _call_local_vhd_util_or_fail(self, local_method, device_path, *args, **kwargs): ++ def _call_local_method_or_fail(self, local_method, device_path, *args, **kwargs): + try: +- return self._call_local_vhd_util(local_method, device_path, *args, **kwargs) ++ return self._call_local_method(local_method, device_path, *args, **kwargs) + except ErofsLinstorCallException as e: + # Volume is locked on a host, find openers. + self._raise_openers_exception(device_path, e.cmd_err) + +- def _call_vhd_util(self, local_method, remote_method, device_path, use_parent, *args, **kwargs): ++ def _call_method(self, local_method, remote_method, device_path, use_parent, *args, **kwargs): + # Note: `use_parent` exists to know if the VHD parent is used by the local/remote method. + # Normally in case of failure, if the parent is unused we try to execute the method on + # another host using the DRBD opener list. In the other case, if the parent is required, +@@ -447,7 +455,7 @@ class LinstorVhdUtil: + + # A. Try to write locally... + try: +- return self._call_local_vhd_util(local_method, device_path, *args, **kwargs) ++ return self._call_local_method(local_method, device_path, *args, **kwargs) + except Exception: + pass + +@@ -504,7 +512,7 @@ class LinstorVhdUtil: + + no_host_found = False + try: +- return call_vhd_util_on_host(self._session, host_ref, remote_method, device_path, remote_args) ++ return call_remote_method(self._session, host_ref, remote_method, device_path, remote_args) + except Exception: + pass + +@@ -520,3 +528,11 @@ class LinstorVhdUtil: + .format(remote_method, device_path, openers) + ) + return util.retry(remote_call, 5, 2) ++ ++ @staticmethod ++ def _zeroize(path, size): ++ if not util.zeroOut(path, size, vhdutil.VHD_FOOTER_SIZE): ++ raise xs_errors.XenError( ++ 'EIO', ++ opterr='Failed to zero out VHD footer {}'.format(path) ++ ) diff --git a/SOURCES/0146-fix-LinstorSR-ensure-we-always-use-real-DRBD-VHD-siz.patch b/SOURCES/0146-fix-LinstorSR-ensure-we-always-use-real-DRBD-VHD-siz.patch new file mode 100644 index 0000000..bfb2bb1 --- /dev/null +++ b/SOURCES/0146-fix-LinstorSR-ensure-we-always-use-real-DRBD-VHD-siz.patch @@ -0,0 +1,146 @@ +From f3726fc7b992ceb2d51e3e1725cc1cb6eff2dc77 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 27 Nov 2023 12:15:43 +0100 +Subject: [PATCH 146/177] fix(LinstorSR): ensure we always use real DRBD/VHD + sizes in inflate/deflate GC calls + +--- + drivers/cleanup.py | 54 +++++++++++++++++++++++++--------------------- + 1 file changed, 30 insertions(+), 24 deletions(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 3c2810cd..8635f488 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -1405,38 +1405,49 @@ class LinstorVDI(VDI): + + self.parentUuid = info.parentUuid + self.sizeVirt = info.sizeVirt +- self._sizeVHD = info.sizePhys +- self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) ++ self._sizeVHD = -1 ++ self.drbd_size = -1 + self.hidden = info.hidden + self.scanError = False + self.vdi_type = vhdutil.VDI_TYPE_VHD + +- def getDriverName(self): +- if self.raw: +- return self.DRIVER_NAME_RAW +- return self.DRIVER_NAME_VHD ++ def getSizeVHD(self, fetch=False): ++ if self._sizeVHD < 0 or fetch: ++ self._sizeVHD = self.sr._vhdutil.get_size_phys(self.uuid) ++ return self._sizeVHD ++ ++ def getDrbdSize(self, fetch=False): ++ if self.drbd_size < 0 or fetch: ++ self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) ++ return self.drbd_size + + def inflate(self, size): + if self.raw: + return + self.sr.lock() + try: ++ # Ensure we use the real DRBD size and not the cached one. ++ # Why? Because this attribute can be changed if volume is resized by user. ++ self.drbd_size = self.getDrbdSize(fetch=True) + self.sr._vhdutil.inflate(self.sr.journaler, self.uuid, self.path, size, self.drbd_size) + finally: + self.sr.unlock() +- self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) +- self._sizeVHD = self.sr._vhdutil.get_size_phys(self.uuid) ++ self.drbd_size = -1 ++ self._sizeVHD = -1 + + def deflate(self): + if self.raw: + return + self.sr.lock() + try: ++ # Ensure we use the real sizes and not the cached info. ++ self.drbd_size = self.getDrbdSize(fetch=True) ++ self._sizeVHD = self.getSizeVHD(fetch=True) + self.sr._vhdutil.force_deflate(self.path, self._sizeVHD, self.drbd_size, zeroize=False) + finally: + self.sr.unlock() +- self.drbd_size = self.sr._vhdutil.get_drbd_size(self.uuid) +- self._sizeVHD = self.sr._vhdutil.get_size_phys(self.uuid) ++ self.drbd_size = -1 ++ self._sizeVHD = -1 + + def inflateFully(self): + if not self.raw: +@@ -1512,6 +1523,7 @@ class LinstorVDI(VDI): + self.children = [] + + def _setParent(self, parent): ++ self.sr._linstor.get_device_path(self.uuid) + self.sr._vhdutil.force_parent(self.path, parent.path) + self.parent = parent + self.parentUuid = parent.uuid +@@ -1531,7 +1543,6 @@ class LinstorVDI(VDI): + self._inflateParentForCoalesce() + VDI._doCoalesce(self) + finally: +- self.parent._sizeVHD = self.sr._vhdutil.get_size_phys(self.parent.uuid) + self.parent.deflate() + + def _activateChain(self): +@@ -1574,7 +1585,7 @@ class LinstorVDI(VDI): + return + inc = self._calcExtraSpaceForCoalescing() + if inc > 0: +- self.parent.inflate(self.parent.drbd_size + inc) ++ self.parent.inflate(self.parent.getDrbdSize() + inc) + + def _calcExtraSpaceForCoalescing(self): + if self.parent.raw: +@@ -1583,19 +1594,19 @@ class LinstorVDI(VDI): + self._getCoalescedSizeData(), self.vdi_type + ) + Util.log("Coalesced size = %s" % Util.num2str(size_coalesced)) +- return size_coalesced - self.parent.drbd_size ++ return size_coalesced - self.parent.getDrbdSize() + + def _calcExtraSpaceForLeafCoalescing(self): +- assert self.drbd_size > 0 +- assert self._sizeVHD > 0 +- deflate_diff = self.drbd_size - LinstorVolumeManager.round_up_volume_size(self._sizeVHD) ++ assert self.getDrbdSize() > 0 ++ assert self.getSizeVHD() > 0 ++ deflate_diff = self.getDrbdSize() - LinstorVolumeManager.round_up_volume_size(self.getSizeVHD()) + assert deflate_diff >= 0 + return self._calcExtraSpaceForCoalescing() - deflate_diff + + def _calcExtraSpaceForSnapshotCoalescing(self): +- assert self._sizeVHD > 0 ++ assert self.getSizeVHD() > 0 + return self._calcExtraSpaceForCoalescing() + \ +- LinstorVolumeManager.round_up_volume_size(self._sizeVHD) ++ LinstorVolumeManager.round_up_volume_size(self.getSizeVHD()) + + ################################################################################ + # +@@ -3152,7 +3163,7 @@ class LinstorSR(SR): + parent.deflate() + + def _calcExtraSpaceNeeded(self, child, parent): +- return self._linstor.compute_volume_size(parent.sizeVirt, parent.vdi_type) - parent.drbd_size ++ return LinstorVhdUtil.compute_volume_size(parent.sizeVirt, parent.vdi_type) - parent.getDrbdSize() + + def _hasValidDevicePath(self, uuid): + try: +@@ -3172,11 +3183,6 @@ class LinstorSR(SR): + finally: + self.unlock() + +- def _prepareCoalesceLeaf(self, vdi): +- # Move diskless path if necessary. We must have an access +- # to modify locally the volume. +- self._linstor.get_device_path(vdi.uuid) +- + def _handleInterruptedCoalesceLeaf(self): + entries = self.journaler.get_all(VDI.JRN_LEAF) + for uuid, parentUuid in entries.iteritems(): diff --git a/SOURCES/0147-feat-linstor-kv-tool-If-no-controller-uri-option-is-.patch b/SOURCES/0147-feat-linstor-kv-tool-If-no-controller-uri-option-is-.patch new file mode 100644 index 0000000..6d24bfa --- /dev/null +++ b/SOURCES/0147-feat-linstor-kv-tool-If-no-controller-uri-option-is-.patch @@ -0,0 +1,53 @@ +From 95d463f76f5227f4dd4726c50695c739882f92a2 Mon Sep 17 00:00:00 2001 +From: BenjiReis +Date: Mon, 27 Nov 2023 14:23:57 +0100 +Subject: [PATCH 147/177] feat(linstor-kv-tool): If no controller uri option is + provided fetch it (#48) + +Signed-off-by: BenjiReis +--- + scripts/linstor-kv-tool | 14 ++++++++++---- + 1 file changed, 10 insertions(+), 4 deletions(-) + +diff --git a/scripts/linstor-kv-tool b/scripts/linstor-kv-tool +index c9070270..b845ec2b 100755 +--- a/scripts/linstor-kv-tool ++++ b/scripts/linstor-kv-tool +@@ -13,6 +13,10 @@ + # + # You should have received a copy of the GNU General Public License + # along with this program. If not, see . ++import sys ++sys.path[0] = '/opt/xensource/sm/' ++ ++from linstorvolumemanager import get_controller_uri + + import argparse + import json +@@ -56,7 +60,7 @@ def remove_all_volumes(controller_uri, group_name): + + def main(): + parser = argparse.ArgumentParser() +- parser.add_argument('-u', '--uri', required=True) ++ parser.add_argument('-u', '--uri', required=False) + parser.add_argument('-g', '--group-name', required=True) + parser.add_argument('-n', '--namespace', default='/') + +@@ -66,12 +70,14 @@ def main(): + action.add_argument('--remove-all-volumes', action='store_true') + + args = parser.parse_args() ++ controller_uri = get_controller_uri() if args.uri is None else args.uri ++ + if args.dump_volumes: +- dump_kv(args.uri, args.group_name, args.namespace) ++ dump_kv(controller_uri, args.group_name, args.namespace) + elif args.remove_volume: +- remove_volume(args.uri, args.group_name, args.remove_volume) ++ remove_volume(controller_uri, args.group_name, args.remove_volume) + elif args.remove_all_volumes: +- remove_all_volumes(args.uri, args.group_name) ++ remove_all_volumes(controller_uri, args.group_name) + + + if __name__ == '__main__': diff --git a/SOURCES/0148-fix-linstorvolumemanager-robustify-SR-destroy-46.patch b/SOURCES/0148-fix-linstorvolumemanager-robustify-SR-destroy-46.patch new file mode 100644 index 0000000..c7b6e7e --- /dev/null +++ b/SOURCES/0148-fix-linstorvolumemanager-robustify-SR-destroy-46.patch @@ -0,0 +1,207 @@ +From b8e4a29597f8dc8034e4ea307cd5e347d6a3b631 Mon Sep 17 00:00:00 2001 +From: BenjiReis +Date: Wed, 29 Nov 2023 15:45:32 +0100 +Subject: [PATCH 148/177] fix(linstorvolumemanager): robustify SR destroy (#46) + +Signed-off-by: Ronan Abhamon +Co-authored-by: BenjiReis +--- + drivers/linstorvolumemanager.py | 118 +++++++++++++++++++++++--------- + 1 file changed, 86 insertions(+), 32 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 47ca4e87..d27c84e6 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1372,35 +1372,63 @@ class LinstorVolumeManager(object): + :param bool force: Try to destroy volumes before if true. + """ + ++ # 1. Ensure volume list is empty. No cost. + if self._volumes: + raise LinstorVolumeManagerError( + 'Cannot destroy LINSTOR volume manager: ' + 'It exists remaining volumes' + ) + ++ # 2. Fetch ALL resource names. ++ # This list may therefore contain volumes created outside ++ # the scope of the driver. ++ resource_names = self._fetch_resource_names(ignore_deleted=False) ++ try: ++ resource_names.remove(DATABASE_VOLUME_NAME) ++ except KeyError: ++ # Really strange to reach that point. ++ # Normally we always have the database volume in the list. ++ pass ++ ++ # 3. Ensure the resource name list is entirely empty... ++ if resource_names: ++ raise LinstorVolumeManagerError( ++ 'Cannot destroy LINSTOR volume manager: ' ++ 'It exists remaining volumes (created externally or being deleted)' ++ ) ++ ++ # 4. Destroying... + controller_is_running = self._controller_is_running() + uri = 'linstor://localhost' + try: + if controller_is_running: + self._start_controller(start=False) + +- # 1. Umount LINSTOR database. ++ # 4.1. Umount LINSTOR database. + self._mount_database_volume( + self.build_device_path(DATABASE_VOLUME_NAME), + mount=False, + force=True + ) + +- # 2. Refresh instance. ++ # 4.2. Refresh instance. + self._start_controller(start=True) + self._linstor = self._create_linstor_instance( + uri, keep_uri_unmodified=True + ) + +- # 3. Destroy database volume. ++ # 4.3. Destroy database volume. + self._destroy_resource(DATABASE_VOLUME_NAME) + +- # 4. Destroy group and storage pools. ++ # 4.4. Refresh linstor connection. ++ # Without we get this error: ++ # "Cannot delete resource group 'xcp-sr-linstor_group_thin_device' because it has existing resource definitions.." ++ # Because the deletion of the databse was not seen by Linstor for some reason. ++ # It seems a simple refresh of the Linstor connection make it aware of the deletion. ++ self._linstor.disconnect() ++ self._linstor.connect() ++ ++ # 4.5. Destroy group and storage pools. + self._destroy_resource_group(self._linstor, self._group_name) + for pool in self._get_storage_pools(force=True): + self._destroy_storage_pool( +@@ -1711,10 +1739,18 @@ class LinstorVolumeManager(object): + if lin.resource_group_list_raise( + [group_name] + ).resource_groups: +- raise LinstorVolumeManagerError( +- 'Unable to create SR `{}`: The group name already exists' +- .format(group_name) +- ) ++ if not lin.resource_dfn_list_raise().resource_definitions: ++ backup_path = cls._create_database_backup_path() ++ logger( ++ 'Group name already exists `{}` without LVs. ' ++ 'Ignoring and moving the config files in {}'.format(group_name, backup_path) ++ ) ++ cls._move_files(DATABASE_PATH, backup_path) ++ else: ++ raise LinstorVolumeManagerError( ++ 'Unable to create SR `{}`: The group name already exists' ++ .format(group_name) ++ ) + + if thin_provisioning: + driver_pool_parts = driver_pool_name.split('/') +@@ -1773,18 +1809,31 @@ class LinstorVolumeManager(object): + ) + + # 2.b. Create resource group. +- result = lin.resource_group_create( +- name=group_name, +- place_count=redundancy, +- storage_pool=group_name, +- diskless_on_remaining=False +- ) +- error_str = cls._get_error_str(result) +- if error_str: ++ rg_creation_attempt = 0 ++ while True: ++ result = lin.resource_group_create( ++ name=group_name, ++ place_count=redundancy, ++ storage_pool=group_name, ++ diskless_on_remaining=False ++ ) ++ error_str = cls._get_error_str(result) ++ if not error_str: ++ break ++ ++ errors = cls._filter_errors(result) ++ if cls._check_errors(errors, [linstor.consts.FAIL_EXISTS_RSC_GRP]): ++ rg_creation_attempt += 1 ++ if rg_creation_attempt < 2: ++ try: ++ cls._destroy_resource_group(lin, group_name) ++ except Exception as e: ++ error_str = 'Failed to destroy old and empty RG: {}'.format(e) ++ else: ++ continue ++ + raise LinstorVolumeManagerError( +- 'Could not create RG `{}`: {}'.format( +- group_name, error_str +- ) ++ 'Could not create RG `{}`: {}'.format(group_name, error_str) + ) + + # 2.c. Create volume group. +@@ -1961,12 +2010,14 @@ class LinstorVolumeManager(object): + ) + return result[0].candidates + +- def _fetch_resource_names(self): ++ def _fetch_resource_names(self, ignore_deleted=True): + resource_names = set() + dfns = self._linstor.resource_dfn_list_raise().resource_definitions + for dfn in dfns: +- if dfn.resource_group_name == self._group_name and \ +- linstor.consts.FLAG_DELETE not in dfn.flags: ++ if dfn.resource_group_name == self._group_name and ( ++ ignore_deleted or ++ linstor.consts.FLAG_DELETE not in dfn.flags ++ ): + resource_names.add(dfn.name) + return resource_names + +@@ -2680,19 +2731,10 @@ class LinstorVolumeManager(object): + + @classmethod + def _mount_database_volume(cls, volume_path, mount=True, force=False): +- backup_path = DATABASE_PATH + '-' + str(uuid.uuid4()) +- + try: + # 1. Create a backup config folder. + database_not_empty = bool(os.listdir(DATABASE_PATH)) +- if database_not_empty: +- try: +- os.mkdir(backup_path) +- except Exception as e: +- raise LinstorVolumeManagerError( +- 'Failed to create backup path {} of LINSTOR config: {}' +- .format(backup_path, e) +- ) ++ backup_path = cls._create_database_backup_path() + + # 2. Move the config in the mounted volume. + if database_not_empty: +@@ -2861,6 +2903,18 @@ class LinstorVolumeManager(object): + ) + ) + ++ @staticmethod ++ def _create_database_backup_path(): ++ path = DATABASE_PATH + '-' + str(uuid.uuid4()) ++ try: ++ os.mkdir(path) ++ return path ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to create backup path {} of LINSTOR config: {}' ++ .format(path, e) ++ ) ++ + @staticmethod + def _get_filtered_properties(properties): + return dict(properties.items()) diff --git a/SOURCES/0149-feat-linstor-manager-extend-API-with-createNodeInter.patch b/SOURCES/0149-feat-linstor-manager-extend-API-with-createNodeInter.patch new file mode 100644 index 0000000..2d0c241 --- /dev/null +++ b/SOURCES/0149-feat-linstor-manager-extend-API-with-createNodeInter.patch @@ -0,0 +1,105 @@ +From 9a3af8ca152d5f94314886acb12f1971892bdf22 Mon Sep 17 00:00:00 2001 +From: BenjiReis +Date: Wed, 29 Nov 2023 16:54:30 +0100 +Subject: [PATCH 149/177] feat(linstor-manager): extend API with + createNodeInterface and setNodePreferredInterface (#47) + +Signed-off-by: BenjiReis +--- + drivers/linstor-manager | 50 ++++++++++++++++++++++++++++++++- + drivers/linstorvolumemanager.py | 16 +++++++++++ + 2 files changed, 65 insertions(+), 1 deletion(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 7726c1b7..2469b506 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -977,6 +977,51 @@ def health_check(session, args): + return format_result() + + ++def create_node_interface(session, args): ++ group_name = args['groupName'] ++ hostname = args['hostname'] ++ name = args['name'] ++ pif_uuid = args['pifUuid'] ++ ++ pif_ref = session.xenapi.PIF.get_by_uuid(pif_uuid) ++ pif = session.xenapi.PIF.get_record(pif_ref) ++ ++ if not pif['currently_attached']: ++ raise XenAPIPlugin.Failure('-1', ['PIF is not plugged']) ++ ++ ip_addr = pif['IP'] if pif['primary_address_type'].lower() == 'ipv4' else pif['IPv6'].split('/')[0] ++ if ip_addr == '': ++ raise XenAPIPlugin.Failure('-1', ['PIF has no IP']) ++ ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ try: ++ linstor.create_node_interface(hostname, name, ip_addr) ++ except Exception as e: ++ raise XenAPIPlugin.Failure('-1', [str(e)]) ++ return str(True) ++ ++ ++def set_node_preferred_interface(session, args): ++ group_name = args['groupName'] ++ hostname = args['hostname'] ++ name = args['name'] ++ ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ try: ++ linstor.set_node_preferred_interface(hostname, name) ++ except Exception as e: ++ raise XenAPIPlugin.Failure('-1', [str(e)]) ++ return str(True) ++ ++ + if __name__ == '__main__': + XenAPIPlugin.dispatch({ + 'prepareSr': prepare_sr, +@@ -1024,5 +1069,8 @@ if __name__ == '__main__': + 'destroyDrbdVolume': destroy_drbd_volume, + 'destroyDrbdVolumes': destroy_drbd_volumes, + 'getDrbdOpeners': get_drbd_openers, +- 'healthCheck': health_check ++ 'healthCheck': health_check, ++ ++ 'createNodeInterface': create_node_interface, ++ 'setNodePreferredInterface': set_node_preferred_interface + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index d27c84e6..fef34d3b 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1529,6 +1529,22 @@ class LinstorVolumeManager(object): + 'Failed to destroy node `{}`: {}'.format(node_name, error_str) + ) + ++ def create_node_interface(self, hostname, name, ip_addr): ++ result = self._linstor.netinterface_create(hostname, name, ip_addr) ++ if not linstor.Linstor.all_api_responses_no_error(result): ++ raise LinstorVolumeManagerError( ++ 'Unable to create interface on `{}`: {}'.format(hostname, ', '.join( ++ [str(x) for x in result])) ++ ) ++ ++ def set_node_preferred_interface(self, hostname, name): ++ result = self._linstor.node_modify(hostname, property_dict={'PrefNic': name}) ++ if not linstor.Linstor.all_api_responses_no_error(result): ++ raise LinstorVolumeManagerError( ++ 'Unable to set preferred interface on `{}`: {}'.format(hostname, ', '.join( ++ [str(x) for x in result])) ++ ) ++ + def get_nodes_info(self): + """ + Get all nodes + statuses, used or not by the pool. diff --git a/SOURCES/0150-fix-LinstorSR-support-VDI.resize-on-thick-volumes.patch b/SOURCES/0150-fix-LinstorSR-support-VDI.resize-on-thick-volumes.patch new file mode 100644 index 0000000..e4f46fe --- /dev/null +++ b/SOURCES/0150-fix-LinstorSR-support-VDI.resize-on-thick-volumes.patch @@ -0,0 +1,28 @@ +From 36459279481994f97bb186e6522c117b9669120a Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 1 Dec 2023 11:55:12 +0100 +Subject: [PATCH 150/177] fix(LinstorSR): support VDI.resize on thick volumes + +--- + drivers/LinstorSR.py | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 1d1b51a8..a5243381 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1939,11 +1939,14 @@ class LinstorVDI(VDI.VDI): + + if self.vdi_type == vhdutil.VDI_TYPE_RAW: + old_volume_size = self.size ++ new_volume_size = LinstorVolumeManager.round_up_volume_size(size) + else: + old_volume_size = self.utilisation + if self.sr._provisioning == 'thin': + # VDI is currently deflated, so keep it deflated. + new_volume_size = old_volume_size ++ else: ++ new_volume_size = LinstorVhdUtil.compute_volume_size(size, self.vdi_type) + assert new_volume_size >= old_volume_size + + space_needed = new_volume_size - old_volume_size diff --git a/SOURCES/0151-fix-linstorvolumemanager-format-correctly-exception-.patch b/SOURCES/0151-fix-linstorvolumemanager-format-correctly-exception-.patch new file mode 100644 index 0000000..7e467ea --- /dev/null +++ b/SOURCES/0151-fix-linstorvolumemanager-format-correctly-exception-.patch @@ -0,0 +1,27 @@ +From 4119be3b2769b7c29013cb5f60e68238a439b474 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 4 Dec 2023 16:36:55 +0100 +Subject: [PATCH 151/177] fix(linstorvolumemanager): format correctly exception + during db mount + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index fef34d3b..a41314e5 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2764,9 +2764,9 @@ class LinstorVolumeManager(object): + # 3. Remove useless backup directory. + try: + os.rmdir(backup_path) +- except Exception: ++ except Exception as e: + raise LinstorVolumeManagerError( +- 'Failed to remove backup path {} of LINSTOR config {}' ++ 'Failed to remove backup path {} of LINSTOR config: {}' + .format(backup_path, e) + ) + except Exception as e: diff --git a/SOURCES/0152-fix-LinstorSR-ensure-we-can-skip-coalesces-if-device.patch b/SOURCES/0152-fix-LinstorSR-ensure-we-can-skip-coalesces-if-device.patch new file mode 100644 index 0000000..8646443 --- /dev/null +++ b/SOURCES/0152-fix-LinstorSR-ensure-we-can-skip-coalesces-if-device.patch @@ -0,0 +1,29 @@ +From 9f3c9d85430ad14214b25fc6d529c88e8c217985 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 4 Dec 2023 18:57:08 +0100 +Subject: [PATCH 152/177] fix(LinstorSR): ensure we can skip coalesces if + device path can't be fetched + +Signed-off-by: Ronan Abhamon +--- + drivers/cleanup.py | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/drivers/cleanup.py b/drivers/cleanup.py +index 8635f488..8c16f689 100755 +--- a/drivers/cleanup.py ++++ b/drivers/cleanup.py +@@ -1548,7 +1548,12 @@ class LinstorVDI(VDI): + def _activateChain(self): + vdi = self + while vdi: +- p = self.sr._linstor.get_device_path(vdi.uuid) ++ try: ++ p = self.sr._linstor.get_device_path(vdi.uuid) ++ except Exception as e: ++ # Use SMException to skip coalesce. ++ # Otherwise the GC is stopped... ++ raise util.SMException(str(e)) + vdi = vdi.parent + + def _setHidden(self, hidden=True): diff --git a/SOURCES/0153-feat-linstor-manager-add-methods-to-modify-destroy-l.patch b/SOURCES/0153-feat-linstor-manager-add-methods-to-modify-destroy-l.patch new file mode 100644 index 0000000..a5c1577 --- /dev/null +++ b/SOURCES/0153-feat-linstor-manager-add-methods-to-modify-destroy-l.patch @@ -0,0 +1,252 @@ +From d607e5c450a9cbe39ea5826c2a4ec7f63751f805 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 19 Dec 2023 14:49:33 +0100 +Subject: [PATCH 153/177] feat(linstor-manager): add methods to + modify/destroy/list net interfaces + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 76 ++++++++++++++++++++++--- + drivers/linstorvolumemanager.py | 98 ++++++++++++++++++++++++++++----- + 2 files changed, 152 insertions(+), 22 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 2469b506..6b45875d 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -265,6 +265,19 @@ def force_destroy_drbd_volume(minor): + if ret: + raise Exception('Failed to destroy volume: {}'.format(stderr)) + ++ ++def get_ip_addr_of_pif(session, pif_uuid): ++ pif_ref = session.xenapi.PIF.get_by_uuid(pif_uuid) ++ pif = session.xenapi.PIF.get_record(pif_ref) ++ ++ if not pif['currently_attached']: ++ raise XenAPIPlugin.Failure('-1', ['PIF is not plugged']) ++ ++ ip_addr = pif['IP'] if pif['primary_address_type'].lower() == 'ipv4' else pif['IPv6'].split('/')[0] ++ if ip_addr == '': ++ raise XenAPIPlugin.Failure('-1', ['PIF has no IP']) ++ return ip_addr ++ + # ------------------------------------------------------------------------------ + + +@@ -983,15 +996,24 @@ def create_node_interface(session, args): + name = args['name'] + pif_uuid = args['pifUuid'] + +- pif_ref = session.xenapi.PIF.get_by_uuid(pif_uuid) +- pif = session.xenapi.PIF.get_record(pif_ref) ++ ip_addr = get_ip_addr_of_pif(session, pif_uuid) + +- if not pif['currently_attached']: +- raise XenAPIPlugin.Failure('-1', ['PIF is not plugged']) ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ try: ++ linstor.create_node_interface(hostname, name, ip_addr) ++ except Exception as e: ++ raise XenAPIPlugin.Failure('-1', [str(e)]) ++ return str(True) + +- ip_addr = pif['IP'] if pif['primary_address_type'].lower() == 'ipv4' else pif['IPv6'].split('/')[0] +- if ip_addr == '': +- raise XenAPIPlugin.Failure('-1', ['PIF has no IP']) ++ ++def destroy_node_interface(session, args): ++ group_name = args['groupName'] ++ hostname = args['hostname'] ++ name = args['name'] + + linstor = LinstorVolumeManager( + get_controller_uri(), +@@ -999,12 +1021,47 @@ def create_node_interface(session, args): + logger=util.SMlog + ) + try: +- linstor.create_node_interface(hostname, name, ip_addr) ++ linstor.destroy_node_interface(hostname, name) + except Exception as e: + raise XenAPIPlugin.Failure('-1', [str(e)]) + return str(True) + + ++def modify_node_interface(session, args): ++ group_name = args['groupName'] ++ hostname = args['hostname'] ++ name = args['name'] ++ pif_uuid = args['pifUuid'] ++ ++ ip_addr = get_ip_addr_of_pif(session, pif_uuid) ++ ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ try: ++ linstor.modify_node_interface(hostname, name, ip_addr) ++ except Exception as e: ++ raise XenAPIPlugin.Failure('-1', [str(e)]) ++ return str(True) ++ ++ ++def list_node_interfaces(session, args): ++ group_name = args['groupName'] ++ hostname = args['hostname'] ++ ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ try: ++ return json.dumps(linstor.list_node_interfaces(hostname)) ++ except Exception as e: ++ raise XenAPIPlugin.Failure('-1', [str(e)]) ++ ++ + def set_node_preferred_interface(session, args): + group_name = args['groupName'] + hostname = args['hostname'] +@@ -1072,5 +1129,8 @@ if __name__ == '__main__': + 'healthCheck': health_check, + + 'createNodeInterface': create_node_interface, ++ 'destroyNodeInterface': destroy_node_interface, ++ 'modifyNodeInterface': modify_node_interface, ++ 'listNodeInterfaces': list_node_interfaces, + 'setNodePreferredInterface': set_node_preferred_interface + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index a41314e5..32d15334 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -872,7 +872,6 @@ class LinstorVolumeManager(object): + ) + return size * 1024 + +- + def set_auto_promote_timeout(self, volume_uuid, timeout): + """ + Define the blocking time of open calls when a DRBD +@@ -1120,7 +1119,6 @@ class LinstorVolumeManager(object): + """ + return get_all_volume_openers(self.get_volume_name(volume_uuid), '0') + +- + def get_volumes_with_name(self): + """ + Give a volume dictionnary that contains names actually owned. +@@ -1529,21 +1527,93 @@ class LinstorVolumeManager(object): + 'Failed to destroy node `{}`: {}'.format(node_name, error_str) + ) + +- def create_node_interface(self, hostname, name, ip_addr): +- result = self._linstor.netinterface_create(hostname, name, ip_addr) +- if not linstor.Linstor.all_api_responses_no_error(result): ++ def create_node_interface(self, node_name, name, ip): ++ """ ++ Create a new node interface in the LINSTOR database. ++ :param str node_name: Node name of the interface to use. ++ :param str name: Interface to create. ++ :param str ip: IP of the interface. ++ """ ++ result = self._linstor.netinterface_create(node_name, name, ip) ++ errors = self._filter_errors(result) ++ if errors: ++ error_str = self._get_error_str(errors) + raise LinstorVolumeManagerError( +- 'Unable to create interface on `{}`: {}'.format(hostname, ', '.join( +- [str(x) for x in result])) +- ) ++ 'Failed to create node interface on `{}`: {}'.format(node_name, error_str) ++ ) + +- def set_node_preferred_interface(self, hostname, name): +- result = self._linstor.node_modify(hostname, property_dict={'PrefNic': name}) +- if not linstor.Linstor.all_api_responses_no_error(result): ++ def destroy_node_interface(self, node_name, name): ++ """ ++ Destroy a node interface in the LINSTOR database. ++ :param str node_name: Node name of the interface to remove. ++ :param str name: Interface to remove. ++ """ ++ result = self._linstor.netinterface_delete(node_name, name) ++ errors = self._filter_errors(result) ++ if errors: ++ error_str = self._get_error_str(errors) + raise LinstorVolumeManagerError( +- 'Unable to set preferred interface on `{}`: {}'.format(hostname, ', '.join( +- [str(x) for x in result])) +- ) ++ 'Failed to destroy node interface on `{}`: {}'.format(node_name, error_str) ++ ) ++ ++ def modify_node_interface(self, node_name, name, ip): ++ """ ++ Modify a node interface in the LINSTOR database. Create it if necessary. ++ :param str node_name: Node name of the interface to use. ++ :param str name: Interface to modify or create. ++ :param str ip: IP of the interface. ++ """ ++ result = self._linstor.netinterface_create(node_name, name, ip) ++ errors = self._filter_errors(result) ++ if not errors: ++ return ++ ++ if self._check_errors(errors, [linstor.consts.FAIL_EXISTS_NET_IF]): ++ result = self._linstor.netinterface_modify(node_name, name, ip) ++ errors = self._filter_errors(result) ++ if not errors: ++ return ++ ++ error_str = self._get_error_str(errors) ++ raise LinstorVolumeManagerError( ++ 'Unable to modify interface on `{}`: {}'.format(node_name, error_str) ++ ) ++ ++ def list_node_interfaces(self, node_name): ++ """ ++ List all node interfaces. ++ :param str node_name: Node name to use to list interfaces. ++ :rtype: list ++ : ++ """ ++ result = self._linstor.net_interface_list(node_name) ++ if not result: ++ raise LinstorVolumeManagerError( ++ 'Unable to list interfaces on `{}`: no list received'.format(node_name) ++ ) ++ ++ interfaces = {} ++ for interface in result: ++ interface = interface._rest_data ++ interfaces[interface['name']] = { ++ 'address': interface['address'], ++ 'active': interface['is_active'] ++ } ++ return interfaces ++ ++ def set_node_preferred_interface(self, node_name, name): ++ """ ++ Set the preferred interface to use on a node. ++ :param str node_name: Node name of the interface. ++ :param str name: Preferred interface to use. ++ """ ++ result = self._linstor.node_modify(node_name, property_dict={'PrefNic': name}) ++ errors = self._filter_errors(result) ++ if errors: ++ error_str = self._get_error_str(errors) ++ raise LinstorVolumeManagerError( ++ 'Failed to set preferred node interface on `{}`: {}'.format(node_name, error_str) ++ ) + + def get_nodes_info(self): + """ diff --git a/SOURCES/0154-fix-LinstorSR-force-a-defined-volume-prefix-if-we-ca.patch b/SOURCES/0154-fix-LinstorSR-force-a-defined-volume-prefix-if-we-ca.patch new file mode 100644 index 0000000..2a5a415 --- /dev/null +++ b/SOURCES/0154-fix-LinstorSR-force-a-defined-volume-prefix-if-we-ca.patch @@ -0,0 +1,24 @@ +From 0fcc4fbd4814578062bf83d7e26f3975cc120cd1 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 12 Jan 2024 10:28:20 +0100 +Subject: [PATCH 154/177] fix(LinstorSR): force a defined volume prefix if we + can't import libs + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index a5243381..a73ae5a2 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -27,6 +27,8 @@ try: + + LINSTOR_AVAILABLE = True + except ImportError: ++ PERSISTENT_PREFIX = 'unknown' ++ + LINSTOR_AVAILABLE = False + + from lock import Lock diff --git a/SOURCES/0155-fix-LinstorSR-explicit-error-message-when-a-group-is.patch b/SOURCES/0155-fix-LinstorSR-explicit-error-message-when-a-group-is.patch new file mode 100644 index 0000000..9e9f9d3 --- /dev/null +++ b/SOURCES/0155-fix-LinstorSR-explicit-error-message-when-a-group-is.patch @@ -0,0 +1,26 @@ +From c83b1aeacc1db0ec6110accaa128eed05eb36294 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 18 Jan 2024 10:24:01 +0100 +Subject: [PATCH 155/177] fix(LinstorSR): explicit error message when a group + is not unique during SR creation + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index a73ae5a2..fed1de2b 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -569,7 +569,9 @@ class LinstorSR(SR.SR): + if group_name and group_name == self._group_name: + raise xs_errors.XenError( + 'LinstorSRCreate', +- opterr='group name must be unique' ++ opterr='group name must be unique, already used by PBD {}'.format( ++ xenapi.PBD.get_uuid(pbd) ++ ) + ) + + if srs: diff --git a/SOURCES/0156-fix-LinstorSR-make-sure-VDI.delete-doesn-t-throw-und.patch b/SOURCES/0156-fix-LinstorSR-make-sure-VDI.delete-doesn-t-throw-und.patch new file mode 100644 index 0000000..900be09 --- /dev/null +++ b/SOURCES/0156-fix-LinstorSR-make-sure-VDI.delete-doesn-t-throw-und.patch @@ -0,0 +1,67 @@ +From a428724082121c6a61de9636efdeb3b0f2aead49 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 18 Jan 2024 11:18:11 +0100 +Subject: [PATCH 156/177] fix(LinstorSR): make sure VDI.delete doesn't throw + under specific conditions + +If we can update the volume state in the KV-store, there is no reason to raise +in case of future failure for another reason. Why? Because the volume will be +deleted by the next GC execution. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 9 ++++++++- + drivers/linstorvolumemanager.py | 13 ++++++++++--- + 2 files changed, 18 insertions(+), 4 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index fed1de2b..1e09e9bb 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1760,7 +1760,14 @@ class LinstorVDI(VDI.VDI): + 'Failed to remove the volume (maybe is leaf coalescing) ' + 'for {} err: {}'.format(self.uuid, e) + ) +- raise xs_errors.XenError('VDIDelete', opterr=str(e)) ++ ++ try: ++ raise xs_errors.XenError('VDIDelete', opterr=str(e)) ++ except LinstorVolumeManagerError as e: ++ if e.code != LinstorVolumeManagerError.ERR_VOLUME_DESTROY: ++ raise xs_errors.XenError('VDIDelete', opterr=str(e)) ++ ++ return + + if self.uuid in self.sr.vdis: + del self.sr.vdis[self.uuid] +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 32d15334..97e53725 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -272,7 +272,8 @@ def demote_drbd_resource(node_name, resource_name): + class LinstorVolumeManagerError(Exception): + ERR_GENERIC = 0, + ERR_VOLUME_EXISTS = 1, +- ERR_VOLUME_NOT_EXISTS = 2 ++ ERR_VOLUME_NOT_EXISTS = 2, ++ ERR_VOLUME_DESTROY = 3 + + def __init__(self, message, code=ERR_GENERIC): + super(LinstorVolumeManagerError, self).__init__(message) +@@ -681,8 +682,14 @@ class LinstorVolumeManager(object): + volume_properties = self._get_volume_properties(volume_uuid) + volume_properties[self.PROP_NOT_EXISTS] = self.STATE_NOT_EXISTS + +- self._volumes.remove(volume_uuid) +- self._destroy_volume(volume_uuid) ++ try: ++ self._volumes.remove(volume_uuid) ++ self._destroy_volume(volume_uuid) ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ str(e), ++ LinstorVolumeManagerError.ERR_VOLUME_DESTROY ++ ) + + def lock_volume(self, volume_uuid, locked=True): + """ diff --git a/SOURCES/0157-fix-LinstorSR-add-drbd-in-the-blacklist-of-multipath.patch b/SOURCES/0157-fix-LinstorSR-add-drbd-in-the-blacklist-of-multipath.patch new file mode 100644 index 0000000..aff4b33 --- /dev/null +++ b/SOURCES/0157-fix-LinstorSR-add-drbd-in-the-blacklist-of-multipath.patch @@ -0,0 +1,31 @@ +From db9f69a83b5402daf90fd1d2c44d9e65e37708c5 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 19 Jan 2024 14:39:17 +0100 +Subject: [PATCH 157/177] fix(LinstorSR): add drbd in the blacklist of + multipath.conf + +If DRBD is installed for the first time, and if the multipathd +service is not restarted, volumes can be blocked by this daemon. + +Note: the drbd-utils RPM adds the same change in /etc/multipath/conf.d/drbd.conf, +but this package is generally installed using XOSTOR UI and we don't display a +reboot message to be performed like the host update view. So it's probably good +to always have this modification whether we have drbd-utils or not. + +Signed-off-by: Ronan Abhamon +--- + multipath/multipath.conf | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/multipath/multipath.conf b/multipath/multipath.conf +index ede40bad..935d3d67 100644 +--- a/multipath/multipath.conf ++++ b/multipath/multipath.conf +@@ -23,6 +23,7 @@ blacklist { + devnode "scini*" + devnode "^rbd[0-9]*" + devnode "^nbd[0-9]*" ++ devnode "^drbd[0-9]*" + } + # Leave this section in place even if empty + blacklist_exceptions { diff --git a/SOURCES/0158-fix-linstorvolumemanager-create-cloned-volumes-on-ho.patch b/SOURCES/0158-fix-linstorvolumemanager-create-cloned-volumes-on-ho.patch new file mode 100644 index 0000000..0f3c89e --- /dev/null +++ b/SOURCES/0158-fix-linstorvolumemanager-create-cloned-volumes-on-ho.patch @@ -0,0 +1,123 @@ +From 4dbb05941ce89601e4766452e87e75a8abfbc63c Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 22 Jan 2024 11:25:25 +0100 +Subject: [PATCH 158/177] fix(linstorvolumemanager): create cloned volumes on + host selected by LINSTOR + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 94 ++------------------------------- + 1 file changed, 3 insertions(+), 91 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 97e53725..5de86a1b 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1245,8 +1245,7 @@ class LinstorVolumeManager(object): + def shallow_clone_volume(self, volume_uuid, clone_uuid, persistent=True): + """ + Clone a volume. Do not copy the data, this method creates a new volume +- with the same size. It tries to create the volume on the same host +- than volume source. ++ with the same size. + :param str volume_uuid: The volume to clone. + :param str clone_uuid: The cloned volume. + :param bool persistent: If false the volume will be unavailable +@@ -1267,95 +1266,8 @@ class LinstorVolumeManager(object): + 'Invalid size of {} for volume `{}`'.format(size, volume_name) + ) + +- # 2. Find the node(s) with the maximum space. +- candidates = self._find_best_size_candidates() +- if not candidates: +- raise LinstorVolumeManagerError( +- 'Unable to shallow clone volume `{}`, no free space found.' +- ) +- +- # 3. Compute node names and search if we can try to clone +- # on the same nodes than volume. +- def find_best_nodes(): +- for candidate in candidates: +- for node_name in candidate.node_names: +- if node_name in ideal_node_names: +- return candidate.node_names +- +- node_names = find_best_nodes() +- if not node_names: +- node_names = candidates[0].node_names +- +- if len(node_names) < self._redundancy: +- raise LinstorVolumeManagerError( +- 'Unable to shallow clone volume `{}`, '.format(volume_uuid) + +- '{} are required to clone, found: {}'.format( +- self._redundancy, len(node_names) +- ) +- ) +- +- # 4. Compute resources to create. +- clone_volume_name = self.build_volume_name(util.gen_uuid()) +- diskless_node_names = self._get_node_names() +- resources = [] +- for node_name in node_names: +- diskless_node_names.remove(node_name) +- resources.append(linstor.ResourceData( +- node_name=node_name, +- rsc_name=clone_volume_name, +- storage_pool=self._group_name +- )) +- +- # 5. Create resources! +- def clean(): +- try: +- self._destroy_volume(clone_uuid, force=True) +- except Exception as e: +- self._logger( +- 'Unable to destroy volume {} after shallow clone fail: {}' +- .format(clone_uuid, e) +- ) +- +- def create(): +- # Note: placed outside try/except block because we create only definition first. +- # There is no reason to call `clean` before the real resource creation. +- volume_properties = self._create_volume_with_properties( +- clone_uuid, clone_volume_name, size, place_resources=False +- ) +- +- # After this point, `clean` can be called for any fail because the clone UUID +- # is really unique. No risk to remove existing data. +- try: +- result = self._linstor.resource_create(resources) +- error_str = self._get_error_str(result) +- if error_str: +- raise LinstorVolumeManagerError( +- 'Could not create cloned volume `{}` of `{}` from ' +- 'SR `{}`: {}'.format( +- clone_uuid, volume_uuid, self._group_name, +- error_str +- ) +- ) +- return volume_properties +- except Exception: +- clean() +- raise +- +- # Retry because we can get errors like this: +- # "Resource disappeared while waiting for it to be ready" or +- # "Resource did not became ready on node 'XXX' within reasonable time, check Satellite for errors." +- # in the LINSTOR server. +- volume_properties = util.retry(create, maxretry=5) +- +- try: +- device_path = self._find_device_path(clone_uuid, clone_volume_name) +- if persistent: +- volume_properties[self.PROP_NOT_EXISTS] = self.STATE_EXISTS +- self._volumes.add(clone_uuid) +- return device_path +- except Exception as e: +- clean() +- raise ++ # 2. Create clone! ++ return self.create_volume(clone_uuid, size, persistent) + + def remove_resourceless_volumes(self): + """ diff --git a/SOURCES/0159-fix-linstorvolumemanager-don-t-align-volumes-on-LVM-.patch b/SOURCES/0159-fix-linstorvolumemanager-don-t-align-volumes-on-LVM-.patch new file mode 100644 index 0000000..005dbad --- /dev/null +++ b/SOURCES/0159-fix-linstorvolumemanager-don-t-align-volumes-on-LVM-.patch @@ -0,0 +1,29 @@ +From 0e86f2c9c1acdf9a669807d0db783cb58ac29499 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 26 Jan 2024 15:29:21 +0100 +Subject: [PATCH 159/177] fix(linstorvolumemanager): don't align volumes on LVM + sector size + +It's the goal of the LINSTOR stack to align properly on the LVM layer. +Otherwise without this patch we get an increase in snapshots and clones with a size of 4MiB. + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 5de86a1b..7410fd7a 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -309,8 +309,8 @@ class LinstorVolumeManager(object): + + DEV_ROOT_PATH = DRBD_BY_RES_PATH + +- # Default LVM extent size. +- BLOCK_SIZE = 4 * 1024 * 1024 ++ # Default sector size. ++ BLOCK_SIZE = 512 + + # List of volume properties. + PROP_METADATA = 'metadata' diff --git a/SOURCES/0160-fix-linstorvolumemanager-assert-with-message-after-l.patch b/SOURCES/0160-fix-linstorvolumemanager-assert-with-message-after-l.patch new file mode 100644 index 0000000..c18ec16 --- /dev/null +++ b/SOURCES/0160-fix-linstorvolumemanager-assert-with-message-after-l.patch @@ -0,0 +1,30 @@ +From 67926a299d10072816286df087cb1fe3a0111717 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 5 Feb 2024 18:01:22 +0100 +Subject: [PATCH 160/177] fix(linstorvolumemanager): assert with message after + log in update_volume_uuid + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 7410fd7a..5f2e06ee 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -972,12 +972,12 @@ class LinstorVolumeManager(object): + deleted VDI. + """ + +- assert volume_uuid != new_volume_uuid +- + self._logger( + 'Trying to update volume UUID {} to {}...' + .format(volume_uuid, new_volume_uuid) + ) ++ assert volume_uuid != new_volume_uuid, 'can\'t update volume UUID, same value' ++ + if not force: + self._ensure_volume_exists(volume_uuid) + self.ensure_volume_is_not_locked(volume_uuid) diff --git a/SOURCES/0161-fix-linstorvolumemanager-retry-resize-if-volume-is-n.patch b/SOURCES/0161-fix-linstorvolumemanager-retry-resize-if-volume-is-n.patch new file mode 100644 index 0000000..ac937cb --- /dev/null +++ b/SOURCES/0161-fix-linstorvolumemanager-retry-resize-if-volume-is-n.patch @@ -0,0 +1,55 @@ +From 0fc311b8ac8da92ba91f99a1ceb97ee7663c88ed Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 5 Feb 2024 23:13:06 +0100 +Subject: [PATCH 161/177] fix(linstorvolumemanager): retry resize if volume is + not up to date + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 30 +++++++++++++++++++++--------- + 1 file changed, 21 insertions(+), 9 deletions(-) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 5f2e06ee..79ac84a4 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -824,18 +824,30 @@ class LinstorVolumeManager(object): + + volume_name = self.get_volume_name(volume_uuid) + self.ensure_volume_is_not_locked(volume_uuid) +- new_size = self.round_up_volume_size(new_size) ++ new_size = self.round_up_volume_size(new_size) / 1024 + +- result = self._linstor.volume_dfn_modify( +- rsc_name=volume_name, +- volume_nr=0, +- size=new_size / 1024 +- ) ++ retry_count = 30 ++ while True: ++ result = self._linstor.volume_dfn_modify( ++ rsc_name=volume_name, ++ volume_nr=0, ++ size=new_size ++ ) + +- self._mark_resource_cache_as_dirty() ++ self._mark_resource_cache_as_dirty() ++ ++ error_str = self._get_error_str(result) ++ if not error_str: ++ break ++ ++ # After volume creation, DRBD volume can be unusable during many seconds. ++ # So we must retry the definition change if the device is not up to date. ++ # Often the case for thick provisioning. ++ if retry_count and error_str.find('non-UpToDate DRBD device') >= 0: ++ time.sleep(2) ++ retry_count -= 1 ++ continue + +- error_str = self._get_error_str(result) +- if error_str: + raise LinstorVolumeManagerError( + 'Could not resize volume `{}` from SR `{}`: {}' + .format(volume_uuid, self._group_name, error_str) diff --git a/SOURCES/0162-fix-LinstorSR-create-DRBD-diskless-if-necessary-for-.patch b/SOURCES/0162-fix-LinstorSR-create-DRBD-diskless-if-necessary-for-.patch new file mode 100644 index 0000000..3c55748 --- /dev/null +++ b/SOURCES/0162-fix-LinstorSR-create-DRBD-diskless-if-necessary-for-.patch @@ -0,0 +1,98 @@ +From 0d61202439fd59c9d0121d71ce63e41e5a7d7f68 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 6 Feb 2024 00:10:32 +0100 +Subject: [PATCH 162/177] fix(LinstorSR): create DRBD diskless if necessary for + each VHD parent + +It's necessary to have all parents during snapshot to create a new VHD child. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 60 ++++++++++++++++++++++++-------------------- + 1 file changed, 33 insertions(+), 27 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 1e09e9bb..53afbefe 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1827,32 +1827,7 @@ class LinstorVDI(VDI.VDI): + return self._attach_using_http_nbd() + + # Ensure we have a path... +- while vdi_uuid: +- path = self._linstor.get_device_path(vdi_uuid) +- if not util.pathexists(path): +- raise xs_errors.XenError( +- 'VDIUnavailable', opterr='Could not find: {}'.format(path) +- ) +- +- # Diskless path can be created on the fly, ensure we can open it. +- def check_volume_usable(): +- while True: +- try: +- with open(path, 'r+'): +- pass +- except IOError as e: +- if e.errno == errno.ENODATA: +- time.sleep(2) +- continue +- if e.errno == errno.EROFS: +- util.SMlog('Volume not attachable because RO. Openers: {}'.format( +- self.sr._linstor.get_volume_openers(vdi_uuid) +- )) +- raise +- break +- util.retry(check_volume_usable, 15, 2) +- +- vdi_uuid = self.sr._vhdutil.get_vhd_info(vdi_uuid).parentUuid ++ self._create_chain_paths(self.uuid) + + self.attached = True + return VDI.VDI.attach(self, self.sr.uuid, self.uuid) +@@ -2379,7 +2354,7 @@ class LinstorVDI(VDI.VDI): + raise xs_errors.XenError('SnapshotChainTooLong') + + # Ensure we have a valid path if we don't have a local diskful. +- self.sr._linstor.get_device_path(self.uuid) ++ self._create_chain_paths(self.uuid) + + volume_path = self.path + if not util.pathexists(volume_path): +@@ -2842,6 +2817,37 @@ class LinstorVDI(VDI.VDI): + self._kill_persistent_nbd_server(volume_name) + self._kill_persistent_http_server(volume_name) + ++ def _create_chain_paths(self, vdi_uuid): ++ # OPTIMIZE: Add a limit_to_first_allocated_block param to limit vhdutil calls. ++ # Useful for the snapshot code algorithm. ++ ++ while vdi_uuid: ++ path = self._linstor.get_device_path(vdi_uuid) ++ if not util.pathexists(path): ++ raise xs_errors.XenError( ++ 'VDIUnavailable', opterr='Could not find: {}'.format(path) ++ ) ++ ++ # Diskless path can be created on the fly, ensure we can open it. ++ def check_volume_usable(): ++ while True: ++ try: ++ with open(path, 'r+'): ++ pass ++ except IOError as e: ++ if e.errno == errno.ENODATA: ++ time.sleep(2) ++ continue ++ if e.errno == errno.EROFS: ++ util.SMlog('Volume not attachable because RO. Openers: {}'.format( ++ self.sr._linstor.get_volume_openers(vdi_uuid) ++ )) ++ raise ++ break ++ util.retry(check_volume_usable, 15, 2) ++ ++ vdi_uuid = self.sr._vhdutil.get_vhd_info(vdi_uuid).parentUuid ++ + # ------------------------------------------------------------------------------ + + diff --git a/SOURCES/0163-fix-LinstorSR-fix-bad-call-to-vhdutil.inflate-bad-ex.patch b/SOURCES/0163-fix-LinstorSR-fix-bad-call-to-vhdutil.inflate-bad-ex.patch new file mode 100644 index 0000000..106c6b7 --- /dev/null +++ b/SOURCES/0163-fix-LinstorSR-fix-bad-call-to-vhdutil.inflate-bad-ex.patch @@ -0,0 +1,33 @@ +From 574d72fc61a205b0a037cfe6a0002f2602f8e99d Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 6 Feb 2024 00:14:11 +0100 +Subject: [PATCH 163/177] fix(LinstorSR): fix bad call to vhdutil.inflate + bad + exception + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 53afbefe..de10f423 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1944,7 +1944,7 @@ class LinstorVDI(VDI.VDI): + else: + if new_volume_size != old_volume_size: + self.sr._vhdutil.inflate( +- self.sr._journaler, self._linstor, self.uuid, self.path, ++ self.sr._journaler, self.uuid, self.path, + new_volume_size, old_volume_size + ) + self.sr._vhdutil.set_size_virt_fast(self.path, size) +@@ -2497,7 +2497,7 @@ class LinstorVDI(VDI.VDI): + self.session.xenapi.VDI.set_sm_config( + vdi_ref, active_vdi.sm_config + ) +- except Exception as e: ++ except Exception: + util.logException('Failed to snapshot!') + try: + self.sr._handle_interrupted_clone( diff --git a/SOURCES/0164-fix-LinstorSR-activate-VG-if-attach-from-config-is-a.patch b/SOURCES/0164-fix-LinstorSR-activate-VG-if-attach-from-config-is-a.patch new file mode 100644 index 0000000..ffdd7b1 --- /dev/null +++ b/SOURCES/0164-fix-LinstorSR-activate-VG-if-attach-from-config-is-a.patch @@ -0,0 +1,24 @@ +From fdd9b4d9038c31c1c54d31b5ea135f6cab85ac24 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 12 Feb 2024 20:54:06 +0100 +Subject: [PATCH 164/177] fix(LinstorSR): activate VG if attach from config is + asked + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index de10f423..b6b15298 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -382,7 +382,7 @@ class LinstorSR(SR.SR): + + def load(self, *args, **kwargs): + # Activate all LVMs to make drbd-reactor happy. +- if self.srcmd.cmd == 'sr_attach': ++ if self.srcmd.cmd in ('sr_attach', 'vdi_attach_from_config'): + activate_lvm_group(self._group_name) + + if not self._has_session: diff --git a/SOURCES/0165-feat-LinstorSR-use-a-specific-resource-group-for-DB-.patch b/SOURCES/0165-feat-LinstorSR-use-a-specific-resource-group-for-DB-.patch new file mode 100644 index 0000000..6ad76cb --- /dev/null +++ b/SOURCES/0165-feat-LinstorSR-use-a-specific-resource-group-for-DB-.patch @@ -0,0 +1,518 @@ +From 51699827c68a7674b379b44bcc9fa51c95c9b9a2 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Tue, 19 Mar 2024 23:09:54 +0100 +Subject: [PATCH 165/177] feat(LinstorSR): use a specific resource group for DB + and HA + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 7 +- + drivers/linstorvolumemanager.py | 253 ++++++++++++++++++++++---------- + 2 files changed, 182 insertions(+), 78 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index b6b15298..3421f79f 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1662,8 +1662,11 @@ class LinstorVDI(VDI.VDI): + volume_name = REDO_LOG_VOLUME_NAME + + self._linstor.create_volume( +- self.uuid, volume_size, persistent=False, +- volume_name=volume_name ++ self.uuid, ++ volume_size, ++ persistent=False, ++ volume_name=volume_name, ++ high_availability=volume_name is not None + ) + volume_info = self._linstor.get_volume_info(self.uuid) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 79ac84a4..4118a28f 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -273,7 +273,8 @@ class LinstorVolumeManagerError(Exception): + ERR_GENERIC = 0, + ERR_VOLUME_EXISTS = 1, + ERR_VOLUME_NOT_EXISTS = 2, +- ERR_VOLUME_DESTROY = 3 ++ ERR_VOLUME_DESTROY = 3, ++ ERR_GROUP_NOT_EXISTS = 4 + + def __init__(self, message, code=ERR_GENERIC): + super(LinstorVolumeManagerError, self).__init__(message) +@@ -298,11 +299,9 @@ class LinstorVolumeManager(object): + """ + + __slots__ = ( +- '_linstor', '_logger', +- '_uri', '_base_group_name', +- '_redundancy', '_group_name', +- '_volumes', '_storage_pools', +- '_storage_pools_time', ++ '_linstor', '_logger', '_redundancy', ++ '_base_group_name', '_group_name', '_ha_group_name', ++ '_volumes', '_storage_pools', '_storage_pools_time', + '_kv_cache', '_resource_cache', '_volume_info_cache', + '_kv_cache_dirty', '_resource_cache_dirty', '_volume_info_cache_dirty' + ) +@@ -348,6 +347,7 @@ class LinstorVolumeManager(object): + # A LINSTOR (resource, group, ...) name cannot start with a number. + # So we add a prefix behind our SR/VOLUME uuids. + PREFIX_SR = 'xcp-sr-' ++ PREFIX_HA = 'xcp-ha-' + PREFIX_VOLUME = 'xcp-volume-' + + # Limit request number when storage pool info is asked, we fetch +@@ -406,8 +406,7 @@ class LinstorVolumeManager(object): + + # Ensure group exists. + group_name = self._build_group_name(group_name) +- groups = self._linstor.resource_group_list_raise([group_name]) +- groups = groups.resource_groups ++ groups = self._linstor.resource_group_list_raise([group_name]).resource_groups + if not groups: + raise LinstorVolumeManagerError( + 'Unable to find `{}` Linstor SR'.format(group_name) +@@ -417,6 +416,7 @@ class LinstorVolumeManager(object): + self._logger = logger + self._redundancy = groups[0].select_filter.place_count + self._group_name = group_name ++ self._ha_group_name = self._build_ha_group_name(self._base_group_name) + self._volumes = set() + self._storage_pools_time = 0 + +@@ -617,7 +617,12 @@ class LinstorVolumeManager(object): + return volume_uuid in self._volumes + + def create_volume( +- self, volume_uuid, size, persistent=True, volume_name=None ++ self, ++ volume_uuid, ++ size, ++ persistent=True, ++ volume_name=None, ++ high_availability=False + ): + """ + Create a new volume on the SR. +@@ -627,6 +632,8 @@ class LinstorVolumeManager(object): + on the next constructor call LinstorSR(...). + :param str volume_name: If set, this name is used in the LINSTOR + database instead of a generated name. ++ :param bool high_availability: If set, the volume is created in ++ the HA group. + :return: The current device path of the volume. + :rtype: str + """ +@@ -635,7 +642,11 @@ class LinstorVolumeManager(object): + if not volume_name: + volume_name = self.build_volume_name(util.gen_uuid()) + volume_properties = self._create_volume_with_properties( +- volume_uuid, volume_name, size, place_resources=True ++ volume_uuid, ++ volume_name, ++ size, ++ True, # place_resources ++ high_availability + ) + + # Volume created! Now try to find the device path. +@@ -651,7 +662,7 @@ class LinstorVolumeManager(object): + 'LINSTOR volume {} created!'.format(volume_uuid) + ) + return device_path +- except Exception as e: ++ except Exception: + # There is an issue to find the path. + # At this point the volume has just been created, so force flag can be used. + self._destroy_volume(volume_uuid, force=True) +@@ -1359,6 +1370,7 @@ class LinstorVolumeManager(object): + + # 4.5. Destroy group and storage pools. + self._destroy_resource_group(self._linstor, self._group_name) ++ self._destroy_resource_group(self._linstor, self._ha_group_name) + for pool in self._get_storage_pools(force=True): + self._destroy_storage_pool( + self._linstor, pool.name, pool.node_name +@@ -1659,6 +1671,16 @@ class LinstorVolumeManager(object): + """ + return self._request_database_path(self._linstor) + ++ @classmethod ++ def get_all_group_names(cls, base_name): ++ """ ++ Get all group names. I.e. list of current group + HA. ++ :param str base_name: The SR group_name to use. ++ :return: List of group names. ++ :rtype: list ++ """ ++ return [cls._build_group_name(base_name), cls._build_ha_group_name(base_name)] ++ + @classmethod + def create_sr( + cls, group_name, ips, redundancy, +@@ -1744,8 +1766,8 @@ class LinstorVolumeManager(object): + driver_pool_name = group_name + base_group_name = group_name + group_name = cls._build_group_name(group_name) +- pools = lin.storage_pool_list_raise(filter_by_stor_pools=[group_name]) +- pools = pools.storage_pools ++ storage_pool_name = group_name ++ pools = lin.storage_pool_list_raise(filter_by_stor_pools=[storage_pool_name]).storage_pools + if pools: + existing_node_names = map(lambda pool: pool.node_name, pools) + raise LinstorVolumeManagerError( +@@ -1754,7 +1776,7 @@ class LinstorVolumeManager(object): + ) + + if lin.resource_group_list_raise( +- [group_name] ++ cls.get_all_group_names(base_group_name) + ).resource_groups: + if not lin.resource_dfn_list_raise().resource_definitions: + backup_path = cls._create_database_backup_path() +@@ -1791,7 +1813,7 @@ class LinstorVolumeManager(object): + + result = lin.storage_pool_create( + node_name=node_name, +- storage_pool_name=group_name, ++ storage_pool_name=storage_pool_name, + storage_driver='LVM_THIN' if thin_provisioning else 'LVM', + driver_pool_name=driver_pool_name + ) +@@ -1807,7 +1829,7 @@ class LinstorVolumeManager(object): + 'Volume group `{}` not found on `{}`. Ignoring...' + .format(group_name, node_name) + ) +- cls._destroy_storage_pool(lin, group_name, node_name) ++ cls._destroy_storage_pool(lin, storage_pool_name, node_name) + else: + error_str = cls._get_error_str(result) + raise LinstorVolumeManagerError( +@@ -1825,49 +1847,28 @@ class LinstorVolumeManager(object): + ) + ) + +- # 2.b. Create resource group. +- rg_creation_attempt = 0 +- while True: +- result = lin.resource_group_create( +- name=group_name, +- place_count=redundancy, +- storage_pool=group_name, +- diskless_on_remaining=False +- ) +- error_str = cls._get_error_str(result) +- if not error_str: +- break +- +- errors = cls._filter_errors(result) +- if cls._check_errors(errors, [linstor.consts.FAIL_EXISTS_RSC_GRP]): +- rg_creation_attempt += 1 +- if rg_creation_attempt < 2: +- try: +- cls._destroy_resource_group(lin, group_name) +- except Exception as e: +- error_str = 'Failed to destroy old and empty RG: {}'.format(e) +- else: +- continue +- +- raise LinstorVolumeManagerError( +- 'Could not create RG `{}`: {}'.format(group_name, error_str) +- ) +- +- # 2.c. Create volume group. +- result = lin.volume_group_create(group_name) +- error_str = cls._get_error_str(result) +- if error_str: +- raise LinstorVolumeManagerError( +- 'Could not create VG `{}`: {}'.format( +- group_name, error_str +- ) +- ) ++ # 2.b. Create resource groups. ++ ha_group_name = cls._build_ha_group_name(base_group_name) ++ cls._create_resource_group( ++ lin, ++ group_name, ++ storage_pool_name, ++ redundancy, ++ True ++ ) ++ cls._create_resource_group( ++ lin, ++ ha_group_name, ++ storage_pool_name, ++ 3, ++ True ++ ) + + # 3. Create the LINSTOR database volume and mount it. + try: + logger('Creating database volume...') + volume_path = cls._create_database_volume( +- lin, group_name, node_names, redundancy, auto_quorum ++ lin, ha_group_name, storage_pool_name, node_names, redundancy, auto_quorum + ) + except LinstorVolumeManagerError as e: + if e.code != LinstorVolumeManagerError.ERR_VOLUME_EXISTS: +@@ -1907,6 +1908,7 @@ class LinstorVolumeManager(object): + logger('Destroying resource group and storage pools after fail...') + try: + cls._destroy_resource_group(lin, group_name) ++ cls._destroy_resource_group(lin, ha_group_name) + except Exception as e2: + logger('Failed to destroy resource group: {}'.format(e2)) + pass +@@ -1914,7 +1916,7 @@ class LinstorVolumeManager(object): + i = min(i, len(node_names) - 1) + while j <= i: + try: +- cls._destroy_storage_pool(lin, group_name, node_names[j]) ++ cls._destroy_storage_pool(lin, storage_pool_name, node_names[j]) + except Exception as e2: + logger('Failed to destroy resource group: {}'.format(e2)) + pass +@@ -1952,7 +1954,7 @@ class LinstorVolumeManager(object): + def build_volume_name(cls, base_name): + """ + Build a volume name given a base name (i.e. a UUID). +- :param str volume_name: The volume name to use. ++ :param str base_name: The volume name to use. + :return: A valid or not device path. + :rtype: str + """ +@@ -2031,7 +2033,7 @@ class LinstorVolumeManager(object): + resource_names = set() + dfns = self._linstor.resource_dfn_list_raise().resource_definitions + for dfn in dfns: +- if dfn.resource_group_name == self._group_name and ( ++ if dfn.resource_group_name in self.get_all_group_names(self._base_group_name) and ( + ignore_deleted or + linstor.consts.FLAG_DELETE not in dfn.flags + ): +@@ -2149,27 +2151,54 @@ class LinstorVolumeManager(object): + return self._storage_pools + + def _create_volume( +- self, volume_uuid, volume_name, size, place_resources ++ self, ++ volume_uuid, ++ volume_name, ++ size, ++ place_resources, ++ high_availability + ): + size = self.round_up_volume_size(size) + self._mark_resource_cache_as_dirty() + ++ group_name = self._ha_group_name if high_availability else self._group_name + def create_definition(): +- self._check_volume_creation_errors( +- self._linstor.resource_group_spawn( +- rsc_grp_name=self._group_name, +- rsc_dfn_name=volume_name, +- vlm_sizes=['{}B'.format(size)], +- definitions_only=True +- ), +- volume_uuid, +- self._group_name +- ) ++ first_attempt = True ++ while True: ++ try: ++ self._check_volume_creation_errors( ++ self._linstor.resource_group_spawn( ++ rsc_grp_name=group_name, ++ rsc_dfn_name=volume_name, ++ vlm_sizes=['{}B'.format(size)], ++ definitions_only=True ++ ), ++ volume_uuid, ++ self._group_name ++ ) ++ break ++ except LinstorVolumeManagerError as e: ++ if ( ++ not first_attempt or ++ not high_availability or ++ e.code != LinstorVolumeManagerError.ERR_GROUP_NOT_EXISTS ++ ): ++ raise ++ ++ first_attempt = False ++ self._create_resource_group( ++ self._linstor, ++ group_name, ++ self._group_name, ++ 3, ++ True ++ ) ++ + self._configure_volume_peer_slots(self._linstor, volume_name) + + def clean(): + try: +- self._destroy_volume(volume_uuid, force=True) ++ self._destroy_volume(volume_uuid, force=True, preserve_properties=True) + except Exception as e: + self._logger( + 'Unable to destroy volume {} after creation fail: {}' +@@ -2201,7 +2230,12 @@ class LinstorVolumeManager(object): + util.retry(create, maxretry=5) + + def _create_volume_with_properties( +- self, volume_uuid, volume_name, size, place_resources ++ self, ++ volume_uuid, ++ volume_name, ++ size, ++ place_resources, ++ high_availability + ): + if self.check_volume_exists(volume_uuid): + raise LinstorVolumeManagerError( +@@ -2230,7 +2264,11 @@ class LinstorVolumeManager(object): + volume_properties[self.PROP_VOLUME_NAME] = volume_name + + self._create_volume( +- volume_uuid, volume_name, size, place_resources ++ volume_uuid, ++ volume_name, ++ size, ++ place_resources, ++ high_availability + ) + + assert volume_properties.namespace == \ +@@ -2331,7 +2369,7 @@ class LinstorVolumeManager(object): + break + self._destroy_resource(resource_name) + +- def _destroy_volume(self, volume_uuid, force=False): ++ def _destroy_volume(self, volume_uuid, force=False, preserve_properties=False): + volume_properties = self._get_volume_properties(volume_uuid) + try: + volume_name = volume_properties.get(self.PROP_VOLUME_NAME) +@@ -2339,7 +2377,8 @@ class LinstorVolumeManager(object): + self._destroy_resource(volume_name, force) + + # Assume this call is atomic. +- volume_properties.clear() ++ if not preserve_properties: ++ volume_properties.clear() + except Exception as e: + raise LinstorVolumeManagerError( + 'Cannot destroy volume `{}`: {}'.format(volume_uuid, e) +@@ -2599,7 +2638,7 @@ class LinstorVolumeManager(object): + + @classmethod + def _create_database_volume( +- cls, lin, group_name, node_names, redundancy, auto_quorum ++ cls, lin, group_name, storage_pool_name, node_names, redundancy, auto_quorum + ): + try: + dfns = lin.resource_dfn_list_raise().resource_definitions +@@ -2621,7 +2660,7 @@ class LinstorVolumeManager(object): + # I don't understand why but this command protect against this bug. + try: + pools = lin.storage_pool_list_raise( +- filter_by_stor_pools=[group_name] ++ filter_by_stor_pools=[storage_pool_name] + ) + except Exception as e: + raise LinstorVolumeManagerError( +@@ -2663,7 +2702,7 @@ class LinstorVolumeManager(object): + resources.append(linstor.ResourceData( + node_name=node_name, + rsc_name=DATABASE_VOLUME_NAME, +- storage_pool=group_name ++ storage_pool=storage_pool_name + )) + # Create diskless resources on the remaining set. + for node_name in diskful_nodes[redundancy:] + diskless_nodes: +@@ -2825,6 +2864,55 @@ class LinstorVolumeManager(object): + # after LINSTOR database volume destruction. + return util.retry(destroy, maxretry=10) + ++ @classmethod ++ def _create_resource_group( ++ cls, ++ lin, ++ group_name, ++ storage_pool_name, ++ redundancy, ++ destroy_old_group ++ ): ++ rg_creation_attempt = 0 ++ while True: ++ result = lin.resource_group_create( ++ name=group_name, ++ place_count=redundancy, ++ storage_pool=storage_pool_name, ++ diskless_on_remaining=False ++ ) ++ error_str = cls._get_error_str(result) ++ if not error_str: ++ break ++ ++ errors = cls._filter_errors(result) ++ if destroy_old_group and cls._check_errors(errors, [ ++ linstor.consts.FAIL_EXISTS_RSC_GRP ++ ]): ++ rg_creation_attempt += 1 ++ if rg_creation_attempt < 2: ++ try: ++ cls._destroy_resource_group(lin, group_name) ++ except Exception as e: ++ error_str = 'Failed to destroy old and empty RG: {}'.format(e) ++ else: ++ continue ++ ++ raise LinstorVolumeManagerError( ++ 'Could not create RG `{}`: {}'.format( ++ group_name, error_str ++ ) ++ ) ++ ++ result = lin.volume_group_create(group_name) ++ error_str = cls._get_error_str(result) ++ if error_str: ++ raise LinstorVolumeManagerError( ++ 'Could not create VG `{}`: {}'.format( ++ group_name, error_str ++ ) ++ ) ++ + @classmethod + def _destroy_resource_group(cls, lin, group_name): + def destroy(): +@@ -2849,6 +2937,12 @@ class LinstorVolumeManager(object): + # `VG/LV`. "/" is not accepted by LINSTOR. + return '{}{}'.format(cls.PREFIX_SR, base_name.replace('/', '_')) + ++ # Used to store important data in a HA context, ++ # i.e. a replication count of 3. ++ @classmethod ++ def _build_ha_group_name(cls, base_name): ++ return '{}{}'.format(cls.PREFIX_HA, base_name.replace('/', '_')) ++ + @classmethod + def _check_volume_creation_errors(cls, result, volume_uuid, group_name): + errors = cls._filter_errors(result) +@@ -2861,6 +2955,13 @@ class LinstorVolumeManager(object): + LinstorVolumeManagerError.ERR_VOLUME_EXISTS + ) + ++ if cls._check_errors(errors, [linstor.consts.FAIL_NOT_FOUND_RSC_GRP]): ++ raise LinstorVolumeManagerError( ++ 'Failed to create volume `{}` from SR `{}`, resource group doesn\'t exist' ++ .format(volume_uuid, group_name), ++ LinstorVolumeManagerError.ERR_GROUP_NOT_EXISTS ++ ) ++ + if errors: + raise LinstorVolumeManagerError( + 'Failed to create volume `{}` from SR `{}`: {}'.format( diff --git a/SOURCES/0166-feat-linstor-manager-add-getNodePreferredInterface-h.patch b/SOURCES/0166-feat-linstor-manager-add-getNodePreferredInterface-h.patch new file mode 100644 index 0000000..303d4da --- /dev/null +++ b/SOURCES/0166-feat-linstor-manager-add-getNodePreferredInterface-h.patch @@ -0,0 +1,73 @@ +From d3d377079f5153fa41220b4df9de176b733cd6a8 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 15 Apr 2024 11:22:18 +0200 +Subject: [PATCH 166/177] feat(linstor-manager): add + `getNodePreferredInterface` helper + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 16 ++++++++++++++++ + drivers/linstorvolumemanager.py | 17 +++++++++++++++++ + 2 files changed, 33 insertions(+) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 6b45875d..a2501d74 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -1062,6 +1062,21 @@ def list_node_interfaces(session, args): + raise XenAPIPlugin.Failure('-1', [str(e)]) + + ++def get_node_preferred_interface(session, args): ++ group_name = args['groupName'] ++ hostname = args['hostname'] ++ ++ linstor = LinstorVolumeManager( ++ get_controller_uri(), ++ group_name, ++ logger=util.SMlog ++ ) ++ try: ++ return linstor.get_node_preferred_interface(hostname) ++ except Exception as e: ++ raise XenAPIPlugin.Failure('-1', [str(e)]) ++ ++ + def set_node_preferred_interface(session, args): + group_name = args['groupName'] + hostname = args['hostname'] +@@ -1132,5 +1147,6 @@ if __name__ == '__main__': + 'destroyNodeInterface': destroy_node_interface, + 'modifyNodeInterface': modify_node_interface, + 'listNodeInterfaces': list_node_interfaces, ++ 'getNodePreferredInterface': get_node_preferred_interface, + 'setNodePreferredInterface': set_node_preferred_interface + }) +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 4118a28f..46c3283d 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1544,6 +1544,23 @@ class LinstorVolumeManager(object): + } + return interfaces + ++ def get_node_preferred_interface(self, node_name): ++ """ ++ Get the preferred interface used by a node. ++ :param str node_name: Node name of the interface to get. ++ :rtype: str ++ """ ++ try: ++ nodes = self._linstor.node_list_raise([node_name]).nodes ++ if nodes: ++ properties = nodes[0].props ++ return properties.get('PrefNic', 'default') ++ return nodes ++ except Exception as e: ++ raise LinstorVolumeManagerError( ++ 'Failed to get preferred interface: `{}`'.format(e) ++ ) ++ + def set_node_preferred_interface(self, node_name, name): + """ + Set the preferred interface to use on a node. diff --git a/SOURCES/0167-fix-linstorvolumemanager-blocks-deletion-of-default-.patch b/SOURCES/0167-fix-linstorvolumemanager-blocks-deletion-of-default-.patch new file mode 100644 index 0000000..92cdc3b --- /dev/null +++ b/SOURCES/0167-fix-linstorvolumemanager-blocks-deletion-of-default-.patch @@ -0,0 +1,28 @@ +From 297296c0ec9c5df659267e1432e4de0ac6af5618 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 15 Apr 2024 11:26:00 +0200 +Subject: [PATCH 167/177] fix(linstorvolumemanager): blocks deletion of default + network interface + +Signed-off-by: Ronan Abhamon +--- + drivers/linstorvolumemanager.py | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 46c3283d..02059d81 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1491,6 +1491,12 @@ class LinstorVolumeManager(object): + :param str node_name: Node name of the interface to remove. + :param str name: Interface to remove. + """ ++ ++ if name == 'default': ++ raise LinstorVolumeManagerError( ++ 'Unable to delete the default interface of a node!' ++ ) ++ + result = self._linstor.netinterface_delete(node_name, name) + errors = self._filter_errors(result) + if errors: diff --git a/SOURCES/0168-feat-linstorvolumemanager-change-logic-of-get_resour.patch b/SOURCES/0168-feat-linstorvolumemanager-change-logic-of-get_resour.patch new file mode 100644 index 0000000..c20b589 --- /dev/null +++ b/SOURCES/0168-feat-linstorvolumemanager-change-logic-of-get_resour.patch @@ -0,0 +1,101 @@ +From 1d940461f9f408b93eef80c09156f74745d62570 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 15 Apr 2024 17:56:47 +0200 +Subject: [PATCH 168/177] feat(linstorvolumemanager): change logic of + `get_resources_info`: - Add a nested level "nodes" for each resource - Add a + "uuid" attr on resources - Rename LINSTOR "uuid" to "linstor-uuid" - Optimize + code + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 1 + + drivers/linstorvolumemanager.py | 34 ++++++++++++++++++++------------- + 2 files changed, 22 insertions(+), 13 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index a2501d74..ed855257 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -884,6 +884,7 @@ def health_check(session, args): + 'controller-uri': '', + 'nodes': {}, + 'storage-pools': {}, ++ 'resources': {}, + 'warnings': [], + 'errors': [] + } +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 02059d81..25c226ac 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1624,7 +1624,7 @@ class LinstorVolumeManager(object): + + storage_pools[pool.node_name].append({ + 'storage-pool-name': pool.name, +- 'uuid': pool.uuid, ++ 'linstor-uuid': pool.uuid, + 'free-size': size, + 'capacity': capacity + }) +@@ -1637,16 +1637,19 @@ class LinstorVolumeManager(object): + :rtype: dict(str, list) + """ + resources = {} +- resource_list = self._linstor.resource_list_raise() ++ resource_list = self._get_resource_cache() ++ volume_names = self.get_volumes_with_name() + for resource in resource_list.resources: + if resource.name not in resources: +- resources[resource.name] = {} ++ resources[resource.name] = { 'nodes': {}, 'uuid': '' } ++ resource_nodes = resources[resource.name]['nodes'] + +- resources[resource.name][resource.node_name] = { ++ resource_nodes[resource.node_name] = { + 'volumes': [], + 'diskful': linstor.consts.FLAG_DISKLESS not in resource.flags, + 'tie-breaker': linstor.consts.FLAG_TIE_BREAKER in resource.flags + } ++ resource_volumes = resource_nodes[resource.node_name]['volumes'] + + for volume in resource.volumes: + # We ignore diskless pools of the form "DfltDisklessStorPool". +@@ -1665,17 +1668,17 @@ class LinstorVolumeManager(object): + else: + allocated_size *= 1024 + +- resources[resource.name][resource.node_name]['volumes'].append({ +- 'storage-pool-name': volume.storage_pool_name, +- 'uuid': volume.uuid, +- 'number': volume.number, +- 'device-path': volume.device_path, +- 'usable-size': usable_size, +- 'allocated-size': allocated_size +- }) ++ resource_volumes.append({ ++ 'storage-pool-name': volume.storage_pool_name, ++ 'linstor-uuid': volume.uuid, ++ 'number': volume.number, ++ 'device-path': volume.device_path, ++ 'usable-size': usable_size, ++ 'allocated-size': allocated_size ++ }) + + for resource_state in resource_list.resource_states: +- resource = resources[resource_state.rsc_name][resource_state.node_name] ++ resource = resources[resource_state.rsc_name]['nodes'][resource_state.node_name] + resource['in-use'] = resource_state.in_use + + volumes = resource['volumes'] +@@ -1684,6 +1687,11 @@ class LinstorVolumeManager(object): + if volume: + volume['disk-state'] = volume_state.disk_state + ++ for volume_uuid, volume_name in volume_names.items(): ++ resource = resources.get(volume_name) ++ if resource: ++ resource['uuid'] = volume_uuid ++ + return resources + + def get_database_path(self): diff --git a/SOURCES/0169-feat-linstor-manager-add-error-codes-to-healthCheck-.patch b/SOURCES/0169-feat-linstor-manager-add-error-codes-to-healthCheck-.patch new file mode 100644 index 0000000..2d2f456 --- /dev/null +++ b/SOURCES/0169-feat-linstor-manager-add-error-codes-to-healthCheck-.patch @@ -0,0 +1,248 @@ +From c377cf900ae07d3b6de4004dd3a65df9086c30b3 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 18 Apr 2024 13:57:37 +0200 +Subject: [PATCH 169/177] feat(linstor-manager): add error codes to healthCheck + helper + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 153 +++++++++++++++++++++++++------- + drivers/linstorvolumemanager.py | 2 +- + 2 files changed, 120 insertions(+), 35 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index ed855257..94aa4fb9 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -27,6 +27,7 @@ import socket + import XenAPI + import XenAPIPlugin + ++from json import JSONEncoder + from linstorjournaler import LinstorJournaler + from linstorvhdutil import LinstorVhdUtil + from linstorvolumemanager import get_controller_uri, get_local_volume_openers, LinstorVolumeManager +@@ -877,6 +878,64 @@ def get_drbd_openers(session, args): + raise + + ++class HealthCheckError(object): ++ __slots__ = ('data') ++ ++ MASK_REPORT_LEVEL = 0x7000000 ++ MASK_TYPE = 0xFF0000 ++ MASK_VALUE = 0XFFFF ++ ++ # 24-26 bits ++ REPORT_LEVEL_WARN = 0x1000000 ++ REPORT_LEVEL_ERR = 0x2000000 ++ ++ # 16-23 bits ++ TYPE_GENERIC = 0x10000 ++ TYPE_NODE = 0x20000 ++ TYPE_STORAGE_POOL = 0x30000 ++ TYPE_VOLUME = 0x40000 ++ TYPE_RESOURCE = 0x50000 ++ ++ # 1-15 bits ++ GENERIC_UNEXPECTED = REPORT_LEVEL_ERR | TYPE_GENERIC | 0 ++ GENERIC_LINSTOR_UNREACHABLE = REPORT_LEVEL_ERR | TYPE_GENERIC | 1 ++ ++ NODE_NOT_ONLINE = REPORT_LEVEL_WARN | TYPE_NODE | 0 ++ ++ STORAGE_POOL_UNKNOWN_FREE_SIZE = REPORT_LEVEL_ERR | TYPE_STORAGE_POOL | 0 ++ STORAGE_POOL_UNKNOWN_CAPACITY = REPORT_LEVEL_ERR | TYPE_STORAGE_POOL | 1 ++ STORAGE_POOL_LOW_FREE_SIZE = REPORT_LEVEL_WARN | TYPE_STORAGE_POOL | 2 ++ ++ VOLUME_UNKNOWN_STATE = REPORT_LEVEL_WARN | TYPE_VOLUME | 0 ++ VOLUME_INVALID_STATE = REPORT_LEVEL_ERR | TYPE_VOLUME | 1 ++ VOLUME_WRONG_DISKLESS_STATE = REPORT_LEVEL_WARN | TYPE_VOLUME | 2 ++ VOLUME_INTERNAL_UNVERIFIED_STATE = REPORT_LEVEL_WARN | TYPE_VOLUME | 3 ++ ++ MAP_CODE_TO_PARAMS = { ++ GENERIC_UNEXPECTED: { 'message' }, ++ GENERIC_LINSTOR_UNREACHABLE: { 'message' }, ++ NODE_NOT_ONLINE: { 'name', 'status' }, ++ STORAGE_POOL_UNKNOWN_FREE_SIZE: { 'name' }, ++ STORAGE_POOL_UNKNOWN_CAPACITY: { 'name' }, ++ STORAGE_POOL_LOW_FREE_SIZE: { 'name', 'threshold' }, ++ VOLUME_UNKNOWN_STATE: { 'node', 'resource', 'number' }, ++ VOLUME_INVALID_STATE: { 'node', 'resource', 'number', 'state' }, ++ VOLUME_WRONG_DISKLESS_STATE: { 'node', 'resource', 'number', 'state' }, ++ VOLUME_INTERNAL_UNVERIFIED_STATE: { 'node', 'resource', 'number', 'state' } ++ } ++ ++ def __init__(self, code, **kwargs): ++ attributes = self.MAP_CODE_TO_PARAMS[code] ++ data = { 'code': code } ++ for attr_name, attr_value in kwargs.items(): ++ assert attr_name in attributes ++ data[attr_name] = attr_value ++ self.data = data ++ ++ def to_json(self): ++ return self.data ++ ++ + def health_check(session, args): + group_name = args['groupName'] + +@@ -885,11 +944,15 @@ def health_check(session, args): + 'nodes': {}, + 'storage-pools': {}, + 'resources': {}, +- 'warnings': [], + 'errors': [] + } + + def format_result(): ++ # See: https://stackoverflow.com/questions/18478287/making-object-json-serializable-with-regular-encoder/18561055#18561055 ++ def _default(self, obj): ++ return getattr(obj.__class__, 'to_json', _default.default)(obj) ++ _default.default = JSONEncoder().default ++ JSONEncoder.default = _default + return json.dumps(result) + + # 1. Get controller. +@@ -912,7 +975,10 @@ def health_check(session, args): + ) + except Exception as e: + # Probably a network issue, or offline controller. +- result['errors'].append('Cannot join SR: `{}`.'.format(e)) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.GENERIC_LINSTOR_UNREACHABLE, ++ message=str(e) ++ )) + return format_result() + + try: +@@ -921,7 +987,11 @@ def health_check(session, args): + result['nodes'] = nodes + for node_name, status in nodes.items(): + if status != 'ONLINE': +- result['warnings'].append('Node `{}` is {}.'.format(node_name, status)) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.NODE_NOT_ONLINE, ++ name=node_name, ++ status=status ++ )) + + # 3. Check storage pool statuses. + storage_pools_per_node = linstor.get_storage_pools_info() +@@ -931,23 +1001,25 @@ def health_check(session, args): + free_size = storage_pool['free-size'] + capacity = storage_pool['capacity'] + if free_size < 0 or capacity <= 0: +- result['errors'].append( +- 'Cannot get free size and/or capacity of storage pool `{}`.' +- .format(storage_pool['uuid']) +- ) +- elif free_size > capacity: +- result['errors'].append( +- 'Free size of storage pool `{}` is greater than capacity.' +- .format(storage_pool['uuid']) +- ) ++ if free_size < 0: ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.STORAGE_POOL_UNKNOWN_FREE_SIZE, ++ name=storage_pool['name'] ++ )) ++ elif capacity < 0: ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.STORAGE_POOL_UNKNOWN_CAPACITY, ++ name=storage_pool['name'] ++ )) + else: + remaining_percent = free_size / float(capacity) * 100.0 + threshold = 10.0 + if remaining_percent < threshold: +- result['warnings'].append( +- 'Remaining size of storage pool `{}` is below {}% of its capacity.' +- .format(storage_pool['uuid'], threshold) +- ) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.STORAGE_POOL_LOW_FREE_SIZE, ++ name=storage_pool['name'], ++ threshold=threshold ++ )) + + # 4. Check resource statuses. + all_resources = linstor.get_resources_info() +@@ -960,33 +1032,46 @@ def health_check(session, args): + if disk_state in ['UpToDate', 'Created', 'Attached']: + continue + if disk_state == 'DUnknown': +- result['warnings'].append( +- 'Unknown state for volume `{}` at index {} for resource `{}` on node `{}`' +- .format(volume['device-path'], volume_index, resource_name, node_name) +- ) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.VOLUME_UNKNOWN_STATE, ++ node=node_name, ++ resource=resource_name, ++ number=volume_index ++ )) + continue + if disk_state in ['Inconsistent', 'Failed', 'To: Creating', 'To: Attachable', 'To: Attaching']: +- result['errors'].append( +- 'Invalid state `{}` for volume `{}` at index {} for resource `{}` on node `{}`' +- .format(disk_state, volume['device-path'], volume_index, resource_name, node_name) +- ) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.VOLUME_INVALID_STATE, ++ node=node_name, ++ resource=resource_name, ++ number=volume_index, ++ state=disk_state ++ )) + continue + if disk_state == 'Diskless': + if resource['diskful']: +- result['errors'].append( +- 'Unintentional diskless state detected for volume `{}` at index {} for resource `{}` on node `{}`' +- .format(volume['device-path'], volume_index, resource_name, node_name) +- ) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.VOLUME_WRONG_DISKLESS_STATE, ++ node=node_name, ++ resource=resource_name, ++ number=volume_index, ++ state=disk_state ++ )) + elif resource['tie-breaker']: + volume['disk-state'] = 'TieBreaker' + continue +- result['warnings'].append( +- 'Unhandled state `{}` for volume `{}` at index {} for resource `{}` on node `{}`' +- .format(disk_state, volume['device-path'], volume_index, resource_name, node_name) +- ) +- ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.VOLUME_INTERNAL_UNVERIFIED_STATE, ++ node=node_name, ++ resource=resource_name, ++ number=volume_index, ++ state=disk_state ++ )) + except Exception as e: +- result['errors'].append('Unexpected error: `{}`'.format(e)) ++ result['errors'].append(HealthCheckError( ++ code=HealthCheckError.GENERIC_UNEXPECTED, ++ message=str(e) ++ )) + + return format_result() + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 25c226ac..94d5c514 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -1623,7 +1623,7 @@ class LinstorVolumeManager(object): + capacity *= 1024 + + storage_pools[pool.node_name].append({ +- 'storage-pool-name': pool.name, ++ 'name': pool.name, + 'linstor-uuid': pool.uuid, + 'free-size': size, + 'capacity': capacity diff --git a/SOURCES/0170-fix-LinstorSR-fix-bad-exception-reference-during-sna.patch b/SOURCES/0170-fix-LinstorSR-fix-bad-exception-reference-during-sna.patch new file mode 100644 index 0000000..0618e2e --- /dev/null +++ b/SOURCES/0170-fix-LinstorSR-fix-bad-exception-reference-during-sna.patch @@ -0,0 +1,36 @@ +From 84e236ff9a967f5df25535213f89e377d51d4b5f Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 24 Apr 2024 15:10:49 +0200 +Subject: [PATCH 170/177] fix(LinstorSR): fix bad exception reference during + snapshot + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 3421f79f..3bd31e9e 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -2500,17 +2500,17 @@ class LinstorVDI(VDI.VDI): + self.session.xenapi.VDI.set_sm_config( + vdi_ref, active_vdi.sm_config + ) +- except Exception: ++ except Exception as e: + util.logException('Failed to snapshot!') + try: + self.sr._handle_interrupted_clone( + active_uuid, clone_info, force_undo=True + ) + self.sr._journaler.remove(LinstorJournaler.CLONE, active_uuid) +- except Exception as e: ++ except Exception as clean_error: + util.SMlog( + 'WARNING: Failed to clean up failed snapshot: {}' +- .format(e) ++ .format(clean_error) + ) + raise xs_errors.XenError('VDIClone', opterr=str(e)) + diff --git a/SOURCES/0171-fix-tapdisk-pause-ensure-LINSTOR-VHD-chain-is-availa.patch b/SOURCES/0171-fix-tapdisk-pause-ensure-LINSTOR-VHD-chain-is-availa.patch new file mode 100644 index 0000000..d0b342a --- /dev/null +++ b/SOURCES/0171-fix-tapdisk-pause-ensure-LINSTOR-VHD-chain-is-availa.patch @@ -0,0 +1,156 @@ +From 66ec66a863f3d5c8c95e5310dd2e9c77335faab5 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Wed, 24 Apr 2024 17:29:26 +0200 +Subject: [PATCH 171/177] fix(tapdisk-pause): ensure LINSTOR VHD chain is + available + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 35 ++--------------------------------- + drivers/linstorvhdutil.py | 38 ++++++++++++++++++++++++++++++++++++++ + drivers/tapdisk-pause | 6 ++++-- + 3 files changed, 44 insertions(+), 35 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 3bd31e9e..c5ed7c58 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1830,7 +1830,7 @@ class LinstorVDI(VDI.VDI): + return self._attach_using_http_nbd() + + # Ensure we have a path... +- self._create_chain_paths(self.uuid) ++ self.sr._vhdutil.create_chain_paths(self.uuid) + + self.attached = True + return VDI.VDI.attach(self, self.sr.uuid, self.uuid) +@@ -2357,7 +2357,7 @@ class LinstorVDI(VDI.VDI): + raise xs_errors.XenError('SnapshotChainTooLong') + + # Ensure we have a valid path if we don't have a local diskful. +- self._create_chain_paths(self.uuid) ++ self.sr._vhdutil.create_chain_paths(self.uuid) + + volume_path = self.path + if not util.pathexists(volume_path): +@@ -2820,37 +2820,6 @@ class LinstorVDI(VDI.VDI): + self._kill_persistent_nbd_server(volume_name) + self._kill_persistent_http_server(volume_name) + +- def _create_chain_paths(self, vdi_uuid): +- # OPTIMIZE: Add a limit_to_first_allocated_block param to limit vhdutil calls. +- # Useful for the snapshot code algorithm. +- +- while vdi_uuid: +- path = self._linstor.get_device_path(vdi_uuid) +- if not util.pathexists(path): +- raise xs_errors.XenError( +- 'VDIUnavailable', opterr='Could not find: {}'.format(path) +- ) +- +- # Diskless path can be created on the fly, ensure we can open it. +- def check_volume_usable(): +- while True: +- try: +- with open(path, 'r+'): +- pass +- except IOError as e: +- if e.errno == errno.ENODATA: +- time.sleep(2) +- continue +- if e.errno == errno.EROFS: +- util.SMlog('Volume not attachable because RO. Openers: {}'.format( +- self.sr._linstor.get_volume_openers(vdi_uuid) +- )) +- raise +- break +- util.retry(check_volume_usable, 15, 2) +- +- vdi_uuid = self.sr._vhdutil.get_vhd_info(vdi_uuid).parentUuid +- + # ------------------------------------------------------------------------------ + + +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 23d8b6a0..17b7790e 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -21,6 +21,7 @@ import distutils.util + import errno + import json + import socket ++import time + import util + import vhdutil + import xs_errors +@@ -141,6 +142,43 @@ class LinstorVhdUtil: + self._session = session + self._linstor = linstor + ++ def create_chain_paths(self, vdi_uuid): ++ # OPTIMIZE: Add a limit_to_first_allocated_block param to limit vhdutil calls. ++ # Useful for the snapshot code algorithm. ++ ++ leaf_vdi_path = self._linstor.get_device_path(vdi_uuid) ++ path = leaf_vdi_path ++ while True: ++ if not util.pathexists(path): ++ raise xs_errors.XenError( ++ 'VDIUnavailable', opterr='Could not find: {}'.format(path) ++ ) ++ ++ # Diskless path can be created on the fly, ensure we can open it. ++ def check_volume_usable(): ++ while True: ++ try: ++ with open(path, 'r+'): ++ pass ++ except IOError as e: ++ if e.errno == errno.ENODATA: ++ time.sleep(2) ++ continue ++ if e.errno == errno.EROFS: ++ util.SMlog('Volume not attachable because RO. Openers: {}'.format( ++ self._linstor.get_volume_openers(vdi_uuid) ++ )) ++ raise ++ break ++ util.retry(check_volume_usable, 15, 2) ++ ++ vdi_uuid = self.get_vhd_info(vdi_uuid).parentUuid ++ if not vdi_uuid: ++ break ++ path = self._linstor.get_device_path(vdi_uuid) ++ ++ return leaf_vdi_path ++ + # -------------------------------------------------------------------------- + # Getters: read locally and try on another host in case of failure. + # -------------------------------------------------------------------------- +diff --git a/drivers/tapdisk-pause b/drivers/tapdisk-pause +index e0bca7be..c316cdfa 100755 +--- a/drivers/tapdisk-pause ++++ b/drivers/tapdisk-pause +@@ -30,6 +30,7 @@ import vhdutil + import lvmcache + + try: ++ from linstorvhdutil import LinstorVhdUtil + from linstorvolumemanager import get_controller_uri, LinstorVolumeManager + LINSTOR_AVAILABLE = True + except ImportError: +@@ -162,11 +163,12 @@ class Tapdisk: + dconf = session.xenapi.PBD.get_device_config(pbd) + group_name = dconf['group-name'] + +- device_path = LinstorVolumeManager( ++ linstor = LinstorVolumeManager( + get_controller_uri(), + group_name, + logger=util.SMlog +- ).get_device_path(self.vdi_uuid) ++ ) ++ device_path = LinstorVhdUtil(session, linstor).create_chain_paths(self.vdi_uuid) + + if realpath != device_path: + util.SMlog( diff --git a/SOURCES/0172-fix-linstorvhdutil-retry-check-on-another-machine-in.patch b/SOURCES/0172-fix-linstorvhdutil-retry-check-on-another-machine-in.patch new file mode 100644 index 0000000..61b2053 --- /dev/null +++ b/SOURCES/0172-fix-linstorvhdutil-retry-check-on-another-machine-in.patch @@ -0,0 +1,73 @@ +From 5a48b3ddc8683c3f6217c1bcce17d4f064efaee9 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 6 May 2024 18:15:00 +0200 +Subject: [PATCH 172/177] fix(linstorvhdutil): retry check on another machine + in case of failure (#54) + +Signed-off-by: Ronan Abhamon +--- + drivers/linstor-manager | 5 +++-- + drivers/linstorvhdutil.py | 19 +++++++++++++++++-- + 2 files changed, 20 insertions(+), 4 deletions(-) + +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 94aa4fb9..47cbd2b7 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -29,7 +29,7 @@ import XenAPIPlugin + + from json import JSONEncoder + from linstorjournaler import LinstorJournaler +-from linstorvhdutil import LinstorVhdUtil ++from linstorvhdutil import LinstorVhdUtil, check_ex + from linstorvolumemanager import get_controller_uri, get_local_volume_openers, LinstorVolumeManager + from lock import Lock + import json +@@ -390,7 +390,8 @@ def check(session, args): + args['ignoreMissingFooter'] + ) + fast = distutils.util.strtobool(args['fast']) +- return str(vhdutil.check(device_path, ignore_missing_footer, fast)) ++ check_ex(device_path, ignore_missing_footer, fast) ++ return str(True) + except Exception as e: + util.SMlog('linstor-manager:check error: {}'.format(e)) + raise +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index 17b7790e..fd2bc8be 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -50,6 +50,16 @@ def call_remote_method(session, host_ref, method, device_path, args): + return response + + ++def check_ex(path, ignoreMissingFooter = False, fast = False): ++ cmd = [vhdutil.VHD_UTIL, "check", vhdutil.OPT_LOG_ERR, "-n", path] ++ if ignoreMissingFooter: ++ cmd.append("-i") ++ if fast: ++ cmd.append("-B") ++ ++ vhdutil.ioretry(cmd) ++ ++ + class LinstorCallException(util.SMException): + def __init__(self, cmd_err): + self.cmd_err = cmd_err +@@ -188,9 +198,14 @@ class LinstorVhdUtil: + 'ignoreMissingFooter': ignore_missing_footer, + 'fast': fast + } +- return self._check(vdi_uuid, **kwargs) # pylint: disable = E1123 ++ try: ++ self._check(vdi_uuid, **kwargs) # pylint: disable = E1123 ++ return True ++ except Exception as e: ++ util.SMlog('Call to `check` failed: {}'.format(e)) ++ return False + +- @linstorhostcall(vhdutil.check, 'check') ++ @linstorhostcall(check_ex, 'check') + def _check(self, vdi_uuid, response): + return distutils.util.strtobool(response) + diff --git a/SOURCES/0173-fix-LinstorSR-explicit-errors-when-database-path-is-.patch b/SOURCES/0173-fix-LinstorSR-explicit-errors-when-database-path-is-.patch new file mode 100644 index 0000000..c24d52c --- /dev/null +++ b/SOURCES/0173-fix-LinstorSR-explicit-errors-when-database-path-is-.patch @@ -0,0 +1,46 @@ +From 13ff45ec98437db8d85e3d9c52d6bb353f8f9656 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Mon, 6 May 2024 21:35:36 +0200 +Subject: [PATCH 173/177] fix(LinstorSR): explicit errors when database path is + fetched + +--- + drivers/LinstorSR.py | 4 ++-- + drivers/linstorvolumemanager.py | 2 +- + 2 files changed, 3 insertions(+), 3 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index c5ed7c58..9f0986cd 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -766,7 +766,7 @@ class LinstorSR(SR.SR): + # is started without a shared and mounted /var/lib/linstor path. + try: + self._linstor.get_database_path() +- except Exception: ++ except Exception as e: + # Failed to get database path, ensure we don't have + # VDIs in the XAPI database... + if self.session.xenapi.SR.get_VDIs( +@@ -774,7 +774,7 @@ class LinstorSR(SR.SR): + ): + raise xs_errors.XenError( + 'SRUnavailable', +- opterr='Database is not mounted' ++ opterr='Database is not mounted or node name is invalid ({})'.format(e) + ) + + # Update the database before the restart of the GC to avoid +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 94d5c514..948d45df 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -2648,7 +2648,7 @@ class LinstorVolumeManager(object): + ) + except Exception as e: + raise LinstorVolumeManagerError( +- 'Unable to get resources during database creation: {}' ++ 'Unable to fetch database resource: {}' + .format(e) + ) + diff --git a/SOURCES/0174-fix-LinstorSR-Misc-fixes-on-destroy.patch b/SOURCES/0174-fix-LinstorSR-Misc-fixes-on-destroy.patch new file mode 100644 index 0000000..c381a90 --- /dev/null +++ b/SOURCES/0174-fix-LinstorSR-Misc-fixes-on-destroy.patch @@ -0,0 +1,133 @@ +From 34e414b5d11363ebcce4c0f3a98d9e8de6c90432 Mon Sep 17 00:00:00 2001 +From: Damien Thenot +Date: Tue, 30 Apr 2024 15:38:34 +0200 +Subject: [PATCH 174/177] fix(LinstorSR): Misc fixes on destroy + +linstor-manager: +- fix on get_drbd_volumes + Failed because the key backing-disk is not always present. + +LinstorSR: +- Refactored some hosts variable to reference the OpaqueRef status + +linstorvolumemanager: +- Add destroying DRBD remnants on each hosts before destroying resource groups +- Correctly remove DB files (#57) + `glob` method only returns the dir when not wildcard + is used in the path. + Therefore the remove call right after will fail everytime. + +Signed-off-by: Damien Thenot +Co-authored-by: Benjamin Reis +Co-authored-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 10 +++++----- + drivers/linstor-manager | 5 ++++- + drivers/linstorvolumemanager.py | 24 +++++++++++++++++++----- + 3 files changed, 28 insertions(+), 11 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 9f0986cd..8d958908 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -650,17 +650,17 @@ class LinstorSR(SR.SR): + opterr='Cannot get controller node name' + ) + +- host = None ++ host_ref = None + if node_name == 'localhost': +- host = util.get_this_host_ref(self.session) ++ host_ref = util.get_this_host_ref(self.session) + else: + for slave in util.get_all_slaves(self.session): + r_name = self.session.xenapi.host.get_record(slave)['hostname'] + if r_name == node_name: +- host = slave ++ host_ref = slave + break + +- if not host: ++ if not host_ref: + raise xs_errors.XenError( + 'LinstorSRDelete', + opterr='Failed to find host with hostname: {}'.format( +@@ -677,7 +677,7 @@ class LinstorSR(SR.SR): + 'groupName': self._group_name, + } + self._exec_manager_command( +- host, 'destroy', args, 'LinstorSRDelete' ++ host_ref, 'destroy', args, 'LinstorSRDelete' + ) + except Exception as e: + try: +diff --git a/drivers/linstor-manager b/drivers/linstor-manager +index 47cbd2b7..f0404b80 100755 +--- a/drivers/linstor-manager ++++ b/drivers/linstor-manager +@@ -241,7 +241,10 @@ def get_drbd_volumes(volume_group=None): + config = json.loads(stdout) + for resource in config: + for volume in resource['_this_host']['volumes']: +- backing_disk = volume['backing-disk'] ++ backing_disk = volume.get('backing-disk') ++ if not backing_disk: ++ continue ++ + match = BACKING_DISK_RE.match(backing_disk) + if not match: + continue +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 948d45df..103f91b7 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -18,7 +18,6 @@ + + import distutils.util + import errno +-import glob + import json + import linstor + import os.path +@@ -1362,13 +1361,27 @@ class LinstorVolumeManager(object): + + # 4.4. Refresh linstor connection. + # Without we get this error: +- # "Cannot delete resource group 'xcp-sr-linstor_group_thin_device' because it has existing resource definitions.." ++ # "Cannot delete resource group 'xcp-sr-linstor_group_thin_device' because it has existing resource definitions.." + # Because the deletion of the databse was not seen by Linstor for some reason. + # It seems a simple refresh of the Linstor connection make it aware of the deletion. + self._linstor.disconnect() + self._linstor.connect() + +- # 4.5. Destroy group and storage pools. ++ # 4.5. Destroy remaining drbd nodes on hosts. ++ # We check if there is a DRBD node on hosts that could mean blocking when destroying resource groups. ++ # It needs to be done locally by each host so we go through the linstor-manager plugin. ++ # If we don't do this sometimes, the destroy will fail when trying to destroy the resource groups with: ++ # "linstor-manager:destroy error: Failed to destroy SP `xcp-sr-linstor_group_thin_device` on node `r620-s2`: The specified storage pool 'xcp-sr-linstor_group_thin_device' on node 'r620-s2' can not be deleted as volumes / snapshot-volumes are still using it." ++ session = util.timeout_call(5, util.get_localAPI_session) ++ for host_ref in session.xenapi.host.get_all(): ++ try: ++ response = session.xenapi.host.call_plugin( ++ host_ref, 'linstor-manager', 'destroyDrbdVolumes', {'volume_group': self._group_name} ++ ) ++ except Exception as e: ++ util.SMlog('Calling destroyDrbdVolumes on host {} failed with error {}'.format(host_ref, e)) ++ ++ # 4.6. Destroy group and storage pools. + self._destroy_resource_group(self._linstor, self._group_name) + self._destroy_resource_group(self._linstor, self._ha_group_name) + for pool in self._get_storage_pools(force=True): +@@ -1381,8 +1394,9 @@ class LinstorVolumeManager(object): + + try: + self._start_controller(start=False) +- for file in glob.glob(DATABASE_PATH + '/'): +- os.remove(file) ++ for file in os.listdir(DATABASE_PATH): ++ if file != 'lost+found': ++ os.remove(DATABASE_PATH + '/' + file) + except Exception as e: + util.SMlog( + 'Ignoring failure after LINSTOR SR destruction: {}' diff --git a/SOURCES/0175-fix-LinstorSR-open-non-leaf-volumes-in-RO-mode-creat.patch b/SOURCES/0175-fix-LinstorSR-open-non-leaf-volumes-in-RO-mode-creat.patch new file mode 100644 index 0000000..b7d09f1 --- /dev/null +++ b/SOURCES/0175-fix-LinstorSR-open-non-leaf-volumes-in-RO-mode-creat.patch @@ -0,0 +1,82 @@ +From 383d6484765a0b316402f1899bc9a2ec2cf81521 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Thu, 20 Jun 2024 22:37:40 +0200 +Subject: [PATCH 175/177] fix(LinstorSR): open non-leaf volumes in RO mode + (create_chain_paths) + +We must never open non-leaf volumes with the write option. +Only read only mode should be used to allow any host to access DRBD data. +Otherwise an attach call on dom-0 can be interrupted because a host already has a read lock. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 10 +++++----- + drivers/linstorvhdutil.py | 5 +++-- + 2 files changed, 8 insertions(+), 7 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 8d958908..21270570 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -1793,10 +1793,10 @@ class LinstorVDI(VDI.VDI): + 'scan SR first to trigger auto-repair' + ) + +- if not attach_from_config or self.sr._is_master: +- writable = 'args' not in self.sr.srcmd.params or \ +- self.sr.srcmd.params['args'][0] == 'true' ++ writable = 'args' not in self.sr.srcmd.params or \ ++ self.sr.srcmd.params['args'][0] == 'true' + ++ if not attach_from_config or self.sr._is_master: + # We need to inflate the volume if we don't have enough place + # to mount the VHD image. I.e. the volume capacity must be greater + # than the VHD size + bitmap size. +@@ -1830,7 +1830,7 @@ class LinstorVDI(VDI.VDI): + return self._attach_using_http_nbd() + + # Ensure we have a path... +- self.sr._vhdutil.create_chain_paths(self.uuid) ++ self.sr._vhdutil.create_chain_paths(self.uuid, readonly=not writable) + + self.attached = True + return VDI.VDI.attach(self, self.sr.uuid, self.uuid) +@@ -2357,7 +2357,7 @@ class LinstorVDI(VDI.VDI): + raise xs_errors.XenError('SnapshotChainTooLong') + + # Ensure we have a valid path if we don't have a local diskful. +- self.sr._vhdutil.create_chain_paths(self.uuid) ++ self.sr._vhdutil.create_chain_paths(self.uuid, readonly=True) + + volume_path = self.path + if not util.pathexists(volume_path): +diff --git a/drivers/linstorvhdutil.py b/drivers/linstorvhdutil.py +index fd2bc8be..7f8efa12 100644 +--- a/drivers/linstorvhdutil.py ++++ b/drivers/linstorvhdutil.py +@@ -152,7 +152,7 @@ class LinstorVhdUtil: + self._session = session + self._linstor = linstor + +- def create_chain_paths(self, vdi_uuid): ++ def create_chain_paths(self, vdi_uuid, readonly=False): + # OPTIMIZE: Add a limit_to_first_allocated_block param to limit vhdutil calls. + # Useful for the snapshot code algorithm. + +@@ -168,7 +168,7 @@ class LinstorVhdUtil: + def check_volume_usable(): + while True: + try: +- with open(path, 'r+'): ++ with open(path, 'r' if readonly else 'r+'): + pass + except IOError as e: + if e.errno == errno.ENODATA: +@@ -186,6 +186,7 @@ class LinstorVhdUtil: + if not vdi_uuid: + break + path = self._linstor.get_device_path(vdi_uuid) ++ readonly = True # Non-leaf is always readonly. + + return leaf_vdi_path + diff --git a/SOURCES/0176-fix-LinstorSR-ensure-_is_master-is-always-set.patch b/SOURCES/0176-fix-LinstorSR-ensure-_is_master-is-always-set.patch new file mode 100644 index 0000000..88b2e98 --- /dev/null +++ b/SOURCES/0176-fix-LinstorSR-ensure-_is_master-is-always-set.patch @@ -0,0 +1,155 @@ +From 0cbabd180b8fc5ecaf2ca8ff066f37bd18706b68 Mon Sep 17 00:00:00 2001 +From: Ronan Abhamon +Date: Fri, 26 Jul 2024 11:32:20 +0200 +Subject: [PATCH 176/177] fix(LinstorSR): ensure `_is_master` is always set + +`_is_master` is not always initialized, and more precisely +in the case of detach where LinstorSR.load method is not called. + +Still in this same situation, this can lead to the deletion of DRBD diskless +on the master while we try to always have one so as not to needlessly +recreate one later. + +Signed-off-by: Ronan Abhamon +--- + drivers/LinstorSR.py | 36 +++++++++++++++++++++--------------- + 1 file changed, 21 insertions(+), 15 deletions(-) + +diff --git a/drivers/LinstorSR.py b/drivers/LinstorSR.py +index 21270570..fdcad17e 100755 +--- a/drivers/LinstorSR.py ++++ b/drivers/LinstorSR.py +@@ -362,9 +362,6 @@ class LinstorSR(SR.SR): + self._linstor = None # Ensure that LINSTOR attribute exists. + self._journaler = None + +- self._is_master = False +- if 'SRmaster' in self.dconf and self.dconf['SRmaster'] == 'true': +- self._is_master = True + self._group_name = self.dconf['group-name'] + + self._vdi_shared_time = 0 +@@ -437,7 +434,7 @@ class LinstorSR(SR.SR): + + return wrapped_method(self, *args, **kwargs) + +- if not self._is_master: ++ if not self.is_master(): + if self.cmd in [ + 'sr_create', 'sr_delete', 'sr_update', 'sr_probe', + 'sr_scan', 'vdi_create', 'vdi_delete', 'vdi_resize', +@@ -472,7 +469,7 @@ class LinstorSR(SR.SR): + + # Ensure we use a non-locked volume when vhdutil is called. + if ( +- self._is_master and self.cmd.startswith('vdi_') and ++ self.is_master() and self.cmd.startswith('vdi_') and + self.cmd != 'vdi_create' + ): + self._linstor.ensure_volume_is_not_locked( +@@ -487,7 +484,7 @@ class LinstorSR(SR.SR): + # + # If the command is a SR command we want at least to remove + # resourceless volumes. +- if self._is_master and self.cmd not in [ ++ if self.is_master() and self.cmd not in [ + 'vdi_attach', 'vdi_detach', + 'vdi_activate', 'vdi_deactivate', + 'vdi_epoch_begin', 'vdi_epoch_end', +@@ -783,6 +780,15 @@ class LinstorSR(SR.SR): + self._kick_gc() + return ret + ++ def is_master(self): ++ if not hasattr(self, '_is_master'): ++ if 'SRmaster' not in self.dconf: ++ self._is_master = self.session is not None and util.is_master(self.session) ++ else: ++ self._is_master = self.dconf['SRmaster'] == 'true' ++ ++ return self._is_master ++ + @_locked_load + def vdi(self, uuid): + return LinstorVDI(self, uuid) +@@ -968,7 +974,7 @@ class LinstorSR(SR.SR): + ) + + def _synchronize_metadata(self): +- if not self._is_master: ++ if not self.is_master(): + return + + util.SMlog('Synchronize metadata...') +@@ -1015,7 +1021,7 @@ class LinstorSR(SR.SR): + if self._vdis_loaded: + return + +- assert self._is_master ++ assert self.is_master() + + # We use a cache to avoid repeated JSON parsing. + # The performance gain is not big but we can still +@@ -1494,7 +1500,7 @@ class LinstorSR(SR.SR): + controller_uri, + self._group_name, + repair=( +- self._is_master and ++ self.is_master() and + self.srcmd.cmd in self.ops_exclusive + ), + logger=util.SMlog +@@ -1796,7 +1802,7 @@ class LinstorVDI(VDI.VDI): + writable = 'args' not in self.sr.srcmd.params or \ + self.sr.srcmd.params['args'][0] == 'true' + +- if not attach_from_config or self.sr._is_master: ++ if not attach_from_config or self.sr.is_master(): + # We need to inflate the volume if we don't have enough place + # to mount the VHD image. I.e. the volume capacity must be greater + # than the VHD size + bitmap size. +@@ -1878,7 +1884,7 @@ class LinstorVDI(VDI.VDI): + ) + + # We remove only on slaves because the volume can be used by the GC. +- if self.sr._is_master: ++ if self.sr.is_master(): + return + + while vdi_uuid: +@@ -1899,7 +1905,7 @@ class LinstorVDI(VDI.VDI): + + def resize(self, sr_uuid, vdi_uuid, size): + util.SMlog('LinstorVDI.resize for {}'.format(self.uuid)) +- if not self.sr._is_master: ++ if not self.sr.is_master(): + raise xs_errors.XenError( + 'VDISize', + opterr='resize on slave not allowed' +@@ -2158,7 +2164,7 @@ class LinstorVDI(VDI.VDI): + # -------------------------------------------------------------------------- + + def _prepare_thin(self, attach): +- if self.sr._is_master: ++ if self.sr.is_master(): + if attach: + attach_thin( + self.session, self.sr._journaler, self._linstor, +@@ -2747,7 +2753,7 @@ class LinstorVDI(VDI.VDI): + + # 0. Fetch drbd path. + must_get_device_path = True +- if not self.sr._is_master: ++ if not self.sr.is_master(): + # We are on a slave, we must try to find a diskful locally. + try: + volume_info = self._linstor.get_volume_info(self.uuid) +@@ -2762,7 +2768,7 @@ class LinstorVDI(VDI.VDI): + must_get_device_path = hostname in volume_info.diskful + + drbd_path = None +- if must_get_device_path or self.sr._is_master: ++ if must_get_device_path or self.sr.is_master(): + # If we are master, we must ensure we have a diskless + # or diskful available to init HA. + # It also avoid this error in xensource.log diff --git a/SOURCES/0177-fix-linstor-check-if-resource-is-tiebreaker-62.patch b/SOURCES/0177-fix-linstor-check-if-resource-is-tiebreaker-62.patch new file mode 100644 index 0000000..6c577c2 --- /dev/null +++ b/SOURCES/0177-fix-linstor-check-if-resource-is-tiebreaker-62.patch @@ -0,0 +1,32 @@ +From 8d6cfc70d30e13cdf6a8ea2be3a534bace9fb936 Mon Sep 17 00:00:00 2001 +From: Damien Thenot +Date: Fri, 26 Jul 2024 14:13:05 +0200 +Subject: [PATCH 177/177] fix(linstor): check if resource is tiebreaker (#62) + +We check if a resource is already a tiebreaker before trying to delete +the resource. +If it is, we do not delete it. + +Signed-off-by: Damien Thenot +--- + drivers/linstorvolumemanager.py | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/drivers/linstorvolumemanager.py b/drivers/linstorvolumemanager.py +index 103f91b7..8bfd1c1b 100755 +--- a/drivers/linstorvolumemanager.py ++++ b/drivers/linstorvolumemanager.py +@@ -812,6 +812,13 @@ class LinstorVolumeManager(object): + volume_name = volume_properties.get(self.PROP_VOLUME_NAME) + + node_name = socket.gethostname() ++ ++ for resource in self._get_resource_cache().resources: ++ if resource.name == volume_name and resource.node_name == node_name: ++ if linstor.consts.FLAG_TIE_BREAKER in resource.flags: ++ return ++ break ++ + result = self._linstor.resource_delete_if_diskless( + node_name=node_name, rsc_name=volume_name + ) diff --git a/SPECS/sm.spec b/SPECS/sm.spec index 36bc788..39f848c 100644 --- a/SPECS/sm.spec +++ b/SPECS/sm.spec @@ -11,7 +11,7 @@ Summary: sm - XCP storage managers Name: sm Version: 2.30.8 -Release: %{?xsrel}.1%{?dist} +Release: %{?xsrel}.1.0.linstor.1%{?dist} Group: System/Hypervisor License: LGPL URL: https://github.com/xapi-project/sm @@ -77,6 +77,7 @@ Requires(post): systemd Requires(preun): systemd Requires(postun): systemd Requires: xenserver-multipath +Requires(post): xenserver-multipath Requires: xenserver-lvm2 >= 2.02.180-11.xs+2.0.2 Requires: python2-bitarray Requires(post): xs-presets >= 1.3 @@ -86,6 +87,9 @@ Conflicts: kernel < 4.19.19-5.0.0 Obsoletes: sm-additional-drivers +# To remove after stable release of LINSTOR. +Provides: sm-linstor + # XCP-ng patches # Generated from our sm repository # git format-patch v2.30.8-13-xcpng..2.30.8-8.2 @@ -121,6 +125,160 @@ Patch1020: 0020-Backport-NFS4-only-support.patch Patch1021: 0021-Backport-probe-for-NFS4-when-rpcinfo-does-not-includ.patch Patch1022: 0022-feat-LargeBlock-backport-of-largeblocksr-51-55.patch Patch1023: 0023-feat-LVHDSR-add-a-way-to-modify-config-of-LVMs-56.patch +Patch1024: 0024-Fix-timeout_call-alarm-must-be-reset-in-case-of-succ.patch +Patch1025: 0025-timeout_call-returns-the-result-of-user-function-now.patch +Patch1026: 0026-Always-remove-the-pause-tag-from-VDIs-in-case-of-fai.patch +Patch1027: 0027-fix-LinstorSR-repair-volumes-only-if-an-exclusive-co.patch +Patch1028: 0028-feat-LinstorSR-Improve-LINSTOR-performance.patch +Patch1029: 0029-feat-LinstorSR-robustify-scan-to-avoid-losing-VDIs-i.patch +Patch1030: 0030-feat-LinstorSR-display-a-correctly-readable-size-for.patch +Patch1031: 0031-feat-linstor-monitord-scan-all-LINSTOR-SRs-every-12-.patch +Patch1032: 0032-fix-LinstorSR-call-correctly-method-in-_locked_load-.patch +Patch1033: 0033-feat-LinstorSR-integrate-minidrbdcluster-daemon.patch +Patch1034: 0034-feat-LinstorSR-ensure-heartbeat-and-redo_log-VDIs-ar.patch +Patch1035: 0035-feat-LinstorSR-protect-sr-commands-to-avoid-forgetti.patch +Patch1036: 0036-fix-LinstorJournaler-ensure-uri-is-not-None-during-l.patch +Patch1037: 0037-feat-LinstorSR-add-an-option-to-disable-auto-quorum-.patch +Patch1038: 0038-fix-LinstorVolumeManager-add-a-workaround-to-create-.patch +Patch1039: 0039-feat-LinstorSR-add-optional-ips-parameter.patch +Patch1040: 0040-feat-LinstorSR-add-a-helper-log_drbd_erofs-to-trace-.patch +Patch1041: 0041-fix-LinstorSR-try-to-restart-the-services-again-if-t.patch +Patch1042: 0042-fix-LinstorSR-robustify-linstor-manager-to-never-inc.patch +Patch1043: 0043-fix-LinstorSR-prevent-starting-controller-during-fai.patch +Patch1044: 0044-feat-LinstorVolumeManager-increase-peer-slots-limit-.patch +Patch1045: 0045-feat-LinstorVolumeManager-add-a-fallback-to-find-con.patch +Patch1046: 0046-fix-var-lib-linstor.mount-ensure-we-always-mount-dat.patch +Patch1047: 0047-feat-LinstorVolumeManager-add-a-fallback-to-find-nod.patch +Patch1048: 0048-feat-LinstorSR-explain-on-which-host-plugins-command.patch +Patch1049: 0049-fix-LinstorSR-create-diskless-path-if-necessary-duri.patch +Patch1050: 0050-feat-LinstorSR-use-HTTP-NBD-instead-of-DRBD-directly.patch +Patch1051: 0051-fix-LinstorSR-find-controller-when-XAPI-unreachable-.patch +Patch1052: 0052-fix-LinstorSR-use-IPs-instead-of-hostnames-in-NBD-se.patch +Patch1053: 0053-fix-LinstorVolumeManager-ensure-we-always-use-IPs-in.patch +Patch1054: 0054-feat-linstor-manager-add-methods-to-add-remove-host-.patch +Patch1055: 0055-feat-LinstorVolumeManager-support-SR-creation-with-d.patch +Patch1056: 0056-feat-LinstorSR-add-a-config-var-to-disable-HTTP-NBD-.patch +Patch1057: 0057-feat-LinstorSr-ensure-LVM-group-is-activated-during-.patch +Patch1058: 0058-feat-linstor-manager-add-method-to-create-LinstorSR-.patch +Patch1059: 0059-fix-LinstorSR-always-set-vdi_path-in-generate_config.patch +Patch1060: 0060-fix-minidrbdcluster-supports-new-properties-like-for.patch +Patch1061: 0061-fix-LinstorSR-enabled-disable-minidrbcluster-with-fi.patch +Patch1062: 0062-fix-linstor-manager-change-linstor-satellite-start-b.patch +Patch1063: 0063-Fix-is_open-call-for-LinstorSR.patch +Patch1064: 0064-fix-linstorvhdutil-fix-boolean-params-of-check-call.patch +Patch1065: 0065-feat-linstor-manager-robustify-exec_create_sr.patch +Patch1066: 0066-fix-cleanup-print-LINSTOR-VDI-UUID-if-error-during-i.patch +Patch1067: 0067-feat-cleanup-raise-and-dump-DRBD-openers-in-case-of-.patch +Patch1068: 0068-feat-linstorvhdutil-trace-DRBD-openers-in-case-of-ER.patch +Patch1069: 0069-fix-linstorvolumemanager-compute-correctly-size-in-a.patch +Patch1070: 0070-feat-LinstorSR-use-DRBD-openers-instead-of-lsof-to-l.patch +Patch1071: 0071-feat-LinstorSR-support-cProfile-to-trace-calls-when-.patch +Patch1072: 0072-fix-LinstorJournaler-reset-namespace-when-get-is-cal.patch +Patch1073: 0073-fix-linstorvhdutil-fix-coalesce-with-VM-running-unde.patch +Patch1074: 0074-fix-linstorvolumemanager-_get_volumes_info-doesn-t-r.patch +Patch1075: 0075-fix-linstorvolumemanager-remove-double-prefix-on-kv-.patch +Patch1076: 0076-feat-LinstorSR-add-linstor-kv-dump-helper-to-print-k.patch +Patch1077: 0077-fix-LinstorSR-disable-VHD-key-hash-usage-to-limit-ex.patch +Patch1078: 0078-fix-minidrbdcluster-ensure-SIGINT-is-handled-correct.patch +Patch1079: 0079-feat-minidrbdcluster-stop-resource-services-at-start.patch +Patch1080: 0080-feat-linstor-manager-add-new-healthCheck-function-to.patch +Patch1081: 0081-fix-LinstorSR-fix-xha-conf-parsing-return-host-ip-no.patch +Patch1082: 0082-fix-LinstorSR-start-correctly-HA-servers-HTTP-NBD-af.patch +Patch1083: 0083-fix-linstorvolumemanager-use-an-array-to-store-diskf.patch +Patch1084: 0084-feat-linstorvolumemanager-support-snaps-when-a-host-.patch +Patch1085: 0085-fix-linstorvolumemanager-support-offline-hosts-when-.patch +Patch1086: 0086-fix-linstorvolumemanager-define-_base_group_name-mem.patch +Patch1087: 0087-feat-linstorvhdutil-modify-logic-of-local-vhdutil-ca.patch +Patch1088: 0088-fix-linstorvolumemanager-robustify-failed-snapshots.patch +Patch1089: 0089-fix-linstorvolumemanager-use-a-namespace-for-volumes.patch +Patch1090: 0090-feat-linstor-kv-dump-rename-to-linstor-kv-tool-add-r.patch +Patch1091: 0091-fix-LinstorSR-handle-correctly-localhost-during-star.patch +Patch1092: 0092-fix-cleanup.py-call-repair-on-another-host-when-EROF.patch +Patch1093: 0093-fix-LinstorSR-avoid-introduction-of-DELETED-volumes.patch +Patch1094: 0094-feat-linstor-kv-tool-remove-all-volumes-supports-jou.patch +Patch1095: 0095-fix-linstorvhdutil-due-to-bad-refactoring-check-call.patch +Patch1096: 0096-feat-linstorvhdutil-ensure-we-use-VHD-parent-to-find.patch +Patch1097: 0097-feat-linstorvolumemanager-force-DRBD-demote-after-fa.patch +Patch1098: 0098-fix-linstorvhdutil-ensure-we-retry-creation-in-all-s.patch +Patch1099: 0099-fix-linstorvhdutil-don-t-retry-local-vhdutil-call-wh.patch +Patch1100: 0100-feat-fork-log-daemon-ignore-SIGTERM.patch +Patch1101: 0101-feat-LinstorSR-wait-for-http-disk-server-startup.patch +Patch1102: 0102-fix-LinstorSR-handle-inflate-resize-actions-correctl.patch +Patch1103: 0103-fix-linstor-manager-add-a-static-iptables-rule-for-D.patch +Patch1104: 0104-feat-LinstorSR-sync-with-last-http-nbd-transfer-vers.patch +Patch1105: 0105-fix-LinstorSR-don-t-check-VDI-metadata-while-listing.patch +Patch1106: 0106-fix-LinstorSR-don-t-check-metadata-when-destroying-s.patch +Patch1107: 0107-fix-linstorvhdutil-handle-correctly-generic-exceptio.patch +Patch1108: 0108-fix-minidrbdcluster-robustify-to-unmount-correctly-L.patch +Patch1109: 0109-fix-minidrbdcluster-handle-correctly-KeyboardInterru.patch +Patch1110: 0110-feat-LinstorSR-use-drbd-reactor-instead-of-minidrbdc.patch +Patch1111: 0111-fix-LinstorSR-ensure-vhdutil-calls-are-correctly-exe.patch +Patch1112: 0112-fix-LinstorSR-replace-bad-param-in-detach_thin-impl.patch +Patch1113: 0113-fix-linstorvolumemanager-remove-usage-of-realpath.patch +Patch1114: 0114-fix-linstorvhdutil-avoid-parent-path-resolution.patch +Patch1115: 0115-fix-LinstorSR-create-parent-path-during-attach.patch +Patch1116: 0116-fix-LinstorSR-retry-if-we-can-t-build-volume-cache.patch +Patch1117: 0117-fix-linstorvolumemanager-reduce-peer-slots-param-to-.patch +Patch1118: 0118-fix-LinstorSR-attach-a-valid-XAPI-session-is_open-is.patch +Patch1119: 0119-fix-LinstorSR-ensure-we-always-have-a-DRBD-path-to-s.patch +Patch1120: 0120-fix-LinstorSR-remove-hosts-ips-param.patch +Patch1121: 0121-fix-LinstorSR-compute-correctly-SR-size-using-pool-c.patch +Patch1122: 0122-fix-blktap2-ensure-we-can-import-this-module-when-LI.patch +Patch1123: 0123-fix-LinstorSR-ensure-volume-cache-can-be-recreated.patch +Patch1124: 0124-fix-linstor-manager-remove-dead-useless-code-in-add-.patch +Patch1125: 0125-fix-LinstorSR-Ensure-we-always-have-a-device-path-du.patch +Patch1126: 0126-fix-LinstorSR-always-use-lock.acquire-during-attach-.patch +Patch1127: 0127-fix-LinstorSR-mare-sure-hostnames-are-unique-at-SR-c.patch +Patch1128: 0128-fix-LinstorSR-ensure-we-can-attach-non-special-stati.patch +Patch1129: 0129-fix-LinstorSR-ensure-we-can-detach-when-deflate-call.patch +Patch1130: 0130-fix-LinstorSR-assume-VDI-is-always-a-VHD-when-the-in.patch +Patch1131: 0131-fix-LinstorSR-remove-SR-lock-during-thin-attach-deta.patch +Patch1132: 0132-fix-LinstorSR-ensure-database-is-mounted-during-scan.patch +Patch1133: 0133-fix-LinstorSR-restart-drbd-reactor-in-case-of-failur.patch +Patch1134: 0134-fix-linstorvolumemanager-retry-in-case-of-failure-du.patch +Patch1135: 0135-fix-linstorvolumemanager-avoid-diskless-creation-whe.patch +Patch1136: 0136-fix-LinstorSR-remove-diskless-after-VDI.detach-calls.patch +Patch1137: 0137-fix-LinstorSR-robustify-_load_vdi_info-in-cleanup.py.patch +Patch1138: 0138-fix-LinstorSR-ensure-detach-never-fails-on-plugin-fa.patch +Patch1139: 0139-fix-LinstorSR-ensure-we-coalesce-only-volumes-with-a.patch +Patch1140: 0140-fix-LinstorSR-don-t-try-to-repair-persistent-volumes.patch +Patch1141: 0141-fix-linstorvhdutil-format-correctly-message-if-vhd-u.patch +Patch1142: 0142-fix-LinstorSR-wait-during-attach-to-open-DRBD-path.patch +Patch1143: 0143-fix-LinstorSR-support-different-volume-sizes-in-clea.patch +Patch1144: 0144-fix-LinstorSR-remove-useless-IPS_XHA_CACHE-var.patch +Patch1145: 0145-fix-LinstorSR-ensure-we-can-deflate-on-any-host-afte.patch +Patch1146: 0146-fix-LinstorSR-ensure-we-always-use-real-DRBD-VHD-siz.patch +Patch1147: 0147-feat-linstor-kv-tool-If-no-controller-uri-option-is-.patch +Patch1148: 0148-fix-linstorvolumemanager-robustify-SR-destroy-46.patch +Patch1149: 0149-feat-linstor-manager-extend-API-with-createNodeInter.patch +Patch1150: 0150-fix-LinstorSR-support-VDI.resize-on-thick-volumes.patch +Patch1151: 0151-fix-linstorvolumemanager-format-correctly-exception-.patch +Patch1152: 0152-fix-LinstorSR-ensure-we-can-skip-coalesces-if-device.patch +Patch1153: 0153-feat-linstor-manager-add-methods-to-modify-destroy-l.patch +Patch1154: 0154-fix-LinstorSR-force-a-defined-volume-prefix-if-we-ca.patch +Patch1155: 0155-fix-LinstorSR-explicit-error-message-when-a-group-is.patch +Patch1156: 0156-fix-LinstorSR-make-sure-VDI.delete-doesn-t-throw-und.patch +Patch1157: 0157-fix-LinstorSR-add-drbd-in-the-blacklist-of-multipath.patch +Patch1158: 0158-fix-linstorvolumemanager-create-cloned-volumes-on-ho.patch +Patch1159: 0159-fix-linstorvolumemanager-don-t-align-volumes-on-LVM-.patch +Patch1160: 0160-fix-linstorvolumemanager-assert-with-message-after-l.patch +Patch1161: 0161-fix-linstorvolumemanager-retry-resize-if-volume-is-n.patch +Patch1162: 0162-fix-LinstorSR-create-DRBD-diskless-if-necessary-for-.patch +Patch1163: 0163-fix-LinstorSR-fix-bad-call-to-vhdutil.inflate-bad-ex.patch +Patch1164: 0164-fix-LinstorSR-activate-VG-if-attach-from-config-is-a.patch +Patch1165: 0165-feat-LinstorSR-use-a-specific-resource-group-for-DB-.patch +Patch1166: 0166-feat-linstor-manager-add-getNodePreferredInterface-h.patch +Patch1167: 0167-fix-linstorvolumemanager-blocks-deletion-of-default-.patch +Patch1168: 0168-feat-linstorvolumemanager-change-logic-of-get_resour.patch +Patch1169: 0169-feat-linstor-manager-add-error-codes-to-healthCheck-.patch +Patch1170: 0170-fix-LinstorSR-fix-bad-exception-reference-during-sna.patch +Patch1171: 0171-fix-tapdisk-pause-ensure-LINSTOR-VHD-chain-is-availa.patch +Patch1172: 0172-fix-linstorvhdutil-retry-check-on-another-machine-in.patch +Patch1173: 0173-fix-LinstorSR-explicit-errors-when-database-path-is-.patch +Patch1174: 0174-fix-LinstorSR-Misc-fixes-on-destroy.patch +Patch1175: 0175-fix-LinstorSR-open-non-leaf-volumes-in-RO-mode-creat.patch +Patch1176: 0176-fix-LinstorSR-ensure-_is_master-is-always-set.patch +Patch1177: 0177-fix-linstor-check-if-resource-is-tiebreaker-62.patch %description This package contains storage backends used in XCP @@ -175,6 +333,14 @@ systemctl start sr_health_check.timer # However it won't start without linstor-controller.service systemctl enable linstor-monitor.service +# XCP-ng: We must reload the multipathd configuration without restarting the service to prevent +# the opening of /dev/drbdXXXX volumes. Otherwise if multipathd opens a DRBD volume, +# it blocks its access to other hosts. +# This command is also important if our multipath conf is modified for other drivers. +if [ $1 -gt 1 ]; then + multipathd reconfigure +fi + %preun %systemd_preun make-dummy-sr.service %systemd_preun mpcount.service @@ -223,6 +389,9 @@ cp -r htmlcov %{buildroot}/htmlcov %files %defattr(-,root,root,-) +/etc/systemd/system/drbd-reactor.service.d/override.conf +/etc/systemd/system/linstor-satellite.service.d/override.conf +/etc/systemd/system/var-lib-linstor.service /etc/udev/scripts/xs-mpath-scsidev.sh /etc/xapi.d/plugins/coalesce-leaf /etc/xapi.d/plugins/lvhd-thin @@ -234,6 +403,7 @@ cp -r htmlcov %{buildroot}/htmlcov /etc/xapi.d/plugins/trim /etc/xensource/master.d/02-vhdcleanup /opt/xensource/bin/blktap2 +/opt/xensource/bin/linstor-kv-tool /opt/xensource/bin/tapdisk-cache-stats /opt/xensource/bin/xe-getarrayidentifier /opt/xensource/bin/xe-get-arrayid-lunnum @@ -243,6 +413,7 @@ cp -r htmlcov %{buildroot}/htmlcov /opt/xensource/libexec/dcopy /opt/xensource/libexec/local-device-change /opt/xensource/libexec/make-dummy-sr +/opt/xensource/libexec/safe-umount /opt/xensource/libexec/usb_change /opt/xensource/libexec/kickpipe /opt/xensource/libexec/set-iscsi-initiator @@ -520,10 +691,16 @@ cp -r htmlcov %{buildroot}/htmlcov /opt/xensource/sm/linstorvolumemanager.py /opt/xensource/sm/linstorvolumemanager.pyc /opt/xensource/sm/linstorvolumemanager.pyo +/opt/xensource/libexec/fork-log-daemon /opt/xensource/libexec/linstor-monitord %{_unitdir}/linstor-monitor.service %changelog +* Thu Oct 03 2024 Ronan Abhamon - 2.30.8-13.1.0.linstor.1 +- Add "Provides": sm-linstor (necessary for the "Requires" of xcp-ng-linstor) +- Add LINSTOR patches +- Reload automatically multipathd config after each update + * Thu Oct 03 2024 Ronan Abhamon - 2.30.8-13.1 - Sync with hotfix XS82ECU1075 - Sync patches with our latest 2.30.8-8.2 branch