Upstream Image Encryption Standardization #560
Assignees
Comments
josephineSei
added
SCS-VP10
|
I have discussed and outlined the spec with @markus-hentsch and currently writing it. |
|
On the user side we'd need to be able to create LUKS-encrypted files. Nova devs made us aware of the QEMU tooling being able to do so allegedly. Here is the Nova implementation:
Here's a shortened example: echo "muchsecretsuchwow" > secret_file.key
qemu-img convert -f raw -O luks --object secret,id=sec,file=secret_file.key -o key-secret=sec \
-o cipher-alg=aes-256 -o cipher-mode=xts -o hash-alg=sha256 \
-o ivgen-alg=plain64 -o ivgen-hash-alg=sha256 \
$INPUT_FILE $OUTPUT_LUKS_FILEIt works within a Docker container running Ubuntu LTS but interestingly it doesn't on my NixOS setup, failing with: Versions of qemu-img are 6.2.0 (Ubuntu) and 8.1.5 (NixOS) respectively. I wonder if there is another system dependency in play here or if it's the version difference. We should investigate because this impacts the ability to create images on the client side. EDIT: Tried it on |
|
As I was adding restore instructions to the user data backup guide in SovereignCloudStack/docs#176 I reproduced the process of creating volumes from previously encrypted (LUKS) images originating from Cinder itself on my DevStack and noticed that Cinder does not verify that the target volume actually has an encrypted type: $ file image.raw
image.raw: LUKS encrypted file, ver 1 [aes, xts-plain64, sha256]
$ file image.key
image.key: data
$ openstack secret store --algorithm aes --bit-length 256 --mode cbc \
--secret-type symmetric --file image.key --name restored-image-key
$ openstack secret list -f value -c "Secret href" -c "Name"
http://10.0.1.116/key-manager/v1/secrets/6ea6b0a8-de50-45b8-90b7-9470c4dd201a restored-image-key
$ export SECRET_ID=6ea6b0a8-de50-45b8-90b7-9470c4dd201a
$ openstack image create --file image.raw \
--property cinder_encryption_key_id=$SECRET_ID \
--property cinder_encryption_key_deletion_policy=on_image_deletion \
restored-image
$ openstack volume create --size 1 \
--image restored-image volume-restored-notype
$ openstack volume show -f value -c type volume-restored-notype
lvmdriver-1 # <- this is an *unencrypted* volume type! (which contains LUKS blocks now)
$ openstack volume create --size 1 \
--image restored-image --type lvmdriver-1-LUKS \
volume-restored-lukstype
$ openstack volume show -f value -c type volume-restored-lukstype
lvmdriver-1-LUKS
$ openstack server create \
--volume volume-restored-notype \
... server-from-untyped-volume
$ openstack server create \
--volume volume-restored-lukstype \
... server-from-luks-volumeThis results in the server It seems that this is an oversight in Cinder in its current implementation? |
|
I tested this with a simple encrypted volume to encrypted image to volume: A seemingly unencrypted volume is created from the encrypted image. But as far as we know, there is no decrypting mechanism implemented in Cinder to go from such a Cinder-specific LUKS encryption in the image to an unencrypted volume. We should definitely file a bug in Cinder for this one. |
How to test, if a server bootedWhen creating a server from a volume, like my above case, it will be shown as active even though this cannot be true To show the boot log of a server the following command can be used: The log will then look something like: But if created from a server with an unencrypted volume from an encrypted image the log file remains empty: |
|
I uploaded the new spec to gerrit: https://review.opendev.org/c/openstack/glance-specs/+/915726 |
|
I reported the bug uncovered in #560 (comment) at https://bugs.launchpad.net/cinder/+bug/2061154 |
I raised this in the IRC, the following was the answer: |
|
We currently evaluate whether we will also need a spec for Cinder. What we definitely need is a blueprint for Cinder to track all development. I have written a blueprint and tried to include all possible points, where there will be implementation needed in Cinder: |
|
We got a review for the Spec (I am currently updating it) and there is one last question to discuss: We wanted to introduce a new container format "encrypted" - so it would be easily visible to anyone, that this image is encrypted. To identify what the underlying format is, we introduced a property: "os_decrypt_container_format". Now Nova and Cinder both would like to have the container format showing up in the original property and Cinder additionally would need some parameter to know whether the encrypted image is compressed or not. The thing here is, that we could check whether encrypted images are always qcow or raw when the container_format and decrypt_container_format is set. The metadata could be set after creating an image in the upload step. So it may be allowed to create an encrypted image with a format neither cinder nor nova could use. We would like to avoid such bad user experience. So we want to ask the following questions in the Glance meeting this thursday:
|
|
From the Cinder meeting today I got the wish to also create a small Cinder spec: https://etherpad.opendev.org/p/cinder-dalmatian-meetings |
|
I created the Cidner spec today: https://review.opendev.org/c/openstack/cinder-specs/+/919499 |
|
We got reviews on the spec, and Markus and I are going through them and answering questions. |
|
I adjusted the Glance spec and removed the container_format 'encrypted', because this was one of the most discussed parts of the spec: https://review.opendev.org/c/openstack/glance-specs/+/915726 |
|
I got feedback on the Glance patch with the question about what to do when the image conversion plugin is activated. Image conversion as I understand it, is not something that can be triggered by a CLI command. Instead when the plugin is activated ALL images that are created will be automatically converted to the ( through the config ) specified format after uploading and before storing it. This creates a few questions regarding encrypted images:
As this optional feature of Glance may already have problems with 1, and we would need to at least forbid uploading encrypted images when the target format is vmdk (maybe also when the target format is qcow2) this would be a lot more of implementation work. So we will for now render this out of scope for the spec. Maybe this can be done after the image encryption is in place and if it is needed by operators and users. |
Seems like this is part of the "Interoperable Image Import"1. Which references the following methods2:
This makes me wonder if this is applicable to images uploaded to Glance by Nova or Cinder at all. This is limited to images from external sources I think. Nonetheless, at the latest when the user is initiating such an image upload we need to make sure that we account for this concerning the image encryption. Footnotes |
|
I am looking into the Cinder Side a little bit more because, we got a comment from Dan Smith regarding image conversion and Glance checking a few Image parameters: The image conversion does only seem to be triggered when a user is uploading an image, not if Cinder or Nova upload one. Forbidding that would put more responsibility on the user side, but we also do that with the image encryption. The check for the virtual size can be omitted, because we only allow 2 types of encrypted images: qcow2 and raw and we want to introduce the "os_decrypt_size" parameter that should discribe the size of the unencrypted image. We may even mandate using this parameter. The format for encrypted raw images is only checkable, when decrypting the image. So while this is a valid point to discuss, as Glance will reject images, that do not have the format, they say they have, the Glance team did not wanted to have the power to encrypt or decrypt images back when we discussed gpg-based image encryption. I doubt that this has changed and i also do not see a good reason, Glance should be able to do this. |
|
I edited the glance spec to forbid image conversion in a more explicit way and included some more comments on the glance spec. |
|
I adjusted the key-managing part of the spec to clarify the behavior of deleting keys and added a part to put images into ERROR state, when they are encrypted and image conversion is enabled and need to be done. |
|
I attended the Glance meeting and asked for more reviews to the spec, so it can be merged. |
|
I adjusted the glance spec as there were a few open questions: https://review.opendev.org/c/openstack/glance-specs/+/915726 |
|
There were a few more questions on a spec. One thing is, that with the standardization we would also OPTIONALLY allow users to set a parameter that advises glance to delete a key on image deletion. This might be something to discuss in a Glance meeting. I alsoe edited the work items and added some more for tests, documentation and a migration script for old cinder-specific parameters. |
|
I spoke with Markus, about all steps that need to be implemented or written (Glance documentation and security guide). We added some steps. Markus may start implementing, when I am on vacation. |
Patchset draftsI started implementing the corresponding patchsets. GlancePatchset draft for Glance (click to expand ...)diff --git a/doc/source/user/common-image-properties.rst b/doc/source/user/common-image-properties.rst
index 24a3fa035..d42ac672f 100644
--- a/doc/source/user/common-image-properties.rst
+++ b/doc/source/user/common-image-properties.rst
@@ -52,14 +52,32 @@ description
A brief human-readable string, suitable for display in a user interface,
describing the image.
-cinder_encryption_key_id
+os_encrypt_key_id
Identifier in the OpenStack Key Management Service for the encryption key for
- the Block Storage Service to use when mounting a volume created from this
- image.
+ the image. Image-consuming services may use this key to decrypt the image
+ data.
-cinder_encryption_key_deletion_policy
+os_encrypt_format
+ Identifies the general format or mechanism used to encrypt the image. In
+ conjunction with 'os_encrypt_cipher', this property determines which
+ mechanism for decryption is to be used by consuming services.
+
+os_encrypt_cipher
+ Identifies the cipher suite used to encrypt the image. In conjunction with
+ 'os_encrypt_format', this property determines which mechanism for decryption
+ is to be used by consuming services.
+
+os_decrypt_container_format
+ Specifies the resulting Image Container Format after the image is decrypted.
+
+os_decrypt_size
+ Specifies the resulting image size after decryption. This property can be
+ used by consuming services to ensure enough space is aviailable and allocated
+ before attempting to decrrypt the image.
+
+os_encrypt_key_deletion_policy
States the condition under which the Image Service will delete the object
- associated with the 'cinder_encryption_key_id' image property. If this
+ associated with the 'os_encrypt_key_id' image property. If this
property is missing, the Image Service will take no action.
This file is the default schema. An operator can modify
diff --git a/etc/metadefs/glance-common-image-props.json b/etc/metadefs/glance-common-image-props.json
index b2263a0ce..95811925f 100644
--- a/etc/metadefs/glance-common-image-props.json
+++ b/etc/metadefs/glance-common-image-props.json
@@ -48,19 +48,39 @@
"description": "A human-readable string describing this image.",
"type": "string"
},
- "cinder_encryption_key_id": {
- "title": "Cinder Encryption Key ID",
- "description": "Identifier in the OpenStack Key Management Service for the encryption key for the Block Storage Service to use when mounting a volume created from this image",
+ "os_encrypt_key_id": {
+ "title": "Image Encryption Key ID",
+ "description": "Identifier in the OpenStack Key Management Service for the encryption key for the image. Image-consuming services may use this key to decrypt the image data.",
"type": "string"
},
- "cinder_encryption_key_deletion_policy": {
- "title": "Cinder Encryption Key Deletion Policy",
- "description": "States the condition under which the Image Service will delete the object associated with the 'cinder_encryption_key_id' image property. If this property is missing, the Image Service will take no action",
+ "os_encrypt_key_deletion_policy": {
+ "title": "Image Encryption Key Deletion Policy",
+ "description": "States the condition under which the Image Service will delete the object associated with the 'os_encrypt_key_id' image property. If this property is missing, the Image Service will take no action.",
"type": "string",
"enum": [
"on_image_deletion",
"do_not_delete"
]
+ },
+ "os_encrypt_format": {
+ "title": "Image Encryption Format",
+ "description": "Identifies the general format or mechanism used to encrypt the image. In conjunction with 'os_encrypt_cipher', this property determines which mechanism for decryption is to be used by consuming services.",
+ "type": "string"
+ },
+ "os_encrypt_cipher": {
+ "title": "Image Encryption Cipher",
+ "description": "Identifies the cipher suite used to encrypt the image. In conjunction with 'os_encrypt_format', this property determines which mechanism for decryption is to be used by consuming services.",
+ "type": "string"
+ },
+ "os_decrypt_container_format": {
+ "title": "Image Container Format after Decryption",
+ "description": "Specifies the resulting Image Container Format after the image is decrypted.",
+ "type": "string"
+ },
+ "os_decrypt_size": {
+ "title": "Image Container Format after Decryption",
+ "description": "Specifies the resulting image size after decryption. This property can be used by consuming services to ensure enough space is aviailable and allocated before attempting to decrrypt the image.",
+ "type": "string"
}
}
}
diff --git a/glance/api/v2/images.py b/glance/api/v2/images.py
index 2b8d0421c..6d99ae646 100644
--- a/glance/api/v2/images.py
+++ b/glance/api/v2/images.py
@@ -88,11 +88,91 @@ class ImagesController(object):
self._key_manager = key_manager.API(CONF)
+ def _register_as_secret_consumer(self, context, image):
+ """Register image as secret consumer.
+
+ Adds an image as a secret consumer to the secret referenced by the
+ os_encrypt_key_id metadata property of the image.
+
+ Raises an exception if the secret consumer registration fails.
+ """
+ props = image.extra_properties
+ encryption_key_id = props.get('os_encrypt_key_id')
+
+ # kwargs as required by barbicanclient.secrets.add_consumer()
+ consumer_data = {
+ 'service': 'glance',
+ 'resource_type': 'image',
+ 'resource_id': image.image_id
+ }
+
+ try:
+ msg = ('Registering secret consumer for image %s at secret %s' %
+ (image.image_id, encryption_key_id))
+ LOG.debug(msg)
+ self._key_manager.add_consumer(context, encryption_key_id,
+ consumer_data)
+ except castellan_exception.Forbidden as e:
+ msg = ("Unable to register image as secret consumer: %s" %
+ e.message)
+ raise webob.exc.HTTPForbidden(explanation=msg)
+ except (castellan_exception.ManagedObjectNotFoundError) as e:
+ msg = ("Unable to register image as secret consumer: %s" %
+ e.message)
+ raise webob.exc.HTTPBadRequest(explanation=msg)
+ # ValueError is thrown by barbicanclient if secret UUID is malformed
+ except ValueError as e:
+ msg = ("Unable to register image as secret consumer: %s" %
+ str(e))
+ raise webob.exc.HTTPBadRequest(explanation=msg)
+ # consider unreachable/broken Key Manager fatal and abort
+ except castellan_exception.KeyManagerError as e:
+ msg = ("Unable to register image as secret consumer: %s" %
+ e.message)
+ raise webob.exc.HTTPServerError(explanation=msg)
+
+ return
+
+ def _validate_encryption_parameters(self, extra_properties):
+ all_keys = {'os_encrypt_format', 'os_encrypt_cipher',
+ 'os_encrypt_key_deletion_policy',
+ 'os_encrypt_key_id', 'os_decrypt_size',
+ 'os_decrypt_container_format'}
+
+ # if none of the encryption parameters were specified, this image does
+ # not count as encrypted
+ if not any(k in extra_properties for k in all_keys):
+ return
+
+ # The os_decrypt_* keys are optional during image creation since this
+ # information might not be available as early as image metadata
+ # creation but only later when data is ready to be pushed to the image.
+ # On the other hand, the os_encrypt_* parameters describe the image
+ # encryption intent and need to be initialized from the start.
+ mandatory_keys = {
+ k for k in all_keys if not k.startswith('os_decrypt')
+ }
+
+ # all mandatory keys must be present
+ for k in mandatory_keys:
+ if k not in extra_properties:
+ msg = ("Image encryption metadata was provided but "
+ "incomplete: the mandatory encryption parameter '%s' "
+ "is missing." % k)
+ raise webob.exc.HTTPBadRequest(explanation=msg)
+
@utils.mutating
def create(self, req, image, extra_properties, tags):
image_factory = self.gateway.get_image_factory(req.context)
image_repo = self.gateway.get_repo(req.context)
try:
+ self._validate_encryption_parameters(extra_properties)
+ is_encrypted = extra_properties.get('os_encrypt_key_id', False)
+ # If the image has an associated encryption key, default the
+ # visibility to private because secrets in the Key Manager are
+ # project-bound and cannot be shared across different projects.
+ if 'visibility' not in image and is_encrypted:
+ image['visibility'] = 'private'
if 'owner' not in image:
image['owner'] = req.context.project_id
@@ -102,6 +182,11 @@ class ImagesController(object):
ks_quota.enforce_image_count_total(req.context, req.context.owner)
image = image_factory.new_image(extra_properties=extra_properties,
tags=tags, **image)
+ # If the image is encrypted, register it as secret consumer.
+ if extra_properties.get('os_encrypt_key_id', False):
+ # Will raise an exception if consumer registration fails and
+ # aborts the creation process.
+ self._register_as_secret_consumer(req.context, image)
image_repo.add(image)
except (exception.DuplicateLocation,
exception.Invalid) as e:
@@ -325,6 +410,16 @@ class ImagesController(object):
try:
image = image_repo.get(image_id)
+ if any(
+ encrypt_param in image.extra_properties
+ for encrypt_param in [
+ "os_encrypt_key_id", "os_encrypt_format",
+ "os_encrypt_cipher", "os_encrypt_key_deletion_policy"
+ ]
+ ):
+ msg = _("Image with encryption properties cannot be target "
+ "for import")
+ raise exception.Conflict(msg)
if image.status == 'active' and import_method != "copy-image":
msg = _("Image with status active cannot be target for import")
raise exception.Conflict(msg)
@@ -614,6 +709,16 @@ class ImagesController(object):
api_pol = api_policy.ImageAPIPolicy(req.context, image,
self.policy)
+ # check for changes to os_encrypt_* image encryption properties
+ # beforehand and reject them; do not apply any partial updates
+ for change in changes:
+ path_root = change['path'][0]
+ if path_root.startswith("os_encrypt_"):
+ LOG.debug("Rejecting change to %s", path_root)
+ raise webob.exc.HTTPBadRequest(
+ "Changes to os_encrypt properties are "
+ "not permitted after image creation")
+
for change in changes:
change_method_name = '_do_%s' % change['op']
change_method = getattr(self, change_method_name)
@@ -706,30 +811,73 @@ class ImagesController(object):
msg = _("Property %s does not exist.")
raise webob.exc.HTTPConflict(msg % path_root)
+ def _unregister_as_secret_consumer(self, context, image):
+ """Unregister image as secret consumer.
+
+ Removes the image from the secret consumers of the secret referenced
+ by the os_encrypt_key_id metadata property of the image.
+
+ Errors are logged but do not raise an exception.
+ """
+ props = image.extra_properties
+
+ encryption_key_id = props.get('os_encrypt_key_id', None)
+ if encryption_key_id is None:
+ return
+
+ # kwargs as required by barbicanclient.secrets.remove_consumer()
+ consumer_data = {
+ 'service': 'glance',
+ 'resource_type': 'image',
+ 'resource_id': image.image_id
+ }
+
+ try:
+ msg = ('Unregistering secret consumer for image %s from secret '
+ '%s' % (image.image_id, encryption_key_id))
+ LOG.debug(msg)
+ self._key_manager.remove_consumer(context, encryption_key_id,
+ consumer_data)
+ except castellan_exception.Forbidden:
+ msg = ('Not allowed to unregister consumer for encryption key %s' %
+ encryption_key_id)
+ LOG.warning(msg)
+ except (castellan_exception.ManagedObjectNotFoundError, KeyError):
+ msg = 'Could not find consumer registration or encryption key ' \
+ 'for key id %s' % encryption_key_id
+ LOG.warning(msg)
+ except ValueError as e:
+ msg = ('Failed to unregister as consumer for encryption key %s '
+ '("%s")' % (encryption_key_id, str(e)))
+ LOG.warning(msg)
+ except castellan_exception.KeyManagerError as e:
+ msg = ('Failed to unregister as consumer for encryption key %s '
+ '("%s")' % (encryption_key_id, str(e)))
+ LOG.warning(msg)
+
def _delete_encryption_key(self, context, image):
props = image.extra_properties
- cinder_encryption_key_id = props.get('cinder_encryption_key_id')
- if cinder_encryption_key_id is None:
+ encryption_key_id = props.get('os_encrypt_key_id')
+ if encryption_key_id is None:
return
- deletion_policy = props.get('cinder_encryption_key_deletion_policy',
- '')
+ deletion_policy = props.get('os_encrypt_key_deletion_policy', '')
if deletion_policy != 'on_image_deletion':
return
try:
- self._key_manager.delete(context, cinder_encryption_key_id)
+ self._key_manager.delete(context, encryption_key_id)
except castellan_exception.Forbidden:
- msg = ('Not allowed to delete encryption key %s' %
- cinder_encryption_key_id)
+ msg = ('Not allowed to delete image encryption key %s' %
+ encryption_key_id)
LOG.warning(msg)
except (castellan_exception.ManagedObjectNotFoundError, KeyError):
- msg = 'Could not find encryption key %s' % cinder_encryption_key_id
+ msg = 'Could not find image encryption key %s' % encryption_key_id
LOG.warning(msg)
except castellan_exception.KeyManagerError:
- msg = ('Failed to delete cinder encryption key %s' %
- cinder_encryption_key_id)
+ msg = ('Failed to delete image encryption key %s' %
+ encryption_key_id)
LOG.warning(msg)
@utils.mutating
@@ -904,6 +1052,7 @@ class ImagesController(object):
"it cannot be found at %(fn)s"), {'fn': file_path})
image.delete()
+ self._unregister_as_secret_consumer(req.context, image)
self._delete_encryption_key(req.context, image)
image_repo.remove(image)
except (glance_store.Forbidden, exception.Forbidden) as e:
diff --git a/glance/tests/unit/v2/test_images_resource.py b/glance/tests/unit/v2/test_images_resource.py
index 0824ea6a2..5eb47f5e1 100644
--- a/glance/tests/unit/v2/test_images_resource.py
+++ b/glance/tests/unit/v2/test_images_resource.py
@@ -73,6 +73,9 @@ FAKEHASHALGO = 'fake-name-for-sha512'
MULTIHASH1 = hashlib.sha512(b'glance').hexdigest()
MULTIHASH2 = hashlib.sha512(b'image_service').hexdigest()
+ENCRYPT_FORMAT = 'LUKS'
+ENCRYPT_CIPHER = 'aes256-xts-plain64'
+
TASK_ID_1 = 'b3006bd0-461e-4228-88ea-431c14e918b4'
TASK_ID_2 = '07b6b562-6770-4c8b-a649-37a515144ce9'
@@ -1324,6 +1327,105 @@ class TestImagesController(base.IsolatedUnitTest):
self.controller.create, request, image=image,
extra_properties=image_properties, tags=[])
+ def test_create_encrypted_registers_secret_consumer(self):
+ request = unit_test_utils.get_fake_request()
+ fake_secret_object = mock.Mock(consumers=[])
+ fake_encryption_key = self.controller._key_manager.store(
+ request.context, fake_secret_object)
+ extra_props = {
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ created_image = self.controller.create(
+ request, image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+ # Make sure that the secret consumer entry is registered correctly
+ self.assertEqual(1, len(fake_secret_object.consumers))
+
+ def test_create_no_encryption_key_id(self):
+ request = unit_test_utils.get_fake_request()
+ extra_props = {
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ self.assertRaisesRegex(
+ webob.exc.HTTPBadRequest,
+ ".*'os_encrypt_key_id' is missing.",
+ self.controller.create,
+ request,
+ image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+
+ def test_create_no_encryption_key_deletion_policy(self):
+ request = unit_test_utils.get_fake_request()
+ extra_props = {
+ 'os_encrypt_key_id': 'some-id',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ self.assertRaisesRegex(
+ webob.exc.HTTPBadRequest,
+ ".*'os_encrypt_key_deletion_policy' is missing.",
+ self.controller.create,
+ request,
+ image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+
+ def test_create_no_encryption_format(self):
+ request = unit_test_utils.get_fake_request()
+ extra_props = {
+ 'os_encrypt_key_id': 'some-id',
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ self.assertRaisesRegex(
+ webob.exc.HTTPBadRequest,
+ ".*'os_encrypt_format' is missing.",
+ self.controller.create,
+ request,
+ image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+
+ def test_create_no_encryption_cipher(self):
+ request = unit_test_utils.get_fake_request()
+ extra_props = {
+ 'os_encrypt_key_id': 'some-id',
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ }
+ self.assertRaisesRegex(
+ webob.exc.HTTPBadRequest,
+ ".*'os_encrypt_cipher' is missing.",
+ self.controller.create,
+ request,
+ image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+
+ def test_create_invalid_encryption_key_id(self):
+ request = unit_test_utils.get_fake_request()
+ extra_props = {
+ 'os_encrypt_key_id': 'invalid',
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ self.assertRaisesRegex(
+ webob.exc.HTTPBadRequest,
+ 'Unable to register image as secret consumer: Key not found.*',
+ self.controller.create,
+ request,
+ image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+
def test_update_no_changes(self):
request = unit_test_utils.get_fake_request()
output = self.controller.update(request, UUID1, changes=[])
@@ -3479,6 +3581,27 @@ class TestImagesController(base.IsolatedUnitTest):
# a task
mock_new_task.assert_not_called()
+ @mock.patch.object(glance.domain.TaskFactory, 'new_task')
+ @mock.patch.object(glance.notifier.ImageRepoProxy, 'get')
+ def test_image_import_rejects_encryption(self, mock_get, mock_new_task):
+ request = unit_test_utils.get_fake_request()
+ mock_get.return_value = FakeImage(status='uploading')
+ mock_get.return_value.extra_properties = {
+ 'os_encrypt_key_id': UUID1,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ for import_method in [
+ "glance-direct", "web-download", "glance-download", "copy-image"
+ ]:
+ self.assertRaises(webob.exc.HTTPConflict,
+ self.controller.import_image,
+ request, UUID4, {'method': {'name':
+ import_method}})
+ # Make sure we failed early and never even created a task
+ mock_new_task.assert_not_called()
+
@mock.patch.object(glance.notifier.ImageRepoProxy, 'get')
@mock.patch('glance.quota.keystone.enforce_image_size_total')
def test_image_import_quota_fail(self, mock_enforce, mock_get):
@@ -3761,7 +3884,7 @@ class TestImagesController(base.IsolatedUnitTest):
fake_encryption_key = self.controller._key_manager.store(
request.context, mock.Mock())
props = {
- 'cinder_encryption_key_id': fake_encryption_key,
+ 'os_encrypt_key_id': fake_encryption_key,
}
image = _domain_fixture(
UUID2, name='image-2', owner=TENANT2,
@@ -3780,8 +3903,10 @@ class TestImagesController(base.IsolatedUnitTest):
fake_encryption_key = self.controller._key_manager.store(
request.context, mock.Mock())
props = {
- 'cinder_encryption_key_id': fake_encryption_key,
- 'cinder_encryption_key_deletion_policy': 'do_not_delete',
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'do_not_delete',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
}
image = _domain_fixture(
UUID2, name='image-2', owner=TENANT2,
@@ -3800,8 +3925,10 @@ class TestImagesController(base.IsolatedUnitTest):
fake_encryption_key = self.controller._key_manager.store(
request.context, mock.Mock())
props = {
- 'cinder_encryption_key_id': fake_encryption_key,
- 'cinder_encryption_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
}
image = _domain_fixture(
UUID2, name='image-2', owner=TENANT2,
@@ -3822,8 +3949,10 @@ class TestImagesController(base.IsolatedUnitTest):
fake_encryption_key = self.controller._key_manager.store(
request.context, mock.Mock())
props = {
- 'cinder_encryption_key_id': fake_encryption_key,
- 'cinder_encryption_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
}
image = _domain_fixture(
UUID2, name='image-2', owner=TENANT2,
@@ -3844,8 +3973,10 @@ class TestImagesController(base.IsolatedUnitTest):
fake_encryption_key = self.controller._key_manager.store(
request.context, mock.Mock())
props = {
- 'cinder_encryption_key_id': fake_encryption_key,
- 'cinder_encryption_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
}
image = _domain_fixture(
UUID2, name='image-2', owner=TENANT2,
@@ -3861,13 +3992,33 @@ class TestImagesController(base.IsolatedUnitTest):
fake_encryption_key)
self.assertEqual(fake_encryption_key, key._id)
+ def test_delete_encrypted_unregisters_secret_consumer(self):
+ request = unit_test_utils.get_fake_request()
+ fake_secret_object = mock.Mock(consumers=[])
+ fake_encryption_key = self.controller._key_manager.store(
+ request.context, fake_secret_object)
+ extra_props = {
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ created_image = self.controller.create(
+ request, image={'name': 'image-1'},
+ extra_properties=extra_props,
+ tags=[])
+ image_id = created_image.image_id
+ self.controller.delete(request, image_id)
+ # Make sure that the secret consumer entry is unregistered
+ self.assertEqual(0, len(fake_secret_object.consumers))
+
def test_delete_encryption_key(self):
request = unit_test_utils.get_fake_request()
fake_encryption_key = self.controller._key_manager.store(
request.context, mock.Mock())
props = {
- 'cinder_encryption_key_id': fake_encryption_key,
- 'cinder_encryption_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
}
image = _domain_fixture(
UUID2, name='image-2', owner=TENANT2,
@@ -3881,45 +4032,16 @@ class TestImagesController(base.IsolatedUnitTest):
self.controller._key_manager.get,
request.context, fake_encryption_key)
- def test_delete_no_encryption_key_id(self):
- request = unit_test_utils.get_fake_request()
- extra_props = {
- 'cinder_encryption_key_deletion_policy': 'on_image_deletion',
- }
- created_image = self.controller.create(request,
- image={'name': 'image-1'},
- extra_properties=extra_props,
- tags=[])
- image_id = created_image.image_id
- self.controller.delete(request, image_id)
- # Ensure that image is deleted
- image = self.db.image_get(request.context, image_id,
- force_show_deleted=True)
- self.assertTrue(image['deleted'])
- self.assertEqual('deleted', image['status'])
-
- def test_delete_invalid_encryption_key_id(self):
- request = unit_test_utils.get_fake_request()
- extra_props = {
- 'cinder_encryption_key_id': 'invalid',
- 'cinder_encryption_key_deletion_policy': 'on_image_deletion',
- }
- created_image = self.controller.create(request,
- image={'name': 'image-1'},
- extra_properties=extra_props,
- tags=[])
- image_id = created_image.image_id
- self.controller.delete(request, image_id)
- # Ensure that image is deleted
- image = self.db.image_get(request.context, image_id,
- force_show_deleted=True)
- self.assertTrue(image['deleted'])
- self.assertEqual('deleted', image['status'])
-
def test_delete_invalid_encryption_key_deletion_policy(self):
request = unit_test_utils.get_fake_request()
+ fake_secret_object = mock.Mock(consumers=[])
+ fake_encryption_key = self.controller._key_manager.store(
+ request.context, fake_secret_object)
extra_props = {
- 'cinder_encryption_key_deletion_policy': 'invalid',
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ 'os_encrypt_key_deletion_policy': 'invalid',
}
created_image = self.controller.create(request,
image={'name': 'image-1'},
@@ -3932,6 +4054,8 @@ class TestImagesController(base.IsolatedUnitTest):
force_show_deleted=True)
self.assertTrue(image['deleted'])
self.assertEqual('deleted', image['status'])
+ # Ensure that the secret consumer was removed
+ self.assertEqual(0, len(fake_secret_object.consumers))
@mock.patch.object(glance.notifier.TaskFactoryProxy, 'new_task')
def test_add_location(self, mock_task):
@@ -4312,6 +4436,56 @@ class TestImagesController(base.IsolatedUnitTest):
'No image found with ID .*',
self.controller.get_locations,
request, str(uuid.uuid4()))
+
+ def test_register_secret_consumer(self):
+ request = unit_test_utils.get_fake_request()
+ fake_secret_object = mock.Mock(consumers=[])
+ fake_encryption_key = self.controller._key_manager.store(
+ request.context, fake_secret_object)
+ props = {
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ image = _domain_fixture(
+ UUID2, name='image-2', owner=TENANT2,
+ checksum='ca425b88f047ce8ec45ee90e813ada91',
+ os_hash_algo=FAKEHASHALGO, os_hash_value=MULTIHASH1,
+ created_at=DATETIME, updated_at=DATETIME, size=1024,
+ virtual_size=3072, extra_properties=props)
+ self.controller._register_as_secret_consumer(request.context, image)
+ self.assertEqual(1, len(fake_secret_object.consumers))
+ self.assertEqual({
+ 'service': 'glance',
+ 'resource_type': 'image',
+ 'resource_id': UUID2
+ },
+ fake_secret_object.consumers[0])
+
+ def test_unregister_secret_consumer(self):
+ request = unit_test_utils.get_fake_request()
+ fake_secret_object = mock.Mock(consumers=[{
+ 'service': 'glance',
+ 'resource_type': 'image',
+ 'resource_id': UUID2
+ }])
+ fake_encryption_key = self.controller._key_manager.store(
+ request.context, fake_secret_object)
+ props = {
+ 'os_encrypt_key_id': fake_encryption_key,
+ 'os_encrypt_key_deletion_policy': 'on_image_deletion',
+ 'os_encrypt_format': ENCRYPT_FORMAT,
+ 'os_encrypt_cipher': ENCRYPT_CIPHER,
+ }
+ image = _domain_fixture(
+ UUID2, name='image-2', owner=TENANT2,
+ checksum='ca425b88f047ce8ec45ee90e813ada91',
+ os_hash_algo=FAKEHASHALGO, os_hash_value=MULTIHASH1,
+ created_at=DATETIME, updated_at=DATETIME, size=1024,
+ virtual_size=3072, extra_properties=props)
+ self.controller._unregister_as_secret_consumer(request.context, image)
+ self.assertEqual(0, len(fake_secret_object.consumers))
class TestImagesControllerPolicies(base.IsolatedUnitTest):
TODO:
CinderPatchset draft for Cinder (click to expand ...)diff --git a/cinder/api/contrib/volume_actions.py b/cinder/api/contrib/volume_actions.py
index 8b80a5edc..5f340d5c0 100644
--- a/cinder/api/contrib/volume_actions.py
+++ b/cinder/api/contrib/volume_actions.py
@@ -24,6 +24,7 @@ from cinder.api import microversions as mv
from cinder.api.openstack import wsgi
from cinder.api.schemas import volume_actions as volume_action
from cinder.api import validation
+from cinder import db
from cinder import exception
from cinder.i18n import _
from cinder.policies import volume_actions as policy
@@ -48,6 +49,17 @@ class VolumeActionsController(wsgi.Controller):
return self._key_mgr
+ def _get_volume_type_encryption_format_and_cipher(self, context, type_id):
+ encryption_ref = db.volume_type_encryption_get(context, type_id)
+ encryption_format = 'luks'
+ encryption_cipher = 'cinder.default'
+ if encryption_ref:
+ if 'provider' in encryption_ref:
+ encryption_format = encryption_ref['provider']
+ if 'cipher' in encryption_ref:
+ encryption_cipher = encryption_ref['cipher']
+ return encryption_format, encryption_cipher
+
@wsgi.response(HTTPStatus.ACCEPTED)
@wsgi.action('os-attach')
@validation.schema(volume_action.attach)
@@ -231,9 +243,17 @@ class VolumeActionsController(wsgi.Controller):
encryption_key_id = volume_utils.clone_encryption_key(
context, self._key_manager, volume.encryption_key_id)
- image_metadata['cinder_encryption_key_id'] = encryption_key_id
- image_metadata['cinder_encryption_key_deletion_policy'] = \
- 'on_image_deletion'
+ # Attach appropriate encryption specification metadata to the
+ # image based on the volume type encryption properties.
+ encryption_format, encryption_cipher = self.\
+ _get_volume_type_encryption_format_and_cipher(
+ context, volume.volume_type.id)
+
+ image_metadata['os_encrypt_key_id'] = encryption_key_id
+ image_metadata['os_encrypt_format'] = encryption_format
+ image_metadata['os_encrypt_cipher'] = encryption_cipher
+ image_metadata[
+ 'os_encrypt_key_deletion_policy'] = 'on_image_deletion'
if req_version >= mv.get_api_version(
mv.UPLOAD_IMAGE_PARAMS):
diff --git a/cinder/image/glance.py b/cinder/image/glance.py
index 9002afb09..5b59814d6 100644
--- a/cinder/image/glance.py
+++ b/cinder/image/glance.py
@@ -518,10 +518,10 @@ class GlanceImageService(object):
if self._image_schema.is_base_property(key) is True and
key != 'schema'}
- # Process 'cinder_encryption_key_id' as a metadata key
- if 'cinder_encryption_key_id' in image.keys():
- image_meta['cinder_encryption_key_id'] = \
- image['cinder_encryption_key_id']
+ # Process 'os_encrypt_key_id' as a metadata key
+ if 'os_encrypt_key_id' in image.keys():
+ image_meta['os_encrypt_key_id'] = \
+ image['os_encrypt_key_id']
# NOTE(aarefiev): nova is expected that all image properties
# (custom or defined in schema-image.json) stores in
diff --git a/cinder/image/image_utils.py b/cinder/image/image_utils.py
index 65141ee1d..6088e9674 100644
--- a/cinder/image/image_utils.py
+++ b/cinder/image/image_utils.py
@@ -32,6 +32,7 @@ import re
import tempfile
from typing import ContextManager, Generator, Optional
+from castellan import key_manager as castellan_key_manager
import cryptography
from cursive import exception as cursive_exception
from cursive import signature_utils
@@ -1064,13 +1065,38 @@ def fetch_to_volume_format(context: context.RequestContext,
disk_format = fixup_disk_format(image_meta['disk_format'])
LOG.debug("%s was %s, converting to %s", image_id, fmt, volume_format)
- convert_image(tmp, dest, volume_format,
- out_subformat=volume_subformat,
- src_format=disk_format,
- run_as_root=run_as_root,
- image_id=image_id,
- data=data,
- disable_sparse=disable_sparse)
+ image_encryption_key_id = image_meta.get('os_encrypt_key_id', None)
+ LOG.debug("Image is encrypted with secret %s", image_encryption_key_id)
+
+ if image_encryption_key_id and disk_format == "qcow2":
+ LOG.debug("Image is qcow2+LUKS")
+ keymgr = castellan_key_manager.API(CONF)
+ passphrase = volume_utils.get_encryption_key_as_passphrase(
+ context, keymgr, image_encryption_key_id)
+ # NOTE(mhen): Due to the source image carrying an os_encrypt_key_id
+ # the key has already been cloned in the key manager for the target
+ # volume during the ExtractVolumeRequestTask flow. That's why the
+ # key file is passed both as source and target passphrase here.
+ with temporary_file(prefix='luks_') as src_pass_file:
+ with open(src_pass_file, 'w') as f:
+ f.write(passphrase)
+ convert_image(tmp, dest, volume_format,
+ out_subformat=volume_subformat,
+ src_format=disk_format,
+ run_as_root=run_as_root,
+ image_id=image_id,
+ data=data,
+ disable_sparse=disable_sparse,
+ src_passphrase_file=src_pass_file,
+ passphrase_file=src_pass_file)
+ else:
+ convert_image(tmp, dest, volume_format,
+ out_subformat=volume_subformat,
+ src_format=disk_format,
+ run_as_root=run_as_root,
+ image_id=image_id,
+ data=data,
+ disable_sparse=disable_sparse)
@contextlib.contextmanager
diff --git a/cinder/tests/unit/api/contrib/test_volume_actions.py b/cinder/tests/unit/api/contrib/test_volume_actions.py
index 90a933eab..32b9b7cd9 100644
--- a/cinder/tests/unit/api/contrib/test_volume_actions.py
+++ b/cinder/tests/unit/api/contrib/test_volume_actions.py
@@ -909,9 +909,12 @@ class VolumeImageActionsTest(test.TestCase):
body=body)
@mock.patch.object(volume_api.API, 'get', fake_volume_get_obj)
+ @mock.patch.object(volume_actions.VolumeActionsController,
+ '_get_volume_type_encryption_format_and_cipher')
@mock.patch.object(volume_api.API, "copy_volume_to_image")
def test_check_image_metadata_copy_encrypted_volume_to_image(
- self, mock_copy_vol):
+ self, mock_copy_vol,
+ mock_get_format_and_cipher):
"""Make sure the encryption image properties exit the controller."""
# all we're interested in is that the 'metadata' dict contains the
@@ -921,6 +924,8 @@ class VolumeImageActionsTest(test.TestCase):
return metadata
mock_copy_vol.side_effect = really_fake_upload_volume
+ mock_get_format_and_cipher.return_value = (
+ mock.sentinel.encrypt_format, mock.sentinel.encrypt_cipher)
FAKE_ID = 'fake-encryption-key-id'
# the controller does a lazy init of the key manager, so we
@@ -941,11 +946,17 @@ class VolumeImageActionsTest(test.TestCase):
res_dict = self.controller._volume_upload_image(req, vol_id, body=body)
sent_meta = res_dict['os-volume_upload_image']
- self.assertIn('cinder_encryption_key_id', sent_meta)
- self.assertEqual(FAKE_ID, sent_meta['cinder_encryption_key_id'])
- self.assertIn('cinder_encryption_key_deletion_policy', sent_meta)
+ self.assertIn('os_encrypt_key_id', sent_meta)
+ self.assertEqual(FAKE_ID, sent_meta['os_encrypt_key_id'])
+ self.assertIn('os_encrypt_format', sent_meta)
+ self.assertEqual(mock.sentinel.encrypt_format,
+ sent_meta['os_encrypt_format'])
+ self.assertIn('os_encrypt_cipher', sent_meta)
+ self.assertEqual(mock.sentinel.encrypt_cipher,
+ sent_meta['os_encrypt_cipher'])
+ self.assertIn('os_encrypt_key_deletion_policy', sent_meta)
self.assertEqual('on_image_deletion',
- sent_meta['cinder_encryption_key_deletion_policy'])
+ sent_meta['os_encrypt_key_deletion_policy'])
@mock.patch.object(volume_api.API, 'get', fake_volume_get_obj)
@mock.patch.object(volume_api.API, "copy_volume_to_image")
@@ -969,8 +980,10 @@ class VolumeImageActionsTest(test.TestCase):
res_dict = self.controller._volume_upload_image(req, id, body=body)
sent_meta = res_dict['os-volume_upload_image']
- self.assertNotIn('cinder_encryption_key_id', sent_meta)
- self.assertNotIn('cinder_encryption_key_deletion_policy', sent_meta)
+ self.assertNotIn('os_encrypt_key_id', sent_meta)
+ self.assertNotIn('os_encrypt_key_format', sent_meta)
+ self.assertNotIn('os_encrypt_key_cipher', sent_meta)
+ self.assertNotIn('os_encrypt_key_deletion_policy', sent_meta)
def test_copy_volume_to_image_volumenotfound(self):
def fake_volume_get_raise_exc(self, context, volume_id):
diff --git a/cinder/tests/unit/image/test_glance.py b/cinder/tests/unit/image/test_glance.py
index b04d59b54..bd361022a 100644
--- a/cinder/tests/unit/image/test_glance.py
+++ b/cinder/tests/unit/image/test_glance.py
@@ -990,7 +990,7 @@ class TestGlanceImageService(test.TestCase):
'deleted', 'deleted_at', 'checksum',
'min_disk', 'min_ram', 'protected',
'visibility',
- 'cinder_encryption_key_id')
+ 'os_encrypt_key_id')
output = {}
@@ -1042,7 +1042,7 @@ class TestGlanceImageService(test.TestCase):
'properties': {},
'owner': None,
'visibility': None,
- 'cinder_encryption_key_id': None
+ 'os_encrypt_key_id': None
}
self.assertEqual(expected, actual)
diff --git a/cinder/tests/unit/test_volume_utils.py b/cinder/tests/unit/test_volume_utils.py
index a78cfee73..7e84d0509 100644
--- a/cinder/tests/unit/test_volume_utils.py
+++ b/cinder/tests/unit/test_volume_utils.py
@@ -16,6 +16,7 @@
"""Tests For miscellaneous util methods used with volume."""
+import binascii
import datetime
import functools
import io
@@ -36,6 +37,7 @@ from cinder.objects import fields
from cinder.tests.unit.backup import fake_backup
from cinder.tests.unit import fake_constants as fake
from cinder.tests.unit import fake_group
+from cinder.tests.unit.keymgr import fake as fake_keymgr
from cinder.tests.unit import fake_snapshot
from cinder.tests.unit import fake_volume
from cinder.tests.unit.image import fake as fake_image
@@ -696,6 +698,16 @@ class CopyVolumeTestCase(test.TestCase):
@ddt.ddt
class VolumeUtilsTestCase(test.TestCase):
+
+ class KeyObject(object):
+ def __init__(self, managed_type, raw):
+ self.secret_type = managed_type
+ self.secret = raw
+ def get_encoded(self):
+ return self.secret
+ def managed_type(self):
+ return self.secret_type
+
def test_null_safe_str(self):
self.assertEqual('', volume_utils.null_safe_str(None))
self.assertEqual('', volume_utils.null_safe_str(False))
@@ -1043,6 +1055,36 @@ class VolumeUtilsTestCase(test.TestCase):
fake.VOLUME_TYPE_ID)
create_key.assert_not_called()
+ def test_get_encryption_key_as_passphrase_from_passphrase(self):
+ ctxt = context.get_admin_context()
+ key_mgr = fake_keymgr.fake_api()
+ self.mock_object(key_manager, 'API', return_value=key_mgr)
+
+ key_raw = b'binarykeypayload'
+ key_hex = binascii.hexlify(key_raw).decode('utf-8')
+ key_id = key_mgr.store(ctxt, self.KeyObject('passphrase', key_raw))
+ passphrase = volume_utils.get_encryption_key_as_passphrase(ctxt,
+ key_mgr,
+ key_id)
+ # secret type 'passphrase' skips hexlify and is used as-is
+ self.assertNotEqual(key_hex, passphrase)
+ self.assertEqual(key_raw.decode('utf-8'), passphrase)
+
+ def test_get_encryption_key_as_passphrase_from_symmetric(self):
+ ctxt = context.get_admin_context()
+ key_mgr = fake_keymgr.fake_api()
+ self.mock_object(key_manager, 'API', return_value=key_mgr)
+
+ key_raw = b'binarykeypayload'
+ key_hex = binascii.hexlify(key_raw).decode('utf-8')
+ key_id = key_mgr.store(ctxt, self.KeyObject('symmetric', key_raw))
+ passphrase = volume_utils.get_encryption_key_as_passphrase(ctxt,
+ key_mgr,
+ key_id)
+ # secret type 'symmetric' implies hexlify conversion
+ self.assertEqual(key_hex, passphrase)
+ self.assertNotEqual(key_raw.decode('utf-8'), passphrase)
+
@ddt.data('<is> True', '<is> true', '<is> yes')
def test_is_replicated_spec_true(self, enabled):
res = volume_utils.is_replicated_spec({'replication_enabled': enabled})
diff --git a/cinder/tests/unit/volume/drivers/test_nfs.py b/cinder/tests/unit/volume/drivers/test_nfs.py
index ab9b46317..0fcf8e676 100644
--- a/cinder/tests/unit/volume/drivers/test_nfs.py
+++ b/cinder/tests/unit/volume/drivers/test_nfs.py
@@ -40,6 +40,8 @@ from cinder.volume import volume_utils
class KeyObject(object):
def get_encoded(arg):
return "asdf".encode('utf-8')
+ def managed_type(self):
+ return "symmetric"
class RemoteFsDriverTestCase(test.TestCase):
diff --git a/cinder/tests/unit/volume/drivers/test_rbd.py b/cinder/tests/unit/volume/drivers/test_rbd.py
index 9952f8a72..d60e23f90 100644
--- a/cinder/tests/unit/volume/drivers/test_rbd.py
+++ b/cinder/tests/unit/volume/drivers/test_rbd.py
@@ -92,6 +92,8 @@ class MockInvalidArgument(MockException):
class KeyObject(object):
def get_encoded(arg):
return "asdf".encode('utf-8')
+ def managed_type(self):
+ return "symmetric"
def common_mocks(f):
diff --git a/cinder/tests/unit/volume/drivers/test_remotefs.py b/cinder/tests/unit/volume/drivers/test_remotefs.py
index a8c69398c..c5ce3e408 100644
--- a/cinder/tests/unit/volume/drivers/test_remotefs.py
+++ b/cinder/tests/unit/volume/drivers/test_remotefs.py
@@ -39,6 +39,8 @@ from cinder.volume import volume_utils
class KeyObject(object):
def get_encoded(arg):
return "asdf".encode('utf-8')
+ def managed_type(self):
+ return "symmetric"
@ddt.ddt
diff --git a/cinder/tests/unit/volume/flows/test_create_volume_flow.py b/cinder/tests/unit/volume/flows/test_create_volume_flow.py
index 05fc0579f..21f803667 100644
--- a/cinder/tests/unit/volume/flows/test_create_volume_flow.py
+++ b/cinder/tests/unit/volume/flows/test_create_volume_flow.py
@@ -1214,7 +1214,59 @@ class CreateVolumeFlowManagerTestCase(test.TestCase):
image_meta['id'] = image_id
image_meta['status'] = 'active'
image_meta['size'] = 1
- image_meta['cinder_encryption_key_id'] = \
+ image_meta['os_encrypt_key_id'] = \
+ '00000000-0000-0000-0000-000000000000'
+ image_location = 'abc'
+
+ fake_db.volume_update.return_value = volume
+ fake_manager._create_from_image(self.ctxt, volume,
+ image_location, image_id,
+ image_meta, fake_image_service)
+
+ fake_driver.create_volume.assert_called_once_with(volume)
+ fake_driver.copy_image_to_encrypted_volume.assert_not_called()
+ fake_driver.copy_image_to_volume.assert_called_once_with(
+ self.ctxt, volume, fake_image_service, image_id,
+ disable_sparse=False)
+ mock_handle_bootable.assert_called_once_with(self.ctxt, volume,
+ image_id=image_id,
+ image_meta=image_meta)
+ mock_cleanup_cg.assert_called_once_with(volume)
+
+ @mock.patch('cinder.volume.flows.manager.create_volume.'
+ 'CreateVolumeFromSpecTask.'
+ '_cleanup_cg_in_volume')
+ @mock.patch('cinder.volume.flows.manager.create_volume.'
+ 'CreateVolumeFromSpecTask.'
+ '_handle_bootable_volume_glance_meta')
+ @mock.patch('cinder.image.image_utils.TemporaryImages.fetch')
+ @mock.patch('cinder.image.image_utils.qemu_img_info')
+ @mock.patch('cinder.image.image_utils.check_virtual_size')
+ def test_create_encrypted_volume_from_enc_qcow2_image(self,
+ mock_check_size,
+ mock_qemu_img,
+ mock_fetch_img,
+ mock_handle_bootable,
+ mock_cleanup_cg):
+ fake_db = mock.MagicMock()
+ fake_driver = mock.MagicMock()
+ fake_driver.capabilities = {}
+ fake_volume_manager = mock.MagicMock()
+ fake_manager = create_volume_manager.CreateVolumeFromSpecTask(
+ fake_volume_manager, fake_db, fake_driver)
+ volume = fake_volume.fake_volume_obj(
+ self.ctxt,
+ encryption_key_id=fakes.ENCRYPTION_KEY_ID,
+ host='host@backend#pool')
+
+ fake_image_service = fake_image.FakeImageService()
+ image_meta = {}
+ image_id = fakes.IMAGE_ID
+ image_meta['id'] = image_id
+ image_meta['status'] = 'active'
+ image_meta['size'] = 1
+ image_meta['disk_format'] = 'qcow2'
+ image_meta['os_encrypt_key_id'] = \
'00000000-0000-0000-0000-000000000000'
image_location = 'abc'
@@ -1525,7 +1577,7 @@ class CreateVolumeFlowManagerGlanceCinderBackendCase(test.TestCase):
'size': 1024,
'owner': owner or self.ctxt.project_id,
'virtual_size': None,
- 'cinder_encryption_key_id': None}
+ 'os_encrypt_key_id': None}
fake_driver.clone_image.return_value = (None, False)
fake_db.volume_get_all.return_value = []
@@ -1588,7 +1640,7 @@ class CreateVolumeFlowManagerGlanceCinderBackendCase(test.TestCase):
'size': 1024,
'owner': owner or self.ctxt.project_id,
'virtual_size': None,
- 'cinder_encryption_key_id': None}
+ 'os_encrypt_key_id': None}
fake_driver.clone_image.return_value = (None, False)
fake_db.volume_get_all.return_value = [image_volume]
@@ -1661,7 +1713,7 @@ class CreateVolumeFlowManagerGlanceCinderBackendCase(test.TestCase):
'size': 1024,
'owner': owner or self.ctxt.project_id,
'virtual_size': None,
- 'cinder_encryption_key_id': None}
+ 'os_encrypt_key_id': None}
fake_driver.clone_image.return_value = (None, False)
fake_db.volume_get_all_by_host.return_value = [image_volume]
diff --git a/cinder/tests/unit/volume/test_image.py b/cinder/tests/unit/volume/test_image.py
index 979a0e3be..e4eee9d0d 100644
--- a/cinder/tests/unit/volume/test_image.py
+++ b/cinder/tests/unit/volume/test_image.py
@@ -816,13 +816,13 @@ class CopyVolumeToImagePrivateFunctionsTestCase(
@mock.patch('cinder.volume.api.API.get_volume_image_metadata',
return_value={'some_key': 'some_value',
- 'cinder_encryption_key_id': 'stale_value'})
+ 'os_encrypt_key_id': 'stale_value'})
def test_merge_volume_image_meta(self, mock_get_img_meta):
# this is what we're passing to copy_volume_to_image
image_meta = {
'container_format': 'bare',
'disk_format': 'raw',
- 'cinder_encryption_key_id': 'correct_value'
+ 'os_encrypt_key_id': 'correct_value'
}
self.assertNotIn('properties', image_meta)
@@ -831,7 +831,7 @@ class CopyVolumeToImagePrivateFunctionsTestCase(
# we've got 'properties' now
self.assertIn('properties', image_meta)
# verify the key_id is what we expect
- self.assertEqual(image_meta['cinder_encryption_key_id'],
+ self.assertEqual(image_meta['os_encrypt_key_id'],
'correct_value')
translate = cinder.image.glance.GlanceImageService._translate_to_glance
@@ -842,5 +842,5 @@ class CopyVolumeToImagePrivateFunctionsTestCase(
# make sure the image would be created in Glance with the
# correct key_id
- self.assertEqual(image_meta['cinder_encryption_key_id'],
- sent_to_glance['cinder_encryption_key_id'])
+ self.assertEqual(image_meta['os_encrypt_key_id'],
+ sent_to_glance['os_encrypt_key_id'])
diff --git a/cinder/tests/unit/volume/test_volume.py b/cinder/tests/unit/volume/test_volume.py
index 00b476191..3dcda4518 100644
--- a/cinder/tests/unit/volume/test_volume.py
+++ b/cinder/tests/unit/volume/test_volume.py
@@ -93,11 +93,15 @@ def create_snapshot(volume_id, size=1, metadata=None, ctxt=None,
class KeyObject(object):
def get_encoded(self):
return "asdf".encode('utf-8')
+ def managed_type(self):
+ return "symmetric"
class KeyObject2(object):
def get_encoded(self):
return "qwert".encode('utf-8')
+ def managed_type(self):
+ return "symmetric"
@ddt.ddt
diff --git a/cinder/volume/drivers/nfs.py b/cinder/volume/drivers/nfs.py
index 95c33e0b7..910f5a94d 100644
--- a/cinder/volume/drivers/nfs.py
+++ b/cinder/volume/drivers/nfs.py
@@ -14,7 +14,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import binascii
import errno
import os
import tempfile
@@ -657,14 +656,12 @@ class NfsDriver(remotefs.RemoteFSSnapDriverDistributed):
# TODO(enriquetaso): handle unencrypted snap->encrypted vol
raise exception.NfsException(message)
keymgr = key_manager.API(CONF)
- new_key = keymgr.get(volume.obj_context, new_encryption_key_id)
- new_passphrase = \
- binascii.hexlify(new_key.get_encoded()).decode('utf-8')
+ new_passphrase = volume_utils.get_encryption_key_as_passphrase(
+ volume.obj_context, keymgr, new_encryption_key_id)
# volume.obj_context is the owner of this request
- src_key = keymgr.get(volume.obj_context, src_encryption_key_id)
- src_passphrase = \
- binascii.hexlify(src_key.get_encoded()).decode('utf-8')
+ src_passphrase = volume_utils.get_encryption_key_as_passphrase(
+ volume.obj_context, keymgr, src_encryption_key_id)
tmp_dir = volume_utils.image_conversion_dir()
with tempfile.NamedTemporaryFile(prefix='luks_',
diff --git a/cinder/volume/drivers/rbd.py b/cinder/volume/drivers/rbd.py
index 39db852d5..bcaff1fdf 100644
--- a/cinder/volume/drivers/rbd.py
+++ b/cinder/volume/drivers/rbd.py
@@ -14,7 +14,6 @@
# under the License.
"""RADOS Block Device Driver"""
-import binascii
import errno
import json
import math
@@ -1110,8 +1109,8 @@ class RBDDriver(driver.CloneableImageVD, driver.MigrateVD,
# Fetch the key associated with the volume and decode the passphrase
keymgr = key_manager.API(CONF)
- key = keymgr.get(context, encryption['encryption_key_id'])
- passphrase = binascii.hexlify(key.get_encoded()).decode('utf-8')
+ passphrase = volume_utils.get_encryption_key_as_passphrase(
+ context, keymgr, encryption['encryption_key_id'])
# create a file
tmp_dir = volume_utils.image_conversion_dir()
@@ -2005,8 +2004,8 @@ class RBDDriver(driver.CloneableImageVD, driver.MigrateVD,
# Fetch the key associated with the volume and decode the passphrase
keymgr = key_manager.API(CONF)
- key = keymgr.get(context, encryption['encryption_key_id'])
- passphrase = binascii.hexlify(key.get_encoded()).decode('utf-8')
+ passphrase = volume_utils.get_encryption_key_as_passphrase(
+ context, keymgr, encryption['encryption_key_id'])
# Decode the dm-crypt style cipher spec into something qemu-img can use
cipher_spec = image_utils.decode_cipher(encryption['cipher'],
diff --git a/cinder/volume/drivers/remotefs.py b/cinder/volume/drivers/remotefs.py
index 40d14b8fa..8f94cbcc7 100644
--- a/cinder/volume/drivers/remotefs.py
+++ b/cinder/volume/drivers/remotefs.py
@@ -14,7 +14,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import binascii
import collections
import errno
import inspect
@@ -454,8 +453,8 @@ class RemoteFSDriver(driver.BaseVD):
# TODO(enriquetaso): share this code w/ the RBD driver
# Fetch the key associated with the volume and decode the passphrase
keymgr = key_manager.API(CONF)
- key = keymgr.get(context, encryption['encryption_key_id'])
- passphrase = binascii.hexlify(key.get_encoded()).decode('utf-8')
+ passphrase = volume_utils.get_encryption_key_as_passphrase(
+ context, keymgr, encryption['encryption_key_id'])
# create a file
tmp_dir = volume_utils.image_conversion_dir()
@@ -1365,9 +1364,8 @@ class RemoteFSSnapDriverBase(RemoteFSDriver):
if encrypted:
keymgr = key_manager.API(CONF)
encryption_key = snapshot.encryption_key_id
- new_key = keymgr.get(snapshot.obj_context, encryption_key)
- src_passphrase = \
- binascii.hexlify(new_key.get_encoded()).decode('utf-8')
+ src_passphrase = volume_utils.get_encryption_key_as_passphrase(
+ snapshot.obj_context, keymgr, encryption_key)
tmp_dir = volume_utils.image_conversion_dir()
@@ -1524,9 +1522,8 @@ class RemoteFSSnapDriverBase(RemoteFSDriver):
# encrypted
keymgr = key_manager.API(CONF)
# Get key for the source volume using the context of this request.
- key = keymgr.get(obj_context,
- snapshot.volume.encryption_key_id)
- passphrase = binascii.hexlify(key.get_encoded()).decode('utf-8')
+ passphrase = volume_utils.get_encryption_key_as_passphrase(
+ obj_context, keymgr, snapshot.volume.encryption_key_id)
tmp_dir = volume_utils.image_conversion_dir()
with tempfile.NamedTemporaryFile(dir=tmp_dir) as tmp_key:
diff --git a/cinder/volume/flows/api/create_volume.py b/cinder/volume/flows/api/create_volume.py
index fe0712897..3569e085b 100644
--- a/cinder/volume/flows/api/create_volume.py
+++ b/cinder/volume/flows/api/create_volume.py
@@ -369,8 +369,7 @@ class ExtractVolumeRequestTask(flow_utils.CinderTask):
encryption_key_id = source_volume['encryption_key_id']
elif image_metadata is not None:
# creating from image
- encryption_key_id = image_metadata.get(
- 'cinder_encryption_key_id')
+ encryption_key_id = image_metadata.get('os_encrypt_key_id')
# NOTE(joel-coffman): References to the encryption key should *not*
# be copied because the key is deleted when the volume is deleted.
@@ -390,6 +389,13 @@ class ExtractVolumeRequestTask(flow_utils.CinderTask):
return new_encryption_key_id
else:
+ if image_metadata is not None:
+ if image_metadata.get('os_encrypt_key_id', False):
+ msg = _("Volume type has no encryption but the source "
+ "image is encrypted. When creating a volume from "
+ "an encrypted image, an encrypted volume type "
+ "must be selected.")
+ raise exception.InvalidInput(reason=msg)
return None
@staticmethod
diff --git a/cinder/volume/flows/manager/create_volume.py b/cinder/volume/flows/manager/create_volume.py
index b21d02c9a..6fff50560 100644
--- a/cinder/volume/flows/manager/create_volume.py
+++ b/cinder/volume/flows/manager/create_volume.py
@@ -10,7 +10,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import binascii
import traceback
import typing
from typing import Any, Optional
@@ -506,14 +505,14 @@ class CreateVolumeFromSpecTask(flow_utils.CinderTask):
"""
keymgr = key_manager.API(CONF)
- key = keymgr.get(context, encryption['encryption_key_id'])
- source_pass = binascii.hexlify(key.get_encoded()).decode('utf-8')
+ source_pass = volume_utils.get_encryption_key_as_passphrase(
+ context, keymgr, encryption['encryption_key_id'])
new_key_id = volume_utils.create_encryption_key(context,
keymgr,
volume.volume_type_id)
- new_key = keymgr.get(context, new_key_id)
- new_pass = binascii.hexlify(new_key.get_encoded()).decode('utf-8')
+ new_pass = volume_utils.get_encryption_key_as_passphrase(
+ context, keymgr, new_key_id)
return (source_pass, new_pass, new_key_id)
diff --git a/cinder/volume/volume_utils.py b/cinder/volume/volume_utils.py
index 8d6a01532..3b28a36f3 100644
--- a/cinder/volume/volume_utils.py
+++ b/cinder/volume/volume_utils.py
@@ -43,6 +43,7 @@ from eventlet import tpool
from keystoneauth1 import loading as ks_loading
from os_brick import encryptors
from os_brick.initiator import connector
+from os_brick import utils as brick_utils
from oslo_concurrency import processutils
from oslo_config import cfg
from oslo_log import log as logging
@@ -981,6 +982,14 @@ def create_encryption_key(context: context.RequestContext,
return encryption_key_id
+def get_encryption_key_as_passphrase(context: context.RequestContext,
+ key_manager: castellan_key_manager.API,
+ encryption_key_id: str) -> str:
+ """Convert encryption key to passphrase based on secret type."""
+ key = key_manager.get(context, encryption_key_id)
+ return brick_utils.get_passphrase_from_secret(key)
+
+
def delete_encryption_key(context: context.RequestContext,
key_manager,
encryption_key_id: str) -> None:
@@ -1190,7 +1199,7 @@ def copy_image_to_volume(driver,
{'image_id': image_id, 'volume_id': volume.id,
'image_location': image_location})
try:
- image_encryption_key = image_meta.get('cinder_encryption_key_id')
+ image_encryption_key = image_meta.get('os_encrypt_key_id')
if volume.encryption_key_id and image_encryption_key:
# If the image provided an encryption key, we have
TODO:
NovaPatchset draft for Nova (click to expand ...)diff --git a/nova/tests/unit/virt/libvirt/test_driver.py b/nova/tests/unit/virt/libvirt/test_driver.py
index 660b671f2e..595acdd624 100644
--- a/nova/tests/unit/virt/libvirt/test_driver.py
+++ b/nova/tests/unit/virt/libvirt/test_driver.py
@@ -9656,9 +9656,11 @@ class LibvirtConnTestCase(test.NoDBTestCase,
@mock.patch.object(libvirt_driver.LibvirtDriver, '_get_volume_encryptor')
@mock.patch('nova.virt.libvirt.host.Host')
@mock.patch('os_brick.encryptors.luks.is_luks')
- def test_connect_volume_luks(self, mock_is_volume_luks, mock_host,
- mock_get_volume_encryptor, mock_allow_native_luksv1,
- mock_get_volume_encryption, mock_get_key_mgr):
+ @mock.patch('os_brick.utils.get_passphrase_from_secret')
+ def test_connect_volume_luks(self, mock_get_passphrase,
+ mock_is_volume_luks, mock_host, mock_get_volume_encryptor,
+ mock_allow_native_luksv1, mock_get_volume_encryption,
+ mock_get_key_mgr):
drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), False)
connection_info = {'driver_volume_type': 'fake',
@@ -9678,13 +9680,10 @@ class LibvirtConnTestCase(test.NoDBTestCase,
mock_is_volume_luks.return_value = True
# Mock out the key manager
- key = u'3734363537333734'
- key_encoded = binascii.unhexlify(key)
- mock_key = mock.Mock()
mock_key_mgr = mock.Mock()
mock_get_key_mgr.return_value = mock_key_mgr
- mock_key_mgr.get.return_value = mock_key
- mock_key.get_encoded.return_value = key_encoded
+ mock_key_mgr.get.return_value = mock.sentinel.key_object
+ mock_get_passphrase.return_value = mock.sentinel.volume_passphrase
# assert that the secret is created for the encrypted volume during
# _connect_volume when _allow_native_luksv1 is True
@@ -9694,28 +9693,34 @@ class LibvirtConnTestCase(test.NoDBTestCase,
drvr._connect_volume(self.context, connection_info, instance,
encryption=encryption)
drvr._host.create_secret.assert_called_once_with('volume',
- uuids.volume_id, password=key)
+ uuids.volume_id, password=mock.sentinel.volume_passphrase)
+ mock_get_passphrase.assert_called_once_with(mock.sentinel.key_object)
mock_encryptor.attach_volume.assert_not_called()
# assert that the encryptor is used if is_luks is False
drvr._host.create_secret.reset_mock()
mock_get_volume_encryption.reset_mock()
+ mock_get_passphrase.reset_mock()
mock_allow_native_luksv1.return_value = False
drvr._connect_volume(self.context, connection_info, instance,
encryption=encryption)
drvr._host.create_secret.assert_not_called()
+ mock_get_passphrase.assert_not_called()
mock_encryptor.attach_volume.assert_called_once_with(self.context,
**encryption)
# assert that we format the volume if it is not already formatted
+ mock_get_passphrase.reset_mock()
mock_allow_native_luksv1.return_value = True
mock_is_volume_luks.return_value = False
drvr._connect_volume(self.context, connection_info, instance,
encryption=encryption)
- mock_encryptor._format_volume.assert_called_once_with(key,
- **encryption)
+ mock_get_passphrase.assert_called_once_with(mock.sentinel.key_object)
+ mock_encryptor._format_volume.assert_called_once_with(
+ mock.sentinel.volume_passphrase,
+ **encryption)
@mock.patch.object(libvirt_driver.LibvirtDriver, '_get_volume_encryptor')
def test_disconnect_volume_luks(self, mock_get_volume_encryptor):
diff --git a/nova/virt/libvirt/driver.py b/nova/virt/libvirt/driver.py
index 37613eb2c6..fde7dd5761 100644
--- a/nova/virt/libvirt/driver.py
+++ b/nova/virt/libvirt/driver.py
@@ -24,7 +24,6 @@ A connection to a hypervisor through libvirt.
Supports KVM, LXC, QEMU, and Parallels.
"""
-import binascii
import collections
from collections import deque
import contextlib
@@ -56,6 +55,7 @@ from os_brick import encryptors
from os_brick.encryptors import luks as luks_encryptor
from os_brick import exception as brick_exception
from os_brick.initiator import connector
+from os_brick import utils as brick_utils
import os_resource_classes as orc
import os_traits as ot
from oslo_concurrency import processutils
@@ -2192,8 +2192,7 @@ class LibvirtDriver(driver.ComputeDriver):
# below.
keymgr = key_manager.API(CONF)
key = keymgr.get(context, encryption['encryption_key_id'])
- key_encoded = key.get_encoded()
- passphrase = binascii.hexlify(key_encoded).decode('utf-8')
+ passphrase = brick_utils.get_passphrase_from_secret(key)
# NOTE(lyarwood): Retain the behaviour of the original os-brick
# encryptors and format any volume that does not identify as
TODO:
os-brickPatchset draft for os-brick (click to expand ...)diff --git a/os_brick/encryptors/base.py b/os_brick/encryptors/base.py
index d6ab4a3..7442f22 100644
--- a/os_brick/encryptors/base.py
+++ b/os_brick/encryptors/base.py
@@ -17,6 +17,7 @@
import abc
from os_brick import executor
+from os_brick import utils
class VolumeEncryptor(executor.Executor, metaclass=abc.ABCMeta):
@@ -47,6 +48,10 @@ class VolumeEncryptor(executor.Executor, metaclass=abc.ABCMeta):
"""
return self._key_manager.get(context, self.encryption_key_id)
+ def _get_encryption_key_as_passphrase(self, context):
+ key = self._get_key(context)
+ return utils.get_passphrase_from_secret(key)
+
@abc.abstractmethod
def attach_volume(self, context, **kwargs):
"""Hook called immediately prior to attaching a volume to an instance.
diff --git a/os_brick/encryptors/cryptsetup.py b/os_brick/encryptors/cryptsetup.py
index 8e66a1d..6912150 100644
--- a/os_brick/encryptors/cryptsetup.py
+++ b/os_brick/encryptors/cryptsetup.py
@@ -13,7 +13,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import binascii
import os
from oslo_concurrency import processutils
@@ -97,10 +96,6 @@ class CryptsetupEncryptor(base.VolumeEncryptor):
return False
return True
- def _get_passphrase(self, key):
- """Convert raw key to string."""
- return binascii.hexlify(key).decode('utf-8')
-
def _open_volume(self, passphrase, **kwargs):
"""Open the LUKS partition on the volume using passphrase.
@@ -143,8 +138,7 @@ class CryptsetupEncryptor(base.VolumeEncryptor):
"any existing volumes using this encryptor to the 'luks' "
"LuksEncryptor or 'luks2' Luks2Encryptor encryptors as soon as "
"possible.")
- key = self._get_key(context).get_encoded()
- passphrase = self._get_passphrase(key)
+ passphrase = self._get_encryption_key_as_passphrase(context)
self._open_volume(passphrase, **kwargs)
diff --git a/os_brick/encryptors/luks.py b/os_brick/encryptors/luks.py
index 2aea339..1e59c26 100644
--- a/os_brick/encryptors/luks.py
+++ b/os_brick/encryptors/luks.py
@@ -13,7 +13,6 @@
# License for the specific language governing permissions and limitations
# under the License.
-import binascii
import os
from oslo_concurrency import processutils
@@ -160,10 +159,6 @@ class LuksEncryptor(base.VolumeEncryptor):
root_helper=self._root_helper,
attempts=3)
- def _get_passphrase(self, key):
- """Convert raw key to string."""
- return binascii.hexlify(key).decode('utf-8')
-
def _open_volume(self, passphrase, **kwargs):
"""Opens the LUKS partition on the volume using passphrase.
@@ -184,8 +179,7 @@ class LuksEncryptor(base.VolumeEncryptor):
original symbolic link to refer to the device mounted by dm-crypt.
"""
- key = self._get_key(context).get_encoded()
- passphrase = self._get_passphrase(key)
+ passphrase = self._get_encryption_key_as_passphrase(context)
try:
self._open_volume(passphrase, **kwargs)
@@ -229,8 +223,7 @@ class LuksEncryptor(base.VolumeEncryptor):
"""Extend an encrypted volume and return the decrypted volume size."""
symlink = self.symlink_path
LOG.debug('Resizing mapping %s to match underlying device', symlink)
- key = self._get_key(context).get_encoded()
- passphrase = self._get_passphrase(key)
+ passphrase = self._get_encryption_key_as_passphrase(context)
self._execute('cryptsetup', 'resize', symlink,
process_input=passphrase,
run_as_root=True, check_exit_code=True,
diff --git a/os_brick/tests/encryptors/test_base.py b/os_brick/tests/encryptors/test_base.py
index e9e3448..398cd63 100644
--- a/os_brick/tests/encryptors/test_base.py
+++ b/os_brick/tests/encryptors/test_base.py
@@ -13,14 +13,40 @@
# License for the specific language governing permissions and limitations
# under the License.
+import binascii
from unittest import mock
+from castellan.common import objects as castellan_objects
from castellan.tests.unit.key_manager import fake
from os_brick import encryptors
from os_brick.tests import base
+def fake__get_key_symmetric(passphrase):
+ raw = bytes(binascii.unhexlify(passphrase))
+ symmetric_key = castellan_objects.symmetric_key.SymmetricKey(
+ 'AES', len(raw) * 8, raw)
+ return symmetric_key
+
+
+def fake__get_key_passphrase(passphrase):
+ passphrase_key = castellan_objects.passphrase.Passphrase(passphrase)
+ return passphrase_key
+
+
+class BaseVolumeEncryptor(encryptors.base.VolumeEncryptor):
+
+ def attach_volume(self, context, **kwargs):
+ pass
+
+ def detach_volume(self, **kwargs):
+ pass
+
+ def extend_volume(self, context, **kwargs):
+ pass
+
+
class VolumeEncryptorTestCase(base.TestCase):
def _create(self):
pass
@@ -85,6 +111,45 @@ class BaseEncryptorTestCase(VolumeEncryptorTestCase):
self._test_get_encryptor('nova.volume.encryptors.nop.NoopEncryptor',
encryptors.nop.NoOpEncryptor)
+ @mock.patch('os_brick.encryptors.base.VolumeEncryptor._get_key')
+ def test__get_encryption_key_as_passphrase_hexlify(self, mock_key):
+ """Test passphrase retrieval for secret type 'symmetric'.
+
+ This should hexlify the secret in _get_encryption_key_as_passphrase.
+ """
+ base_enc = BaseVolumeEncryptor(
+ root_helper=self.root_helper,
+ connection_info=self.connection_info,
+ keymgr=self.keymgr
+ )
+ fake_key_plain = 'passphrase-in-clear-text'
+ fake_key_hexlified = binascii.hexlify(fake_key_plain.encode('utf-8'))
+
+ mock_key.return_value = fake__get_key_symmetric(fake_key_hexlified)
+ passphrase = base_enc._get_encryption_key_as_passphrase(
+ mock.sentinel.context)
+ mock_key.assert_called_once_with(mock.sentinel.context)
+ self.assertEqual(passphrase, fake_key_hexlified.decode('utf-8'))
+
+ @mock.patch('os_brick.encryptors.base.VolumeEncryptor._get_key')
+ def test__get_encryption_key_as_passphrase(self, mock_key):
+ """Test passphrase retrieval for secret type 'passphrase'.
+
+ This should skip the hexlify step in _get_encryption_key_as_passphrase.
+ """
+ base_enc = BaseVolumeEncryptor(
+ root_helper=self.root_helper,
+ connection_info=self.connection_info,
+ keymgr=self.keymgr
+ )
+ fake_key_plain = 'passphrase-in-clear-text'
+
+ mock_key.return_value = fake__get_key_passphrase(fake_key_plain)
+ passphrase = base_enc._get_encryption_key_as_passphrase(
+ mock.sentinel.context)
+ mock_key.assert_called_once_with(mock.sentinel.context)
+ self.assertEqual(passphrase, fake_key_plain)
+
def test_get_error_encryptors(self):
encryption = {'control_location': 'front-end',
'provider': 'ErrorEncryptor'}
diff --git a/os_brick/tests/encryptors/test_luks.py b/os_brick/tests/encryptors/test_luks.py
index 13436eb..a0823f0 100644
--- a/os_brick/tests/encryptors/test_luks.py
+++ b/os_brick/tests/encryptors/test_luks.py
@@ -254,17 +254,13 @@ class LuksEncryptorTestCase(test_base.VolumeEncryptorTestCase):
@mock.patch('os_brick.utils.get_device_size')
@mock.patch.object(luks.LuksEncryptor, '_execute')
- @mock.patch.object(luks.LuksEncryptor, '_get_passphrase')
- @mock.patch.object(luks.LuksEncryptor, '_get_key')
- def test_extend_volume(self, mock_key, mock_pass, mock_exec, mock_size):
+ @mock.patch.object(luks.LuksEncryptor, '_get_encryption_key_as_passphrase')
+ def test_extend_volume(self, mock_pass, mock_exec, mock_size):
encryptor = self.encryptor
res = encryptor.extend_volume(mock.sentinel.context)
self.assertEqual(mock_size.return_value, res)
- mock_key.assert_called_once_with(mock.sentinel.context)
- mock_key.return_value.get_encoded.assert_called_once_with()
- key = mock_key.return_value.get_encoded.return_value
- mock_pass.assert_called_once_with(key)
+ mock_pass.assert_called_once_with(mock.sentinel.context)
mock_exec.assert_called_once_with(
'cryptsetup', 'resize', encryptor.dev_path,
process_input=mock_pass.return_value, run_as_root=True,
diff --git a/os_brick/tests/test_utils.py b/os_brick/tests/test_utils.py
index d7d65f6..a5921e3 100644
--- a/os_brick/tests/test_utils.py
+++ b/os_brick/tests/test_utils.py
@@ -18,6 +18,7 @@ import io
import time
from unittest import mock
+from castellan.common import objects as castellan_objects
import ddt
from os_brick import exception
@@ -108,6 +109,35 @@ class TestUtils(base.TestCase):
'dd', 'if=/dev/fake', 'of=/dev/null', 'count=1',
run_as_root=True, root_helper=mock_execute._root_helper)
+ @mock.patch('binascii.hexlify')
+ @ddt.data(
+ castellan_objects.passphrase.Passphrase(mock.sentinel.passphrase),
+ castellan_objects.symmetric_key.SymmetricKey('AES',
+ mock.sentinel.bitlength,
+ mock.sentinel.key),
+ castellan_objects.opaque_data.OpaqueData(mock.sentinel.key),
+ castellan_objects.private_key.PrivateKey('RSA',
+ mock.sentinel.bitlength,
+ mock.sentinel.key),
+ castellan_objects.public_key.PublicKey('RSA',
+ mock.sentinel.bitlength,
+ mock.sentinel.key),
+ castellan_objects.x_509.X509(mock.sentinel.key)
+ )
+ def test_get_passphrase_from_secret(self, secret, mock_hexlify):
+ """Test proper passphrase processing of different secret types."""
+ if secret.managed_type() == 'passphrase':
+ passphrase = utils.get_passphrase_from_secret(secret)
+ mock_hexlify.assert_not_called()
+ self.assertEqual(mock.sentinel.passphrase, passphrase)
+ else:
+ hexlified_bytes = mock.MagicMock()
+ hexlified_bytes.decode.return_value = mock.sentinel.passphrase
+ mock_hexlify.return_value = hexlified_bytes
+ passphrase = utils.get_passphrase_from_secret(secret)
+ mock_hexlify.assert_called_once_with(mock.sentinel.key)
+ self.assertEqual(mock.sentinel.passphrase, passphrase)
+
class TestRetryDecorator(base.TestCase):
diff --git a/os_brick/utils.py b/os_brick/utils.py
index f31e270..c45402c 100644
--- a/os_brick/utils.py
+++ b/os_brick/utils.py
@@ -14,6 +14,7 @@
from __future__ import annotations
+import binascii
import functools
import inspect
import logging as py_logging
@@ -456,6 +457,35 @@ def check_valid_device(executor: executor.Executor, path: str) -> bool:
return info is not None
+def get_passphrase_from_secret(key) -> str:
+ """Convert encryption key retrieved from the Key Manager into a passphrase.
+
+ If the secret type is 'passphrase', assume that the key is already in
+ a suitable string format and simply return it.
+ In any other case, assume a binary key that needs to be converted into
+ an ASCII representation using binascii.hexlify().
+
+ Cinder uses 'symmetric' in conjunction with binascii.hexlify() to
+ handle encryption keys for its own volumes and resulting volume images.
+ Nova uses the 'passphrase' type instead for its qcow2+LUKS images which
+ are directly passed to LUKS as passphrase input. User-defined Glance
+ images may reference secrets of any type (defaulting to 'opaque') which
+ we optimistically assume to represent binary keys too (unless their
+ type is 'passphrase' explicitly).
+
+ :param key: Key Manager Secret containing the encryption key
+ :type key: castellan.common.objects.maanged_object.ManagedObject
+ :return: passphrase
+ :rtype: str
+ """
+ if key.managed_type() == 'passphrase':
+ LOG.info("os_brick.utils.get_passphrase_from_secret: PASSPHRASE")
+ return key.get_encoded().decode('utf-8')
+ else:
+ LOG.info("os_brick.utils.get_passphrase_from_secret: HEXLIFY")
+ return binascii.hexlify(key.get_encoded()).decode('utf-8')
+
+
class Anything(object):
"""Object equal to everything."""
def __eq__(self, other):
TODO:
|
|
So far I've been able to verify the following functionality of the implementation:
Devstack example process (click to expand ...)source /opt/stack/devstack/openrc admin admin
BASIC_TYPE=$(openstack volume type list -f value -c Name | grep lvm)
openstack volume type create \
--property volume_backend_name="$BASIC_TYPE" \
--encryption-provider luks \
--encryption-cipher aes-xts-plain64 \
--encryption-key-size 256 \
--encryption-control-location front-end \
"${BASIC_TYPE}-LUKS"
wget https://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-disk.img
qemu-img convert -O raw cirros-0.6.2-x86_64-disk.img cirros.raw
echo "muchsecretsuchwow" > secret_file.key
export INPUT_FILE="cirros.raw"
qemu-img convert -f raw -O luks \
--object secret,id=sec,file=secret_file.key \
-o key-secret=sec \
-o cipher-alg=aes-256 \
-o cipher-mode=xts \
-o hash-alg=sha256 \
-o ivgen-alg=plain64 \
-o ivgen-hash-alg=sha256 \
$INPUT_FILE "$INPUT_FILE.luks"
TOKEN=$(openstack token issue -f value -c id)
if [ $(openstack secret list -f value --name image-encryption-key | wc -l) == 0 ]; then openstack secret store --file secret_file.key --name image-encryption-key; fi
SID=$(openstack secret list -f value --name image-encryption-key | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)
openstack image create --file cirros.raw.luks \
--property os_encrypt_key_id=$SID \
--property os_encrypt_key_deletion_policy=none \
--property os_encrypt_format=luks \
--property os_encrypt_cipher=aes-xts-plain64 \
custom-encrypted-cirros
echo $SID
# b8fa610b-8aaf-4e88-af92-28880d0eb04b
openstack secret consumer list $SID
# +---------+---------------+--------------------------------------+---------------------+
# | Service | Resource type | Resource id | Created |
# +---------+---------------+--------------------------------------+---------------------+
# | glance | image | f4e7361b-7332-405a-aaa1-586161f09c87 | 2024-08-06T14:42:50 |
# +---------+---------------+--------------------------------------+---------------------+
openstack volume create --size 1 --image custom-encrypted-cirros test-volume-encimg
# Invalid input received: Volume type has no encryption but the source
# image is encrypted. When creating a volume from an encrypted image,
# an encrypted volume type must be selected. (HTTP 400) (Request-ID:
# req-9e6e1e3e-8709-4cf5-a91d-68f7ad95a01b)
openstack volume create --type lvmdriver-1-LUKS --size 1 \
--image custom-encrypted-cirros test-volume-encimg
openstack image create --volume test-volume-encimg img-from-enc-vol
openstack image show img-from-enc-vol -c properties
# +------------+-------------------------------------------------------------------------------------------------------------------------------------+
# | Field | Value |
# +------------+-------------------------------------------------------------------------------------------------------------------------------------+
# | properties | os_encrypt_cipher='aes-xts-plain64', os_encrypt_format='luks', os_encrypt_key_deletion_policy='on_image_deletion', |
# | | os_encrypt_key_id='a43c84b7-44b0-45b5-b64c-15a13d00777d', os_hidden='False', owner_specified.openstack.md5='', |
# | | owner_specified.openstack.object='images/custom-encrypted-cirros', owner_specified.openstack.sha256='', signature_verified='False' |
# +------------+-------------------------------------------------------------------------------------------------------------------------------------+ |
|
Using a non-passphrase type secret works with the changes when making sure to hexlify the secret befor using Mimicking the hexlify as a userecho -n "muchsecretsuchwow" > secret_file.key
# it is important that the file does not end with a newline!
python3 -c "import binascii; print(binascii.hexlify('$(cat secret_file.key)'.encode('utf-8')).decode('utf-8'), end='')" > secret_file.hex
# secret_file.key <- plaintext passphrase
# secret_file.hex <- hexlified passphrase
# store the secret in plaintext
openstack secret store --file secret_file.key --secret-type symmetric ...
# encrypt the image file using the hexlified passphrase
qemu-img convert -f raw -O luks --object secret,id=sec,file=secret_file.hex ... IMG_FILE
openstack image create --file IMG_FILE --property os_encrypt_key_id= ...Full DevStack example with hexlify (click to expand ...)echo -n "muchsecretsuchwow" > secret_file.key
# it is important that the file does not end with a newline!
python3 -c "import binascii; print(binascii.hexlify('$(cat secret_file.key)'.encode('utf-8')).decode('utf-8'), end='')" \
> secret_file.hex
export INPUT_FILE="cirros.raw"
qemu-img convert -f raw -O luks \
--object secret,id=sec,file=secret_file.hex \
-o key-secret=sec \
-o cipher-alg=aes-256 \
-o cipher-mode=xts \
-o hash-alg=sha256 \
-o ivgen-alg=plain64 \
-o ivgen-hash-alg=sha256 \
$INPUT_FILE "$INPUT_FILE.luks.hexlified"
# note: do *not* store the hexlified key here, it will be hexlified by
# Cinder later as long as the secret-type is not 'passphrase'!
if [ $(openstack secret list -f value --name image-encryption-key | wc -l) == 0 ]; \
then openstack secret store --file secret_file.key \
--name image-encryption-key --secret-type symmetric; fi
SID=$(openstack secret list -f value --name image-encryption-key | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)
openstack image create --file cirros.raw.luks.hexlified \
--property os_encrypt_key_id=$SID \
--property os_encrypt_key_deletion_policy=none \
--property os_encrypt_format=luks \
--property os_encrypt_cipher=aes-xts-plain64 \
custom-encrypted-cirros-hexlify
openstack volume create --type lvmdriver-1-LUKS --size 1 \
--image custom-encrypted-cirros-hexlify test-volume-encimg-hexlify
openstack server create --flavor m1.tiny --network admin-private \
--volume test-volume-encimg-hexlify --security-group admin-access-group \
server-from-encimg-vol-hexlifyCinder will later retrieve the Barbican secret and hexlify it to use it as a passphrase. This works already. Skipping hexlify with 'passphrase' secret typeIt is much more convenient for users to skip the hexlify step and use os-brick needs to be patched to differentiate between secret types in the same fashion as Cinder. UPDATE: Nova also needs to be patched, because it does the hexlify itself in the libvirt driver in case of native LUKS attachment through QEMU: https://github.com/openstack/nova/blob/bb2d7f9cad577f3a32cb9523e2b1d9a6d6db3407/nova/virt/libvirt/driver.py#L2193-L2196 Since we now need the same code (differentiating between secret types and hexlifying on demand) at three places (Cinder, Nova, os-brick) I think it would be best to de-duplicate the code into os-brick and import it from os-brick in all other services. EDIT: Updated #560 (comment) with corresponding patchset drafts for Nova and os-brick. Footnotes |
Qcow2 image conversion in CinderUsing my Cinder patchset draft as implemented so far, I tested how and where qcow2+LUKS images would be handled using two test cases, one for non-hexlify and one for hexlify: test cases with qcow2+LUKS images (click to expand ...)echo -n "muchsecretsuchwow" > secret_file.key
# it is important that the file does not end with a newline!
python3 -c "import binascii; print(binascii.hexlify('$(cat secret_file.key)'.encode('utf-8')).decode('utf-8'), end='')" \
> secret_file.hex
export INPUT_FILE="cirros.raw"
# CASE A: PASSPHRASE (NO HEXLIFY)
qemu-img convert -f raw -O qcow2 \
--object secret,id=sec,file=secret_file.key \
-o encrypt.format=luks \
-o encrypt.key-secret=sec \
-o encrypt.cipher-alg=aes-256 \
-o encrypt.cipher-mode=xts \
-o encrypt.hash-alg=sha256 \
-o encrypt.ivgen-alg=plain64 \
-o encrypt.ivgen-hash-alg=sha256 \
$INPUT_FILE "$INPUT_FILE.luks.qcow2"
# secret type = passphrase
if [ $(openstack secret list -f value --name qcow-image-passphrase | wc -l) == 0 ]; \
then openstack secret store --file secret_file.key \
--name qcow-image-passphrase --secret-type passphrase; fi
SID=$(openstack secret list -f value --name qcow-image-passphrase | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)
openstack image create --file $INPUT_FILE.luks.qcow2 \
--disk-format qcow2 \
--property os_encrypt_key_id=$SID \
--property os_encrypt_key_deletion_policy=none \
--property os_encrypt_format=luks \
--property os_encrypt_cipher=aes-xts-plain64 \
test-enc-qcow2luks
openstack volume create --type lvmdriver-1-LUKS --size 1 \
--image test-enc-qcow2luks test-qcow2luks-img
# ------------------------------------------------------------
# CASE B: BINARY KEY (WITH HEXLIFY)
qemu-img convert -f raw -O qcow2 \
--object secret,id=sec,file=secret_file.hex \
-o encrypt.format=luks \
-o encrypt.key-secret=sec \
-o encrypt.cipher-alg=aes-256 \
-o encrypt.cipher-mode=xts \
-o encrypt.hash-alg=sha256 \
-o encrypt.ivgen-alg=plain64 \
-o encrypt.ivgen-hash-alg=sha256 \
$INPUT_FILE "$INPUT_FILE.luks.qcow2.hexlified"
# secret type = symmetric
if [ $(openstack secret list -f value --name qcow-image-key | wc -l) == 0 ]; \
then openstack secret store --file secret_file.key \
--name qcow-image-key --secret-type symmetric; fi
SID=$(openstack secret list -f value --name qcow-image-key | cut -d' ' -f1 | rev | cut -d'/' -f1 | rev)
openstack image create --file $INPUT_FILE.luks.qcow2.hexlified \
--disk-format qcow2 \
--property os_encrypt_key_id=$SID \
--property os_encrypt_key_deletion_policy=none \
--property os_encrypt_format=luks \
--property os_encrypt_cipher=aes-xts-plain64 \
test-enc-qcow2luks-hex
openstack volume create --type lvmdriver-1-LUKS --size 1 \
--image test-enc-qcow2luks-hex test-qcow2luks-img-hexBoth test cases currently exhibit the same error pattern as quoted below: Their conversion is attempted by Problem with this function is that it calls Judging from this example, the best place for implementing the flattening/re-encryption of qcow2+LUKS images seems to be Most Cinder backend drivers seem to call
We'll either need to take care of them individually or insert safeguards that prevent them from consuming qcow2+LUKS images in case we are unable to provide a suitable implementation for them. UPDATE: I adjusted |
Upstream PatchsetsI uploaded all patchsets upstream now effectively deprecating the drafts in #560 (comment):
The gerrit topic label is Work should continue there. @josephineSei the Glance spec patchset at https://review.opendev.org/c/openstack/glance-specs/+/915726 seems to have been added to the old "image-encryption" topic label. Can you change it to "LUKS-image-encryption"? That's where the new Cinder Spec and the above patchsets now reside for the image encryption rework. Remaining TODOs:
|
Metadata naming deprecation/transitioningAs discussed with @josephineSei today, there is an open question to be discussed with the OpenStack community on how to handle the transition from the old Glance image metadata naming ( In the current patchset I simply replaced the old names.
It may be advisable to introduce a deprecation period and make Glance and Cinder still accept the old metadata names when parsing image metadata. |
|
I wrote a mail to the OpenStack discuss ML to invite Cinder and Glance people to a discussion on this topic next monday on irc.
|
|
We discussed the topic today with upstream (Logs are here: https://meetings.opendev.org/meetings/image_encryption/2024/image_encryption.2024-08-19-13.00.log.html) and will do the following:
I will also write another mail to the ML like rosmaita suggested. To ask ops whether they are okay with the upgrading workflow of:
|
|
I am writing said extension, but currently having problems testing it with my devstack: This Error does not occur with cinder: This seems to be a devstack specific bug, as the I asked in IRC and on the ML, I hope I get an answer. (https://lists.openstack.org/archives/list/[email protected]/thread/JOCR5V26IPW6IJIUJIAVXPFXJEPXOPFU/) |
I revisited the patchsets for Glance and Cinder and added support for
I added deprecation notes in the source code (or naming of the unit test methods) accordingly so that all places where stuff needs to be removed after the deprecation period can be easily identified. |
|
I got an answer to my problem on the mailing list and can now continue implementation and testing. |
|
We attended the Cinder meeting in IRC today and got attention on the os-brick patch. |
|
I adjusted the patchsets to fix failing tests for os-brick and Glance; os-brick is now verified by Zuul, |
|
I somehow still have problems with the migration script : it does not seem to be triggerd by glance-manage. I still put my code in an upstream patch as WIP: https://review.opendev.org/c/openstack/glance/+/926905 |
I tried another few things, but at this point I would either need another deployment to test or some help from upstream - I already asked in the glance channel and the patch got review priority. |
|
I raised attention again for all the patches, in the irc glance channel and in the pop-up team meeting. |
|
Cinder and Glance patchsets have been updated to only deprecate |
|
I fixed some errors in the glance db data-migration patch. And looked through the Zuul logs for the Nova and Cidner patches. |
|
Upstream, the Cinder Tempest tests in Zuul currently fail during volume cloning of encrypted volumes1: Source code of the Tempest test: https://github.com/openstack/cinder-tempest-plugin/blob/1aa0a56fca85ce23343950431e28f41c4fa811c5/cinder_tempest_plugin/scenario/test_volume_encrypted.py#L150-L157 I started debugging this on my DevStack but so far haven't been able to reproduce a volume in ERROR state when cloning an existing encrypted one. I happened to notice one difference though:
I don't think cloning the secret name without any suffix (1) is desired at all because this is confusing for users when their image secret appears multiple times in Barbican by name. This also seems to hint at differences of secret clone handling between (1) and (2). I will keep investigating. Footnotes |
|
The recheck for the Nova Patch led to a green zuul pipeline. Both should not be related to the migration patch. I still wait for feedback from the Glance team, because the migration is not triggered on devstack. |
I adjusted the Cinder patchset to remove the name when cloning a secret. |
|
We got a review on the Glance patchset requesting the following changes:
I'll look into those. |
Possible changes to disk_formatInitially, we proposed to introduce a new On 2024-08-28 in the weekly Glance IRC meeting2, @josephineSei and I couldn't participate due to a conflicting meeting but the image encryption patchsets were discussed nonetheless and an interesting point was raised: Then we received the following comment on the Glance patchset from Dan Smith (Nova):
As a result, we once again need to rethink and rework our encryption format approach in the Cinder and Glance patchsets to address this, it seems. Footnotes |
|
After reading through the Spec from Dan and looking into our own spec: https://review.opendev.org/c/openstack/glance-specs/+/915726/11/specs/2024.2/approved/glance/standardized_image_encryption.rst I would like to propose an update to our specs for Cinder and Glance. We should definitely state it there, because people will find these specs also when searching for documentation. Here is what we could do in Glance: https://review.opendev.org/c/openstack/glance-specs/+/927819/1/specs/2024.2/approved/glance/standardized_image_encryption.rst |
3 tasks
|
In addition to your spec updates and Dan Smith's comment:
I don't think we could or should go ahead and simply introduce some arbitrary I participated in today's Cinder IRC meeting, raised awareness of the topic again and asked people to attend next monday's popup meeting. |
|
I attended today's Glance meeting: seems like we're back to the drawing board regarding the image metadata attributes associated with encrypted images and need to discuss this in the next PTG with the teams of Nova, Glance and Cinder: Source: https://meetings.opendev.org/meetings/glance/2024/glance.2024-09-05-14.00.log.html Update:
|
|
I think It would be good to have this conversation at the PTG in a cross-project session together with Cinder (and maybe Nova) to get everyone to agree on one way. But even more important: I think we (I can try to do so) should reach out to Dan Smith to sketch out one or more possible ways of implementing the image encryption - all BEFORE the PTG happens. He is the one working on the image format checker, so he knows best, what information is needed to verify that there is no malicious image upload. |
|
@josephineSei and I held today's image encryption popup team meeting on IRC1. About formats:
About vulnerabilities:
(past CVEs were based on qemu encountering a VMDK image with special instructions and executing them; but in this case, only the outer LUKS layer should actually be visible to qemu but that needs to be verified) Footnotes |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently there is the possibility in Cinder to encrypt volumes and in Nova to use qcow2 encrypted images (still under development).
Both can lead to and use LUKS-encrpyted images, but those are different and not aligned:
@markus-hentsch also found out in #541 that uploading a LUKS-encrypted image (that was created from a volume) to another cloud in combination with setting a few parameteres (cinder_encryption_key, etc...) will result in an image that can be used to create an encrpyted and functional volume.
As a user it would be good to have a streamlined operation to use encrypted images in openstack for both volumes and ephemeral storage and to also allow interoperability between clouds.
Therefore we need and will propose standardized parameters to describe and detect an encrypted image, which might be similar to the parameters described here: https://specs.openstack.org/openstack/cinder-specs/specs/zed/image-encryption.html
But will use the LUKS encryption.
So those encrypted images could be natively mounted in Nova or just formed into a volume (raw LUKS images can be directly used, qcow images need to be flattened).
With such a way encrypted backup images can be easily downloaded and transferred to another cloud.
This is a result from a lengthy discussion at the PTG with Nova, Cinder and Glance ( https://etherpad.opendev.org/p/dalmatian-ptg-cinder#L376 )
Followup tasks may be to implement re-encryption to fully change keys for LUKS volumes and images.
The text was updated successfully, but these errors were encountered: