Add standard for volume backup functionality

[markus-hentsch]

No description provided.

[markus-hentsch]

Proposal written. Test script implemented.

Cleanup function of the test script needs some polish according to some erronous behavior I observed:

• wait for resources to be in available state before attempting deletion
• wait for backups to finish being cleaned up before starting volume deletion (dependency)
• make errors in cleanup fatal for test execution (resoning: deletion should work just as fine as creation when API works correctly and we need a clean base to test on)

[josephineSei]

WE do have the problem that we should have both in mind: large deployments and edge cloud scenarios with only one rack somewhere. I am a bit undecided how to deal with a possible second storage backend in the latter case.

[markus-hentsch]

Proposal written. Test script implemented.

Cleanup function of the test script needs some polish according to some erronous behavior I observed:

* wait for resources to be in available state before attempting deletion

* wait for backups to finish being cleaned up before starting volume deletion (dependency)

* make errors in cleanup fatal for test execution (resoning: deletion should work just as fine as creation when API works correctly and we need a clean base to test on)

Done and tested.

[anjastrunk]

LGTM in general. Just one addition requested in section "Conformance Tests".

[josephineSei]

I think only this one question is left for me.

[berendt]

I think only this one question is left for me.

Wouldn't it even be a good idea to require backups to be stored on a different storage backend? Then there would always be a "Medienbruch".

So I would prefer that the backup function itself does not necessarily have to be available, but if it is available it always has a dedicated storage backend.

Then you can assume that if the function is available, it also enables real backups.

[neuroserve]

Just a quick feedback we got from a customer accustomed to VMWare: The backup functions (provided by Veeam) are integrated in the webfrontend (VCloud Director) there. And that's why they were looking for similar functions in Horizon, too.

[markus-hentsch]

I think only this one question is left for me.

Wouldn't it even be a good idea to require backups to be stored on a different storage backend? Then there would always be a "Medienbruch".

So I would prefer that the backup function itself does not necessarily have to be available, but if it is available it always has a dedicated storage backend.

Then you can assume that if the function is available, it also enables real backups.

Very good point. I added a new choice to the "Options considered" section. I think it boils down to whether we want to favor feature availability vs. backup reliability.

Personally, I'd actually favor reliability now that I think about it due to the fact the term "backup" does imply some kind of availability guarantees.

[markus-hentsch]

Just a quick feedback we got from a customer accustomed to VMWare: The backup functions (provided by Veeam) are integrated in the webfrontend (VCloud Director) there. And that's why they were looking for similar functions in Horizon, too.

Thanks for sharing this. @neuroserve do you happen to know whether VMWare/Veeam makes any assumptions or decisions regarding the location of backups in relation to the main storage, i.e., are backups always located in a separate storage backend?

[neuroserve]

AFAIK the Veeam solution is quite flexible and can use "any" S3-Backend (for example). The backup destination is not tied to the infrastructure, where the VCloud Director is located.

[markus-hentsch]

The discussion in the IaaS call on 2024-07-03 did not produce a clear winner among the options (feature availability vs. backup reliability).
However, a third option was proposed:

• option c)
  • mandate backup feature and API availability AND make it transparent to users ("discoverability") how much additional reliability is provided
    • place of discoverability is not obvious (no place for metadata in the cinder backup API)
      • api extensions?
      • Gaia-X self-descriptionsVCs
      • new discoverability service that we work on introducing for flavors (-> public cloud SIG draft spec) spec

... which also favors feature availability while trying to make the potential shortcomings in backup reliability transparent for the user in case a separate storage backend is not configured for backups by the CSP.
This however requires some kind of self-description functionality as stated, which we do not have yet.

There has been no further feedback since then. The current iteration of the standard draft implements the feature availability option, which also provides the basis for the newly suggested option described above. For now, I will keep it that way.

[josephineSei]

This is looking good. Most comments are for the reference of OpenStack-sepcific terms, where we might be more broad to also integrate other implementations of the APIs.

[josephineSei]

As it is not possible to know as a user, whether a volume backup storage is different from the normal volume storage. We should encourage CSPs to give that information to customers. This could also be done via gaiax credentials, couldn't it? Maybe we need another issue for this part.

[markus-hentsch]

This would require a deep dive into the Gaia-X Ontology to figure out whether this can be expressed using Gaia-X Credentials (formerly Self-Descriptions) appropriately.

As long as we don't have a standardized and proven way of representing arbitrary self-description information about an SCS cloud and its services in Gaia-X Credentials, I'm hesitant to add any such suggestions to the standard yet.

[josephineSei]

Yeah, I thought so - Maybe we can discuss in the next Standardization SIG meeting or tomorrow, if and how we can use the gaia-x credentials for tests.

[josephineSei]

This looks good to me.
Someone from a CSP or from the IaaS team should also approve.