Skip to content

Commit

Permalink
Decouple standard flavors from flavor naming (#319)
Browse files Browse the repository at this point in the history 
* Attempt at decoupling standard flavors from flavor naming

Signed-off-by: Matthias Büchse <[email protected]>

* made ssd flavors recommended (maybe even remove them entirely)

Signed-off-by: Matthias Büchse <[email protected]>

* First shot at conformance test for the newly split-up standards

Signed-off-by: Matthias Büchse <[email protected]>

* Appease flake8 by removing unused imports

Signed-off-by: Matthias Büchse <[email protected]>

* Fix scs-0100-v4-flavors.yaml (empty lists) as well as mode for flavors-openstack.py (executable)

Signed-off-by: Matthias Büchse <[email protected]>

* Appeased markdown-lint; harmonized extra-spec key with script: scs:cpu-type instead of scs:vCPU-type

Signed-off-by: Matthias Büchse <[email protected]>

* Adjusted/amended Conformance Tests sections according to scs-0001-v1.

Signed-off-by: Matthias Büchse <[email protected]>

* Make properties a first-class subject of this standard

Any flavor that carries any of these must also have the corresponding semantics, otherwise no discoverability

Signed-off-by: Matthias Büchse <[email protected]>

* SSD flavors are mandatory (revert b7d655a)

Signed-off-by: Matthias Büchse <[email protected]>

* Rename scs:name-v4 to v2 because the naming scheme has not changed since v2

Signed-off-by: Matthias Büchse <[email protected]>

* Make requirements more precise, enhance discoverability

Signed-off-by: Matthias Büchse <[email protected]>

* Change scs-0100-v3 in place because the change doesn't render any implementation unconformant

Signed-off-by: Matthias Büchse <[email protected]>

* Tool to generate input for osism's flavor manager from this standard's yaml file

Signed-off-by: Matthias Büchse <[email protected]>

* Include v1 names, make name handling more explicit

Signed-off-by: Matthias Büchse <[email protected]>

* Include v1 names

Signed-off-by: Matthias Büchse <[email protected]>

* Appease flake8: remove unused variable

Signed-off-by: Matthias Büchse <[email protected]>

* Added improvements to standard flavors.

Define cpu-type and diskN-types and cross-reference flavor-naming.
Mention live-migration consequence.

Signed-off-by: Kurt Garloff <[email protected]>

---------

Signed-off-by: Matthias Büchse <[email protected]>
Signed-off-by: Kurt Garloff <[email protected]>
Co-authored-by: Kurt Garloff <[email protected]>
  • Loading branch information
@mbuechse @garloff
mbuechse and garloff authored Sep 6, 2023
1 parent 3bcd99f commit 3baa7f0
Show file tree
Hide file tree
Showing 7 changed files with 637 additions and 68 deletions.
73 changes: 9 additions & 64 deletions Standards/scs-0100-v3-flavor-naming.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ replaces: scs-0100-v2-flavor-naming.md

## Introduction

This is the standard v3.0 for SCS Release 5.
This is the standard v3.1 for SCS Release 5.
Note that we intend to only extend it (so it's always backwards compatible),
but try to avoid changing in incompatible ways.
(See at the end for the v1 to v2 transition where we have not met that
Expand All @@ -22,10 +22,7 @@ The flavors are pre-defined by the operator, the customer can not change these.
OpenStack providers thus typically offer a large selection of flavors.

While flavors can be discovered (`openstack flavor list`), it is helpful for users (DevOps teams),
to have

- A naming scheme that is used across all SCS flavors, so flavor names have the same meaning everywhere.
- Have a guaranteed set of flavors available on all SCS clouds, so these do not need to be discovered.
to have a naming scheme that is used across all SCS flavors, so flavor names have the same meaning everywhere.

While not all details will be encoded in the name, the key features should be obvious:
Number of vCPUs, RAM, Root Disk.
Expand Down Expand Up @@ -226,56 +223,6 @@ so users can expect some level of parallelism and independence.
- SCS-2C-4-3x10 <- Cloud decides type and creates three 10GB volumes
- ~~SCS-2C-4-**1.5n**~~ <- You must not specify disk sizes which are not in full GiBs

## Standard SCS flavors

These are flavors that must exist on standard SCS clouds (x86-64).

We recommend disk sizes to be 5, 10, 20, 50, 100, 200, 500, 1000GB, 2000GB.
We expect the most used vCPU:Mem[GiB] ratio to be 1:4.

| vCPU:RAM ratio | Mandatory Flavors | Recommended Flavors |
| -------------- | ------------------------- | ------------------- |
| 1:4 | SCS-1V-4 | SCS-1V-4-10 |
| 2:8 | SCS-2V-8 | SCS-2V-8-20 |
| 4:16 | SCS-4V-16, SCS-4V-16-100s | SCS-4V-16-50 |
| 8:32 | SCS-8V-32 | SCS-8V-32-100 |
| 1:2 | SCS-1V-2 | SCS-1V-2-5 |
| 2:4 | SCS-2V-4, SCS-2V-4-20s | SCS-2V-4-10 |
| 4:8 | SCS-4V-8 | SCS-4V-8-20 |
| 8:16 | SCS-8V-16 | SCS-8V-16-50 |
| 16:32 | SCS-16V-32 | SCS-16V-32-100 |
| 1:8 | SCS-1V-8 | SCS-1V-8-20 |
| 2:16 | SCS-2V-16 | SCS-2V-16-50 |
| 4:32 | SCS-4V-32 | SCS-4V-32-100 |
| 1:1 | SCS-1L-1 | SCS-1L-1-5 |

Note that all vCPUs of SCS standard flavors are oversubscribed — the smallest `1L-1`
flavor allows for heavy oversubscription (note the `L`), and thus can be offered very
cheaply — imagine jump hosts ...

The design allows for small clouds (with CPUs with 16 Threads, 64GiB RAM
compute hosts) to offer all flavors.

Note that the flavors with fixed size root disks have all moved to Recommended
in version 3 of the standard. This means that they are not a certification requirement any longer,
but we still recommend implementing these for backwards compatibility reasons.
Disks types are not specified (and expected to be n or h typically).

However, two flavors with SSD+ root disks have been added in v3, as defined in
[scs-0110-v1-ssd-flavors.md](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0110-v1-ssd-flavors.md)

Note: Compared to previous drafts, we have heavily reduced the variations
on disk sizes — this reflects that for the standard networked cinder
disks, you can pass `block_device_mapping_v2` on server (VM) creation to
allocate a boot disk of any size you desire. We have scaled the few
recommended disk sizes with the amount of RAM. For each flavor there is
also one _without_ a pre-attached disk — these are meant to be used
to boot from a volume (either created beforehand or allocated on-the-fly
with `block_device_mapping_v2`, e.g.
`openstack server create --flavor SCS-1V:2 --block-device-mapping sda=IMGUUID:image:12:true`
to create a bootable 12G cinder volume from image `IMGUUID` that gets tied to the VM
instance life cycle.)

## Naming policy compliance

You are allowed to understate your performance; you may implement a SCS-1V-1-5 flavor with
Expand Down Expand Up @@ -306,7 +253,7 @@ You must not offer flavors with the `SCS-` prefix which do not follow this namin
You must not extend the SCS naming scheme with your own extensions; you are encouraged however
to suggest extensions that we can discuss and add to the official scheme.

## Validation
## Conformance Tests

There is a script in [`flavor-name-check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/flavor-naming/flavor-name-check.py)
which can be used to decode, validate and construct flavor names.
Expand All @@ -319,17 +266,11 @@ on the flavor list compliance of the cloud environment.

The script `flavor-names-openstack.py` talks to the OpenStack API of the
cloud specified by the `OS_CLOUD` environment and queries properties and checks
the names for standards compliance and completeness w.r.t. the mandatory
flavor list. It goes beyond the above example in checking that the discoverable
the names for standards compliance.
It goes beyond the above example in checking that the discoverable
features of flavors (vCPUs, RAM, Disk) match what the flavor names claim.
This is used for SCS-compatible compliance testing.

## Operational tooling

The [openstack-flavor-manager](https://github.com/osism/openstack-flavor-manager) is able to
create all standard, mandatory SCS flavors for you. It takes input that can be generated by
`flavor-manager-input.py`.

## Extensions

Extensions provide a possibility for providers that offer a very differentiated set
Expand Down Expand Up @@ -526,6 +467,10 @@ an image is considered broken by the SCS team.

## Previous standard versions

Previous versions up to version 3.0 contained the list of
mandatory/recommended flavors, which has been moved to
[a standard of its own](scs-0103-v1-standard-flavors.md).

[Version 1 of the standard](scs-0100-v1-flavor-naming.md)
used a slightly different naming syntax while the logic was exactly the same.
What is a `-` in v2 used to be a `:`; `_` used to be `-`. The reason for
Expand Down
150 changes: 150 additions & 0 deletions Standards/scs-0103-v1-standard-flavors.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
---
title: SCS Standard Flavors and Properties
type: Standard
status: Draft
track: IaaS
---

## Introduction

## Motivation

In OpenStack environments there is a need to define different flavors for instances.
The flavors are pre-defined by the operator, the customer can not change these.
OpenStack providers thus typically offer a large selection of flavors.

While flavors can be discovered (`openstack flavor list`), it is helpful for users (DevOps teams),
to have a guaranteed set of flavors available on all SCS clouds, so these need not be discovered.

## Properties (extra specs)

The following extra specs are recognized, together with the respective semantics:

- `scs:name-vN=NAME` (where `N` is `1` or `2`, and `NAME` is some string) means that the
flavor is one of the
standard SCS flavors, and the requirements of Section "Standard SCS flavors" below apply.
- `scs:cpu-type=shared-core` means that _at least 20% of a core in >99% of the time_,
measured over the course of one month (1% is 7,2 h/month). The `cpu-type=shared-core`
corresponds to the `V` cpu modifier in the [flavor-naming spec](./scs-0100-v3-flavor-naming.md),
other options are `crowded-core` (`L`), `dedicated-thread` (`T`) and `dedicated-core` (`C`).
- `scs:diskN-type=ssd` (where `N` is a nonnegative integer, usually `0`) means that the
root disk `N` must support 1000 _sequential_ IOPS per VM and it must be equipped with
power-loss protection; see [scs-0110-v1-ssd-flavors](./scs-0110-v1-ssd-flavors.md).
The `disk`N`-type=ssd` setting corresponds to the `s` disk modifier, other options
are `nvme` (`p`), `hdd` (`h`) and `network` (`n`). Only flavors without disk and
those with `diskN-type=network` can be expected to support live-migration.

Whenever ANY of these are present on ANY flavor, the corresponding semantics must be satisfied.

## Standard SCS flavors

These are flavors that must exist on standard SCS clouds (x86-64).

### Mandatory

| Recommended name | vCPUs | vCPU type | RAM [GiB] | Root disk [GB] | Disk type |
| ---------------- | ------ | ------------- | ---------- | --------------- | ---------- |
| SCS-1V-4 | 1 | shared-core | 4 | | |
| SCS-2V-8 | 2 | shared-core | 8 | | |
| SCS-4V-16 | 4 | shared-core | 16 | | |
| SCS-4V-16-100s | 4 | shared-core | 16 | 100 | ssd |
| SCS-8V-32 | 8 | shared-core | 32 | | |
| SCS-1V-2 | 1 | shared-core | 2 | | |
| SCS-2V-4 | 2 | shared-core | 4 | | |
| SCS-2V-4-20s | 2 | shared-core | 4 | 20 | ssd |
| SCS-4V-8 | 4 | shared-core | 8 | | |
| SCS-8V-16 | 8 | shared-core | 16 | | |
| SCS-16V-32 | 16 | shared-core | 32 | | |
| SCS-1V-8 | 1 | shared-core | 8 | | |
| SCS-2V-16 | 2 | shared-core | 16 | | |
| SCS-4V-32 | 4 | shared-core | 32 | | |
| SCS-1L-1 | 1 | crowded-core | 1 | | |

### Recommended

| Recommended name | vCPUs | vCPU type | RAM [GiB] | Root disk [GB] | Disk type |
| ---------------- | ------ | ------------- | ---------- | --------------- | ---------- |
| SCS-1V-4-10 | 1 | shared-core | 4 | 10 | (any) |
| SCS-2V-8-20 | 2 | shared-core | 8 | 20 | (any) |
| SCS-4V-16-50 | 4 | shared-core | 16 | 50 | (any) |
| SCS-8V-32-100 | 8 | shared-core | 32 | 100 | (any) |
| SCS-1V-2-5 | 1 | shared-core | 2 | 5 | (any) |
| SCS-2V-4-10 | 2 | shared-core | 4 | 10 | (any) |
| SCS-4V-8-20 | 4 | shared-core | 8 | 20 | (any) |
| SCS-8V-16-50 | 8 | shared-core | 16 | 50 | (any) |
| SCS-16V-32-100 | 16 | shared-core | 32 | 100 | (any) |
| SCS-1V-8-20 | 1 | shared-core | 8 | 20 | (any) |
| SCS-2V-16-50 | 2 | shared-core | 16 | 50 | (any) |
| SCS-4V-32-100 | 4 | shared-core | 32 | 100 | (any) |
| SCS-1L-1-5 | 1 | crowded-core | 1 | 5 | (any) |

### Guarantees and properties

The figures given in the table (number of CPUs, amount of RAM, root disk size) must match
precisely the corresponding figures in the flavor.

In addition, the following properties must be set (in the `extra_specs`):

- `scs:name-v1` to the recommended name, but with each dash AFTER the first one replaced by a colon,
- `scs:name-v2` to the recommended name,
- `scs:cpu-type` to `shared-core` or `crowded-core`, reflecting the vCPU type,
- `scs:disk0-type` not set if no disk is provided, otherwise set to `ssd` or some other
value, reflecting the disk type.

### Remarks

We expect the most used vCPU:RAM[GiB] ratio to be 1:4.

Note that all vCPUs of SCS standard flavors are oversubscribed — the smallest `1L-1`
flavor allows for heavy oversubscription (note the `L`), and thus can be offered very
cheaply — imagine jump hosts ...

The design allows for small clouds (with CPUs with 16 Threads, 64GiB RAM
compute hosts) to offer all flavors.

Except for the two flavors with SSD root volume, disks types are not specified
(and expected to be network disks (Ceph/Cinder) or local SATA/SAS disks typically).

We only included a limited variation of disk sizes — this reflects that
for the standard networked cinder
disks, you can pass `block_device_mapping_v2` on server (VM) creation to
allocate a boot disk of any size you desire. We have scaled the few
recommended disk sizes with the amount of RAM. For each flavor there is
also one _without_ a pre-attached disk — these are meant to be used
to boot from a volume (either created beforehand or allocated on-the-fly
with `block_device_mapping_v2`, e.g.
`openstack server create --flavor SCS-1V-2 --block-device-mapping sda=IMGUUID:image:12:true`
to create a bootable 12G cinder volume from image `IMGUUID` that gets tied to the VM
instance life cycle.)

## Conformance Tests

The script `flavors-openstack.py` will read the lists of mandatory and recommended flavors
from a yaml file provided as command-line argument, connect to an OpenStack installation,
and check whether the flavors are present and their extra specs are correct. Missing
flavors will be reported on various logging channels: error for mandatory, info for
recommended flavors. Incorrect extra specs will be reported as error in any case.
The return code will be non-zero if the test could not be performed or if any error was
reported.

## Operational tooling

The [openstack-flavor-manager](https://github.com/osism/openstack-flavor-manager) is able to
create all standard, mandatory SCS flavors for you. It takes input that can be generated by
`flavor-manager-input.py`.

## Previous standard versions

The list of standard flavors used to be part of the flavor naming standard up until
[version 3](scs-0100-v3-flavor-naming.md). The following changes have been made to
the list in comparison with said standard:

- the flavor names have been turned into recommendations, and
- the properties have been introduced in order to help discoverability.

Note that the flavors with fixed size root disks have all moved to Recommended
with [scs-0100-v3](scs-0100-v3-flavor-naming.md).
This means that they are not a certification requirement any longer,
but we still recommend implementing these for backwards compatibility reasons.
Also in that standard, two flavors with SSD+ root disks have been added, as defined in
[scs-0110-v1-ssd-flavors.md](scs-0110-v1-ssd-flavors.md)
5 changes: 5 additions & 0 deletions Tests/iaas/scs-0100-v3-flavors.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
SCS-Spec:
# Empty lists so flavor-names-openstack.py only checks the names for scs-0100.
# The list of flavors will be checked with flavors-openstack.py for scs-0103.
MandatoryFlavors: []
RecommendedFlavors: []
Loading

0 comments on commit 3baa7f0

Please sign in to comment.