Injecting Behaviors into Mimic

Behaviors are a way to provide deterministic error injection into Mimic functionality. For instance, mimic will always successfully create a server, immediately. But perhaps you want to see how your service or client handles a server build timing out. Or a 400 response.

Goals

1. Provide an out-of-band method to specify behaviors, so the actual payload to the system under test does not need to change based on whether it is being run against Mimic or a production system.
2. Support different behaviors for different functionality at the same time. For instance, we want to be able to cause server creation 500 failures at the same time as identity authentication 403 failures.
3. Support different behaviors for the same functionality at the same time. For instance, we want to be able to cause server creation 500 failures for some servers, and also server creation 400 failures for some other servers, and also default 202 success for other servers, depending on certain criteria.

Concepts

These explain some terminology and how behaviors are modeled in mimic. Some of these also correspond to objects (in the Python sense) in Mimic, and some only partially correspond to objects in Mimic.

Note that this does not actually cover all the behavior-related objects (in the Python sense) in Mimic. Please see the behavior object model section for that.

Event

An event is the Mimic functionality that we want to declare behaviors for. Examples are: server creation, server deletion, authentication against identity, etc. (Only server creation and authentication against identity are implemented).

mimic.model.behaviors.EventDescription This does not correspond exactly with any object in mimic, but the closest is mimic.model.behaviors.EventDescription. An event would be a global instance of EventDescription - it'd be declared as: server_creation = EventDescription().

An instance of EventDescription has a default behavior, which is the behavior that gets used when there are no injected behaviors specified. This would be successfully creating a server, for instance, in the case of server creation.

It also contains the bijection (a mapping and its reverse) between criteria and behaviors.

Criterion

A pair of (attribute, predicate) to match against to determine whether or not to apply a behavior to an event. By predicate we just mean a callable that returns True or False, given the value for the attribute. For example:

• The server name in the JSON passed to the create server endpoint - maybe we only want to apply the behavior if the server name matches "fail_test.". The criterion would be (server_name, predicate that regexp-matches against "fail_test.").

• The metadata in the JSON passed to the create server endpoint. Maybe we only want to apply the behavior if certain metadata is passed in. So the criterion would be (metadata, predicate that matches the metadata dictionary against a predefined dictionary).

mimic.model.behaviors.Criterion

Criterion takes a name and a predicate. This API is currently in flux - see this issue for more details.

Criteria

A set of criterion to match against to determine whether or not to apply a behavior to an event. All of them have to apply in order for a behavior to apply. Mimic actually always applies criteria (as opposed to a single criterion)

In Mimic, this corresponds to a list of criterion, and no other separate object.

Behavior

A way in which Mimic behaves. This is just a function that does something. Injected behaviors are functions that do something else other than the normal thing.

In Mimic, the normal thing looks like:

@<event>.declare_default_behavior
def do_my_normal_thing():
    ...

The injected behavior looks like:

@<event>.declare_behavior_creator("other_behavior_name")
def do_other_behavior(parameters):
    ...

Object Models

Implementation Guide

This guide assumes that you will be implmenting a control plane and behaviors in multiple steps:

1. The REST endpoints and behavior storage objects
2. The default behavior for a single event
3. Injected behaviors for the same event.
4. Adding additional events

We suggest implementing numbers 1-2 in one PR, number 3 in another PR, and number 4 in other PRs, whether you are adding a control plane to an existing plugin or writing a new plugin.

Assuming that you are modifying an existing plugin:

• For numbers 1-2, the REST endpoint and tests are already templated and provided for you, and do not require very much code, the default behavior wouldn't change any existing behavior, so no additional tests are needed.

• For number 3, when adding new behaviors, tests for those new behaviors can be added.

If you are providing a new plugin at the same time as the control plane, we'd suggest either:

• Implementing the the plugin first, or at least the part you want to provide injection behavior for first, and then adding the control plane.
• Implementing numbers 1 and 2, and only for one event you want to provide a control plane for. Provide tests for the default behavior for that single event. Implement other events in later PRs.

REST Endpoints

Default Behavior

We will be using server creation as an example here. The code is in mimic.model.nova_objects.

1. Declare an event for which behaviors apply:

   server_creation = EventDescription()

2. Define a default behavior for the event. This is just a decorated callable (the decorator makes use of the previously declared server_creation) which take arbitrary parameters - the parameters it requires is defined by the code which uses it (see the next step).

   @server_creation.declare_default_behavior
   def default_create_behavior(<arbitrary_params>):
       # create server code
       # set response code to 202
       # return server JSON
   

   The pseudocode may not be the actual behavior - maybe it just creates the server and returns a server, and the handler sets the response code and returns the server JSON isntead. But this is the general idea.

3. In the code that normally handles the event (such as creating a server), call the behavior function.

   def handle_server_creation(..., behavior_registry_collection, ...):
       server_creation_behavior_registry = (
           behavior_registry_collection.registry_by_event(server_creation))
       behavior_to_apply = (
           server_creation_behavior_registry.behavior_for_attributes({}))
       return behavior_to_apply(<arbitrary_parameters>)
   

   If you are modifying an existing plugin, this function probably previously looked like:

   def handle_server_creation(...):
       # create server code
       # set response code to 202
       # return server JSON
   

   Note that this looks exactly like the previous step's default_create_behavior. That's because the easiest way to create a default behavior is probably to move all the existing code to that function, unless you want to factor some code out for injected error behavior

Injected Behaviors

Adding Additional Events