Skip to content

Injecting Behaviors into Mimic

Jump to bottom Edit New page
cyli edited this page Jun 12, 2015 · 12 revisions

Behaviors are a way to provide deterministic error injection into Mimic functionality. For instance, mimic will always successfully create a server, immediately. But perhaps you want to see how your service or client handles a server build timing out. Or a 400 response.

Goals

  1. Provide an out-of-band method to specify behaviors, so the actual payload to the system under test does not need to change based on whether it is being run against Mimic or a production system.
  2. Support different behaviors for different functionality at the same time. For instance, we want to be able to cause server creation 500 failures at the same time as identity authentication 403 failures.
  3. Support different behaviors for the same functionality at the same time. For instance, we want to be able to cause server creation 500 failures for some servers, and also server creation 400 failures for some other servers, and also default 202 success for other servers, depending on certain criteria.

Concepts

These explain some terminology and how behaviors are modeled in mimic. Some of these also correspond to objects (in the Python sense) in Mimic, and some only partially correspond to objects in Mimic.

Note that this does not actually cover all the behavior-related objects (in the Python sense) in Mimic. Please see the behavior storage section for that.

Event

An event is the Mimic functionality that we want to declare behaviors for. Examples are: server creation, server deletion, authentication against identity, etc. (Only server creation and authentication against identity are implemented).

This corresponds what to mimic.model.behaviors.EventDescription. An event would be a global instance of EventDescription - it'd be declared as: server_creation = EventDescription().

An instance of EventDescription has a default behavior, which is the behavior that gets used when there are no injected behaviors specified. This would be successfully creating a server, for instance, in the case of server creation.

It also contains the bijection (a mapping and its reverse) between criteria and behaviors.

Criterion

A pair of (attribute, predicate) to match against to determine whether or not to apply a behavior to an event. By predicate we just mean a callable that returns True or False, given the value for the attribute. For example:

  • The server name in the JSON passed to the create server endpoint - maybe we only want to apply the behavior if the server name matches "fail_test.". The criterion would be (server_name, predicate that regexp-matches against "fail_test.").

  • The metadata in the JSON passed to the create server endpoint. Maybe we only want to apply the behavior if certain metadata is passed in. So the criterion would be (metadata, predicate that matches the metadata dictionary against a predefined dictionary).

mimic.model.behaviors.Criterion

Criterion takes a name and a predicate. This API is currently in flux - see this issue for more details.

Criteria

A set of criterion to match against to determine whether or not to apply a behavior to an event. All of them have to apply in order for a behavior to apply. Mimic actually always applies criteria (as opposed to a single criterion)

In Mimic, this corresponds to a list of criterion, and no other separate object.

Behavior

A way in which Mimic behaves. This is just a function that does something. Injected behaviors are functions that do something else other than the normal thing.

In Mimic, the normal thing looks like:

@event.declare_default_behavior
def do_my_normal_thing(*arbitrary_parameters):
    ...

(see the implementation guide for default behaviors). The injected behavior looks like:

@event.declare_behavior_creator("other_behavior_name")
def create_other_behavior_callable(parameters):
    ...
    def do_other_behavior(*arbitrary_parameters):
        # do something with both parameters and arbitrary_parameters
    return do_other_behavior

(see the implementation guide for injected behaviors for more information)

Behavior storage

This section describes the behavior storage mechanism in Mimic.

BehaviorRegistryCollection

This is the is the top-level store, and contains behaviors for multiple events. It is up to the implementer of the control plane where an instance of this should be stored.

When retrieving a BehaviorRegistry from a BehaviorRegistryCollection, if one does not exist for a particular event, one is created and then returned.

BehaviorRegistry

This contains a instance of a mimic.model.behaviors.EventDescription, which if you'll remember from the Event section contains a bijection of criteria to behaviors.

That bijection is ordered, with criteria looking like:

[Criterion(name="criterion1", predicate=lambda ...),
 Criterion(name="criterion2", predicate=lambda ...),
 Criterion(name="criterion3", predicate=lambda ...)]

When BehaviorRegistry.behavior_for_attributes is called, it takes a dictionary whose keys are some subset of ("criterion1", "criterion2", "criterion3"), and whose values should be evaluatable by the predicates.

It iterates through that mapping/bijection, and the first behavior whose criteria matches the attribute dictionary will be returned.

Uniqueness is not enforced. There could be two behaviors with the same set of criteria, or even the same behavior with the same set of criteria twice. But the first one is always the one that gets returned.

Implementation Guide

This guide assumes that you will be implmenting a control plane and behaviors in multiple steps:

  1. The default behavior for a single event
  2. Injected behaviors for the same event.
  3. The REST endpoints and behavior storage objects
  4. Adding additional events

We suggest implementing the default behavior (1) in one PR, some injected behavior(s) and the REST endpoints in another PR (2-3), and additional events (4) in other PRs, whether you are adding a control plane to an existing plugin or writing a new plugin.

Assuming that you are modifying an existing plugin:

  • The default behavior wouldn't change any existing behavior, so no additional tests are needed.

  • When adding new behaviors, tests for those new behaviors can be added at the same time. The REST endpoint and tests are already templated and provided for you, and do not require very much code. And the templated tests require that there be at least 1 behavior in addition to the default behavior.

If you are providing a new plugin at the same time as the control plane, we'd suggest either:

  • Implementing the the plugin first, or at least the part you want to provide injection behavior for first, and then adding the control plane.
  • Implementing the default behavior from teh start, and only for one event you want to provide a control plane for. Provide tests for the default behavior for that single event. Implement the rest endpoints and additional behaviors in another PR, and then other events in later PRs.

Default Behavior

We will be using server creation as an example here. The code is in mimic.model.nova_objects.

  1. Declare an event for which behaviors apply:

    server_creation = EventDescription()

  2. Define a default behavior for the event. This is just a decorated callable (the decorator makes use of the previously declared server_creation) which take arbitrary parameters - the parameters it requires is defined by the code which uses it (see the next step).

    @server_creation.declare_default_behavior
def default_create_behavior(<arbitrary_params>):
    # create server code
    # set response code to 202
    # return server JSON

    The pseudocode may not be the actual behavior - maybe it just creates the server and returns a server, and the handler sets the response code and returns the server JSON isntead. But this is the general idea.

  3. Create a BehaviorRegistryCollection somewhere in the plugin's code.

    We recommend placing this in your regional session store (bottom level of the session store diagram here), so that there is one set of behaviors per plugin per region. You can also place it on your global plugin session store (second-to-last level of bottom level of the session store diagram here), if you want the behaviors to apply to all regions. Or if your plugin does not support regions.

    The identity behavior collection, for instance, is stored on the global resource object the behavior being injected affects everything).

  4. In the code that normally handles the event (such as creating a server), call the behavior function.

    def handle_server_creation(..., behavior_registry_collection, ...):
    server_creation_behavior_registry = (
        behavior_registry_collection.registry_by_event(server_creation))
    behavior_to_apply = (
        server_creation_behavior_registry.behavior_for_attributes({}))
    return behavior_to_apply(<arbitrary_parameters>)

    This function accepts as a parameter the BehaviorRegistryCollection created in the previous step. It knows that the event it handles is server_creation. From that, it obtains a behavior to apply.

    In this case, we pass behavior_for_attributes an empty attribute dictionary, because we have not defined any criteria yet or any other behaviors - this will get is the default behavior for now. This will change in the next section.

    Note that the above implementation is just a suggestion. For example, if handle_server_creation is an instance method, as it is in Nova, behavior_registry_collection might be an instance attribute instead. Alternately, the function could accept or refer to just a BehaviorRegistry instead of a whole BehaviorRegistryCollection.

    If you are modifying an existing plugin, this function probably previously looked like:

    def handle_server_creation(...):
    # create server code
    # set response code to 202
    # return server JSON

    Note that this looks exactly like the previous step's default_create_behavior. That's because the easiest way to create a default behavior is probably to move all the existing code to that function, unless you want to factor some code out for injected error behavior

Injected Behaviors

REST Endpoints

Adding Additional Events

Add a custom footer
Add a custom sidebar
Clone this wiki locally