Skip to content

Latest commit

 

History

History
81 lines (56 loc) · 7.15 KB

build-avoidance.md

File metadata and controls

81 lines (56 loc) · 7.15 KB

Build Avoidance Example

This example discusses how build avoidance mechanisms can be implemented using Eiffel events.

Introduction

A common use case in continuous integration and delivery systems is avoidance of unnecessary builds: when a number of components are built from the same source repository, one doesn't want to rebuild all of them simply because something in the repo changed - instead one wants to rebuild only the parts that were affected.1

In a centralized scenario where everything is handled in a single orchestrating job, this is relatively straight forward: analyze the change, trigger the needed builds, wait for them to finish and package the result. In a decentralized continuous integration and delivery system based on descriptive (rather than prescriptive) principles a different approach is needed.

Compared to many other examples, only a very small number of link types is included: ELEMENT, PREVIOUS_VERSION, REUSED_ARTIFACT and COMPOSITION. These four are the types relevant to the example; other link types are of course legal and feasible, but not included here.

A JSON array of all events used in this example can be found here.

Event Graph

alt text

Event-by-Event Explanation

SCS1, SCS2

The EiffelSourceChangeSubmittedEvents signal that source changes have been submitted to a branch of interest (typically a shared development branch or "mainline"). They would normally point to EiffelSourceChangeCreatedEvents documenting the change in greater detail; for the sake of brevity this has been left out of the example.

CDefC1, CDefC2

For each source change, a new EiffelCompositionDefinedEvent is emitted, including it in ELEMENT links. Note that each composition references its predecessor via a PREVIOUS_VERSION link.

ArtCA1, ArtCB1, ArtCC1, ArtCD1, ArtCE1

The EiffelArtifactCreatedEvents representing the set of components (A, B, C, D and E) built from CDefC1. In this example it is assumed that each component is built independently in a decentralized fashion, and that each such build determines whether a new artifact is needed. Strategies for making that decision are discussed below.

ArtCA2, ArtCE2

The EiffelArtifactCreatedEvents representing the two artifacts built from CDefC2. Again, each component is built independently. In this case, artifacts A and E are rebuilt.

ArtRB2, ArtRC2, ArtRD2

The EiffelArtifactReusedEvents declaring that components B, C and D were in fact not rebuilt for composition CDefC2, but that an already existing artifact is logically equivalent and therefore reused. These events use two links to declare this: COMPOSITION and REUSED_ARTIFACT.

CDefS1, CDefS2

There are the EiffelCompositionDefinedEvents defining the larger system composed of components A, B, C, D and E: whenever all components required to be rebuilt for a given source change are available, a new system composition is emitted.

Build Avoidance Solution

As discussed in the introduction, in a centralized scenario where a single actor controls the entire process end-to-end, arranging this type of build avoidance is trivial. In a distributed and heterogeneous scenario it is more complicated, as one actor cannot be assumed to, and indeed should not, know more than its immediate concerns. In other words, the actor defining CDefS1 and CDefS2 does not know who builds or does not build components A, B, C, D or E or on which basis that decision is made. Conversely, the builders of the components do not presume to know who will be using them: they are ignorant of the system level, and may indeed be included in any number of downstream compositions. How then does one determine when to issue a new EiffelCompositionDefinedEvent and which component artifacts to include in it?

Facing this scenario it is easy to turn to either prescriptive solutions (e.g. CDefC1 and CDefC2 instructing downstream actors what to do) or convention based solutions (e.g. B is usually wrapped in Activity events, so a set of Activity events without an EiffelArtifactCreatedEvent is interpreted as a skipped build). The Eiffel protocol supports this scenario through EiffelArtifactReusedEvents, however.

Whenever a component builder creates a new artifact it issues an EiffelArtifactCreatedEvent, but when it decides not to it instead issues an EiffelArtifactReusedEvent. This allows the system level to follow a single very simple rule: whenever a relevant EiffelCompositionDefinedEvent has been linked to by either an EiffelArtifactCreatedEvent or an EiffelArtifactReusedEvents for each component, define a new system level composition.

A Note on Link Traversal

If one wishes to find changes that were included in a given system revision in this scenario, there are two options which produce different results.

If one is only interested in what is new in the given system revision (e.g. finding out which issues have been addressed since the previous version) one can follow ELEMENT and COMPOSITION links to any source changes.

If instead one wants to find all source revisions included in the system revision, regardless of whether they are new or old, one also includes any REUSED_ARTIFACT links in the query.

   


 

1: Some will argue that this situation is to be avoided, and that a better option is to split the source into multiple repositories. While not arguing the point, this may or may not be a feasible option, and so one makes the best of the cards one is dealt.