Skip to content

Unit Test Guidelines and Recipes

Jump to bottom Edit New page
Mattia Dal Ben edited this page Dec 18, 2023 · 2 revisions

This document is a guideline for writing unit tests. If you are in a particular situation where these guidelines don't apply feel free to veer away from them. (Maybe contribute a new recipe?)

Before moving on be sure to read our Gherkin Tests Guidelines. All our tests should follow that format.

Rules

Rule 1: Make sure your tests are clear and easy to read.

Rule 2: If you need to do a hack to make your tests work please refer to rule 1.

Guideline 1: Test case and method names

Rule: Use the following guidelines for test case naming and method/function names.

Test case names

@Test
[methodName]Should[Action]With/When[Conditions]()

## Example ##
@Test
applyShouldNotDisableLoopbackDeviceWithOldVersionOfNM()

givens statements

given[Something]With(params);

## Example ##
givenMockedDeviceWith("1-5", "ttyACM17", NMDeviceType.NM_DEVICE_TYPE_MODEM);
givenNetworkConfigMapWith("net.interfaces", "1-5,");

whens statements

when[methodName]IsCalled[With]();

## Example ##
whenApplyIsCalledWith(this.netConfig);
whenBuildIpv4SettingsIsCalledWith(this.networkProperties, "wlan0");
whenGetInterfacesIsCalled();

thens statements

Due to the wide range of potential "then" methods, it is difficult to express a general guideline.

then[methodName]IsCalled();
then[Action]Occurs();
thenValueIs(“value“);
thenNoExecptionsOccured();

## Example ##
thenNoExceptionWasThrown();
thenNullPointerExceptionWasThrown();
thenNetworkSettingsDidNotChangeForDevice("eth0");

Motivation

  • Test case names, methods and functions should read like English.

  • Any non-technical person should be able to read a test and have a vague understanding of how they work.

  • We should have a consistent naming convention to make it easier for us to understand at a glance what the unit test is doing.

Note: the max method name is 65535+ characters, so do not be afraid to make the method names long. Self-documenting code.

Examples

@Test
public void applyShouldDisableGPSWithMissingGPSConfiguration() throws DBusException, IOException {
    givenBasicMockedDbusConnector();
    givenMockedDeviceWith("1-5", "ttyACM17", NMDeviceType.NM_DEVICE_TYPE_MODEM, NMDeviceState.NM_DEVICE_STATE_ACTIVATED,
            true, false, false);
    givenMockedDeviceList();

    givenNetworkConfigMapWith("net.interfaces", "1-5,");
    givenNetworkConfigMapWith("net.interface.1-5.config.ip4.status", "netIPv4StatusEnabledWAN");
    givenNetworkConfigMapWith("net.interface.1-5.config.dhcpClient4.enabled", true);
    givenNetworkConfigMapWith("net.interface.1-5.config.apn", "myAwesomeAPN");

    whenApplyIsCalledWith(this.netConfig);

    thenNoExceptionWasThrown();
    thenConnectionUpdateWasCalledFor("ttyACM17");
    thenActivateConnectionWasCalledFor("ttyACM17");
    thenLocationSetupWasCalledWith(EnumSet.of(MMModemLocationSource.MM_MODEM_LOCATION_SOURCE_3GPP_LAC_CI), false);
}

Guideline 2: Prefer static data in expected values

Rule: Prefer static data in expected values.

Motivation:

  • If we replicate the code logic in our tests we would need unit tests for our unit tests.

  • The unit tests should be our single source of truth about the expected output given an input.

  • One exception might be for alternative implementations: if you have a method that computes the inverse square root of a real number, you might want to test it against a known good implementation of the inverse square root. I.e. there’s a single source of truth for the expected values, which is the output of the known good implementation.

Examples

public int add(int a, int b) {
    return a + b;
}

@Test
public void addShouldWorkWithTwoIntegers() {
    whenAddIsCalledWith(2, 2);
    
    // thenResultEquals(2+2); <- WRONG! It's like calling add(2, 2)!
    thenResultEquals(4);
}

Guideline 3: Build reusable and easily readable given methods

Rule: Instead of having gigantic givens method without any parameter, try and parametrise these methods in such a way that the parameters of the given methods match the inputs of the test case.

Motivation:

  • Assigning static values in method invocation as parameters are preferred, so when reading the tests, you understand exactly what happens. This makes tests more flexible, and easier to understand.

  • They are more flexible

  • Parameterisation + Reuse > Magic Methods

  • Promotes code reuse

  • Better understandability/readability of what are the inputs of a test case

Examples

Link to the actual code: https://github.com/eclipse/kura/blob/7a54e07345b680e9df85bc53c76a6e0ef0416dd1/kura/test/org.eclipse.kura.nm.test/src/test/java/org/eclipse/kura/nm/NMDbusConnectorTest.java

class networkImplTest{
  @Test
  void testWifiBadExample(){
    givenNetworkWifiAdapterWithNetworkAndWanSupport();
  }
  
  @Test
  void testWifiGoodExample(){
    givenNetworkAdapterWith("wlan0", Type.Wifi, EnabledForWan);
  }
  
  @Test
  void EthernetBadExample(){
    givenNetworkEthernetAdapterWithNetworkAndWanSupport();
  }
  
  @Test
  void EthernetGoodExampleWithMultipleDevices(){
    givenNetworkAdapterWith("eth0", Type.Ethernet, EnabledForWan);
    givenNetworkAdapterWith("eth1", Type.Ethernet, EnabledForWan);
    givenNetworkAdapterWith("wlan0", Type.Wifi, Disabled);
  }
}

Notice how the given methods can be used across multiple test cases, and keep the document extremely clean and readable.

Guideline 4: Rely on JUnit resetting variables

Rule: Rely on JUnit cleaning mechanism.

Motivation:

  • Rely on JUnit cleaning mechanism.

    1. Instance variables in a Class will be cleaned/reset at the start of every test.

Reference:

Examples

public class negativeExampleTest {

    private Map<String, Object> properties;
    private Options options;
    private Object returnValue;

    @Test
    public void shouldReturnEnabled() {
        givenProperty("something", true);
        givenOptionsInstatiated();

        whenIsEnabled();

        thenReturnValueIs(true);
    }

    @Before
    public void cleanUp() { // WRONG! This is not needed!
        this.properties = new HashMap<>();
        this.options = null;
        this.returnValue = null;
    }
}

Guideline 5: Test the code we wrote, not the underlying libraries

Rule: Mock the underlying libraries so that we can test the interfacing code

Motivation:

  • Think about the goal of what you are testing. Remember to test the code we wrote, not the underlying libraries. Never be afraid to MOCK!

  • Sometimes can be hard to understand where our code ends and the underlying library code starts. Keep in mind that we should test our code not others. We rely on the fact that libraries are working as expected.

  • The line between integration tests and unit tests

  • We mock the libraries so that we can appropriately test how our code interfaces with them.

Examples

When interfacing with an external service through a library (eg. NetworkManager via dbus-java, Docker via Docker-java) you might be tempted to test the entire system. This is wrong: you want to test the code that interfaces with the library, not the library itself.

Examples:

Guideline 6: Do not call mocked objects methods in the tests

Rule: Do not call the methods of the objects you mocked in the tests. These methods should only be called by the production code you’re testing

Motivation:

  • What if an expectation depends on a method being called only once? We’re changing the results by inspecting them.

  • This introduces side-effects to your methods

Examples:

Recipes

Strategies you can directly apply to your unit tests.

Recipe 1: Singleton testing recipe

When testing a Singleton class use reflection to provide that class with any required mocked libraries.

Please check out the Example Here.

Recipe 2: Parametric Testing

One way to increase the feature coverage of a class is to develop tests that cover as many paths or uses as possible. Since it is not possible or efficient to replicate a test changing only the inputs or some internal variable, the parametric testing can be used to increase the code/feature coverage in an easy way. A parametric test is a test that is executed multiple times with different parameters.

An example of this feature can be found below, extracted from the https://github.com/eclipse/kura/blob/develop/kura/test/org.eclipse.kura.nm.test/src/test/java/org/eclipse/kura/nm/MMSimTypeTest.java . The test is compatible with JUnit4.

@RunWith(Parameterized.class)
public static class MMSimTypeToMMSimTypeTest {

    @Parameters
    public static Collection<Object[]> SimTypeParams() {
        List<Object[]> params = new ArrayList<>();
        params.add(new Object[] { new UInt32(0x00), MMSimType.MM_SIM_TYPE_UNKNOWN });
        params.add(new Object[] { new UInt32(0x01), MMSimType.MM_SIM_TYPE_PHYSICAL });
        params.add(new Object[] { new UInt32(0x02), MMSimType.MM_SIM_TYPE_ESIM });
        params.add(new Object[] { new UInt32(0x14), MMSimType.MM_SIM_TYPE_UNKNOWN });
        return params;
    }

    private final UInt32 inputIntValue;
    private final MMSimType expectedSimType;
    private MMSimType calculatedSimType;

    public MMSimTypeToMMSimTypeTest(UInt32 intValue, MMSimType simType) {
        this.inputIntValue = intValue;
        this.expectedSimType = simType;
    }

    @Test
    public void shouldReturnCorrectMMSImType() {
        whenCalculateMMSimType();
        thenCalculatedMMSimTypeIsCorrect();
    }

    private void whenCalculateMMSimType() {
        this.calculatedSimType = MMSimType.toMMSimType(this.inputIntValue);
    }

    private void thenCalculatedMMSimTypeIsCorrect() {
        assertEquals(this.expectedSimType, this.calculatedSimType);
    }
}

The @RunWith(Parameterized.class) annotation signals that the following test is a parametric one and it should be run multiple times with the parameters described with the @Parameter annotation.

This contains a collection on Object[] with the parameters to be passed to the test constructor (MMSimTypeToMMSimTypeTest(UInt32 intValue, MMSimType simType)).

The test in the test class can follow the usual Gerkin style.

Since the parameters are passed to all the test methods inside the class, it is useful to aggregate multiple test suites (parametric or not) in a single class. To achieve this, the Enclosed or Suite runners can be used. An example is shown below:

@RunWith(Enclosed.class)
public class MMSimTypeTest {

    @RunWith(Parameterized.class)
    public static class MMSimTypeToMMSimTypeTest {

    ...
    }

    @RunWith(Parameterized.class)
    public static class MMSimTypeToSimTypeTest {

        ...
    }

In this case, two (parametric) tests are aggregated together and different sets of parameters are applied to the tests.

Recipe 3: Exceptions handling

There are basically two different approaches, the usual try-catch method and the usage of JUnit APIs. We opted for using the try-catch approach since it is easier to understand and allows to easily check for side effects.

The usual pattern will be as follows:

private Exception occurredException;

...

@Test
public void myTestShouldThrowKuraException() {
    givenX();
    
    whenMyMethodIsCalled();
    
    thenExceptionOccurred(KuraException.class);
    thenDirtyFileWasRemoved();
}

...

private void whenMyMethodIsCalled() {
    try {
        this.testedObject.myMethod();
    } catch (Exception e) {
        this.occurredException = e;
    } 
}

...

private void thenNoExceptionOccurred() {
    String errorMessage = "Empty message";
    if (Objects.nonNull(this.occurredException)) {
        StringWriter sw = new StringWriter();
        this.occurredException.printStackTrace(new PrintWriter(sw));

        errorMessage = String.format("No exception expected, \"%s\" found. Caused by: %s",
                this.occurredException.getClass().getName(), sw.toString());
    }

    assertNull(errorMessage, this.occurredException);
}

private <E extends Exception> void thenExceptionOccurred(Class<E> expectedException) {
	assertNotNull(this.occurredException);
	assertEquals(expectedException.getName(),
	        this.occurredException.getClass().getName());
}

In the example above, we have the then clause that compares for equality of the exceptions class names. Using this approach, when the test fails we have logged the expected output and the actual one directly by JUnit. It is possible to integrate the error message with the method assertEquals(String message, Object o1, Object o2); like, for example, embedding the stack trace.

User Documentation: https://eclipse-kura.github.io/kura/. Found a problem? Open a new issue or start a discussion.

Getting Started

Developer resources

Kura Release Process

Obsolete
Clone this wiki locally