Skip to content

Integration Tests

Jump to bottom Edit New page
Mattia Dal Ben edited this page Dec 18, 2023 · 1 revision

Integration tests are implemented by using JUnit to test OSGi services in an environment that is closer to the target one.

The tests developed so far use the tycho-surefire-plugin to start an OSGi runtime as part of the regular Maven build process. The used goal is test, the plugin is configured in Kura test pom and invoked in the test projects (e.g. here).

The behavior of the plugin is the following (quoting from test goal documentation):

Executes tests in an OSGi runtime.

The goal launches an OSGi runtime and executes the project's tests in that runtime. The "test runtime" consists of the bundle built in this project and its transitive dependencies, plus some Equinox and test harness bundles. The bundles are resolved from the target platform of the project. Note that the test runtime does typically not contain the entire target platform. If there are implicitly required bundles (e.g. org.apache.felix.scr to make declarative services work), they need to be added manually through an extraRequirements configuration on the target-platform-configuration plugin.

The bundle containing the tests must use the eclipse-test-plugin packaging type, and it will be an OSGi bundle itself. The test project has the same structure as the usual OSGi bundle projects developed by tycho.

Sources for integration tests must be placed in the src/main/java folder instead of src/test/java.

White box and black box testing

Both white box testing and black box testing is possible using the tycho-surefire-plugin:

  • Black box testing: The test bundle interacts with the services under test only through public APIs (for example the ones defined in the org.eclipse.kura.api bundle) and their configuration (e.g through the ConfigurationService).

  • White box testing: The test bundle inteacts with private APIs of the services under test.
    This can be acheived by turning the test bundle into a fragment of the bundle that provides the services under test by adding a Fragment-Host directive in the MANIFEST.MF of the test bundle. This allows the test bundle to access all of the Java packages included in the bundle under test regardless of whether they are exported or not. Access to package private items is possible as well by placing the test classes in the same Java package as the required item.

The preferred approach for integration tests is usually black box testing, if this approach is implemented, it is usually not necessary to change the tests after a refactor of the code under test.
If changes of the test is needed after the refactor, it is likely that the refactor indroduced API/behavior breakage.

Service binding

Integration tests usually interact with the components under test at the OSGi service level. Test bundle will probably want to create/track some services in the runtime. Currently we have two approaches for achieving this:

Declarative Services Approach

Using this approach, service tracking is demanded to Declarative Services. The test class will be both a JUnit test class and a Declarative Services component implementation. Example: ScriptFilterTest.

The test class will contain the following items:

  • JUnit items: The class will contain methods annotated with JUnit annotations like @Test, @Before, @BeforeClass etc.

  • Declarative Services items: Since the test class is also a Declarative Service component implementation, it can contain activate/deactivate/update methods (e.g. 1) and service bind/unbind methods (e.g 1 2 3) That can be used for binding the required services using DS references.

The test bundle must contain a Declarative Services component definition XML file for the test component class (e.g 1), in the same format used for the non-test components.

At runtime, multiple instances of the test class will be created by DS and JUnit.

  • DS will discover the component definition and will try to resolve and instantiate the test class injecting the dependencies in the usual way.

  • Junit will create one instance of the test class per test method and will try to execute the test.

JUnit test execution and DS component instantiation will happen concurrently.

This has the following implications:

  • Since the JUnit test class instances and DS component instance are different, the resources injected in the DS component must be smuggled to the JUnit instances using static fields.

  • Since test execution and DS component creation happens concurrently, some syncronization logic must be manually implemented to ensure that all of the required resources have been injected before that the tests that use them are executed.

See also 1 for an alternative explanation.

Manual service tracking

It is also possible to perform service binding manually without using DS, the ServiceUtil class provides some utility methods for tracking services and create/delete/update configurations using the ConfigurationService. Most of the methods return CompletableFutures that complete when the services are tracked.

The ServiceUtil class is available only in Kura 5/ESF 7 or newer.

If this approach is used, there is no need to write a DS component implementation and use static fields.

An example of this approach is at 1, in this case the @BeforeClass method tracks the ConfigurationService and creates and tracks a Modbus Driver instance. This example does not use ServiceUtil but a similar utility class, because the bundle also targets older Kura/ESF versions.

Adding additional bundles to the runtime

As stated by the goal documentation reported above, not all of the bundles included in the target platform will be included in the test runtime.

The tycho-surefire-plugin will try to resolve the test project by transitively satisfying all of the constraints specified in its MANIFEST.MF file (e.g Import-Package).

It often happens that the test bundle has some implicit dependencies that are not expressed in its MANIFEST.MF file. This happens for example when the test bundle interacts with the service under test only with the public APIs included in the org.eclipse.kura.api bundle. In this case the test bundle will import some packages from the org.eclipse.kura.api bundle, and this will cause the tycho-surefire-plugin to include org.eclipse.kura.api in the test runtime. This is however not enough to have the bundle that provides the implementation of the service under test to be started.

For example, a bundle that tests the Wire Script filter will using its public APIs will probably import the org.eclipse.kura.wire package from org.eclipse.kura.api in order to be able to interact with the script filter as a wire component. But this will not cause the org.eclipse.kura.wire.script.filter.provider bundle (that contains the implementation of the script filter) to be started as well, since the test bundle does not explicitly depend on it.

This can be fixed at least in the following ways:

  • Add the dependency bundle to the extraRequirements configuration of the tycho target-platform-configuration plugin, in the test parent pom. If this is done, the added plugin will be started in the test runtime of all of the child projects, potentially increasing the build time.

  • Add a Require-Bundle directive for the dependency bundle in the test bundle MANIFEST.MF. The Fragment-Host directive will also count as an explicit dependency. The Require-Bundle directive causes the test bundle to import all of the packages exported by the taget bundle, if the target bundle does not export any package, it may not be possible to use this apporach.

Resource cleanup

All of the test methods are executed in the same test runtime, so if the tests modify the environment for example creating and/or updating configurations, the changes will be retained across different test executions. This must be considered during test design.

Debugging

OSGi console

The test runtime can be configured to spawn an OSGi console accessible using the following command:

telnet localhost 5002

This can be useful for troubleshooting to understand which bundles / DS components are active and if there are unsatified components.

This can be enabled by temporarily uncommenting the following lines:

It may also be useful to add some sleep invocations in test code to stop tests at a specific point.

Java debugger

It is possible to attach Eclipse debugger to the test runtime using a Remote Debug configuration.

In order to enable this, the maven build must be executed enabling the test-debug profile (ESF Kura).

This can be done by building the Maven test project with the following command:

mvn clean install -Ptest-debug

When the test runtime is launched in this way, runtime startup will block until a debugger is connected to port 8000.

User Documentation: https://eclipse-kura.github.io/kura/. Found a problem? Open a new issue or start a discussion.

Getting Started

Developer resources

Kura Release Process

Obsolete
Clone this wiki locally