The Evaluation Process

The evaluation process is executed by the rre-core module, but needs to be driven by some runtime container. The picture below illustrates the overal process:

Here, you can see that the most part of the process is in charge of the rre-core module. It takes care about (1) starting the target search platform, configuring it (4), loading the corpus (6), iterating over the user-defined ratings (3-9), executing queries (11), computing metrics (12), and finally cleaning up things (13).

Once the process succeeded and the evaluation output (14) has been generated, the runtime container (e.g. Maven Plugin) takes such data and outputs it in other formats (e.g. PDF, spreadsheet or RRE Server).

It's important to underline that RRE doesn't require a running instance of the target search platform. It will take care to internally manage its whole lifecycle as part of the evaluation process. Concretely, if your project is using Solr x.y.z then you just need to configure (in pom.xml) the correct RRE version and nothing else. During the evaluation process, RRE will start/load/query/stop an embedded Solr instance.