FAQ

What is RRE?

The Rated Ranking Evaluator (RRE) is a search quality evaluation tool which, as the name suggests, evaluates the quality of results coming from a search infrastructure. Shortly, it can be outlined as follows:

• A set of search quality evaluation tools
• A search quality evaluation framework (currently targeting Apache Solr and Elasticsearch, though now you can adapt to your own Search API
• Multi (search) platform
• Written in Java
• It can be used also in non-Java projects
• Licensed under Apache 2.0
• Open to contributions!

What it offers out of the box?

The current implementation provides a rich ecosystem, with

• a rich Domain Model for modelling your evaluation context
• a core module that can be used programmatically in your application
• a Maven Plugin which allow you to inject the evaluation process as part of the build cycle
• a Maven Archetype for creating a project skeleton with everything configured (useful also if you don't use Java as main programming language)
• a Maven Report Plugin for producing different kinds of output
• a server module called RRE Server which allows you to have a real-time updated view of the executed evaluation processes.

...and what if I don't use Java?

You can have your main project(s) written elsewhere in somewhat language, you can have the configuration of the search platform you're using wherever you like. At the same time you should have a Maven (empty) project created with our Maven Archetype, which is pre-configured with all RRE plugins, and properly configured, it will point to your configuration files (which again, can be elsewhere, not under that project). That means in order to run the evaluation process you need on your machine

• Java (8 or later)
• Apache Maven (3.x)

after that, follow the Maven Archetype instructions for creating a skeleton project, configure the project so it will know where the evaluation inputs are, and enjoy.
For any question, feel free to open an issue or contact us!

What are the currently supported metrics?

You can see an updated list on the Evaluation Measures.

How can I implement a custom metric?

Other than the metrics above, which are available as part of the system itself, RRE comes with a kind of mini-framework for building and pluggin-in new metrics. You can information about the metric frameworks here. After the custom metric has been implemented you must inform RRE about it. This process actually depends on the runtime container where RRE is running; for example, if you're using the Maven Plugin, then configuring a new metric is trivial, just add its FQDN in the configuration section of the plugin, in the pom.xml:

<plugin>
    <groupId>io.sease</groupId>
    <artifactId>rre-maven-solr-plugin</artifactId>
    <version>7.1.0</version>
    <configuration>
        ...
        <metrics>
            ....
            <param>my.package.MyCustomMetric</param>
        </metrics>
    </configuration>
   ...
</plugin>

How do I change the port for RRE Server?

Change the default port from 8080 to something else by adding --server.port=8090 to the startup command.