Skip to content

Commit

Permalink
An example of custom sampling condition (#211)
Browse files Browse the repository at this point in the history
  • Loading branch information
knopers8 authored and Barthelemy committed Jul 19, 2019
1 parent cce0862 commit 4f9ff2f
Show file tree
Hide file tree
Showing 6 changed files with 100 additions and 3 deletions.
9 changes: 8 additions & 1 deletion Framework/example-default.json
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,14 @@
}
],
"subSpec": "0",
"samplingConditions": []
"samplingConditions": [
{
"condition": "custom",
"moduleName": "QcExample",
"className": "o2::quality_control_modules::example::ExampleCondition",
"threshold": "120"
}
]
},
{
"id": "mftclusters",
Expand Down
3 changes: 2 additions & 1 deletion Modules/Example/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ---- Library ----

add_library(QcExample src/BenchmarkTask.cxx src/ExampleTask.cxx
src/FakeCheck.cxx)
src/FakeCheck.cxx src/ExampleCondition.cxx)

target_include_directories(
QcExample
Expand All @@ -15,6 +15,7 @@ add_root_dictionary(QcExample
HEADERS include/Example/BenchmarkTask.h
include/Example/ExampleTask.h
include/Example/FakeCheck.h
include/Example/ExampleCondition.h
LINKDEF include/Example/LinkDef.h
BASENAME QcExample)

Expand Down
50 changes: 50 additions & 0 deletions Modules/Example/include/Example/ExampleCondition.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
// Copyright CERN and copyright holders of ALICE O2. This software is
// distributed under the terms of the GNU General Public License v3 (GPL
// Version 3), copied verbatim in the file "COPYING".
//
// See http://alice-o2.web.cern.ch/license for full licensing information.
//
// In applying this license CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.

///
/// \file ExampleCondition.h
/// \author Piotr Konopka
///

#ifndef QC_MODULE_EXAMPLE_EXAMPLECONDITION_H
#define QC_MODULE_EXAMPLE_EXAMPLECONDITION_H

#include "QualityControl/TaskInterface.h"
#include <Framework/DataSamplingCondition.h>
#include <TROOT.h>

namespace o2::quality_control_modules::example
{

/// \brief Example of custom Data Sampling Condition
/// \author Piotr Konopka

/// \brief A DataSamplingCondition which approves messages which have their first byte in the payload higher than
/// specified value.
class ExampleCondition : public o2::framework::DataSamplingCondition
{
public:
/// \brief Constructor.
ExampleCondition() : DataSamplingCondition(){};
/// \brief Default destructor
~ExampleCondition() override = default;

/// \brief Reads 'threshold'
void configure(const boost::property_tree::ptree& config) override;
/// \brief Makes a positive decision if first byte is higher than 'threshold'
bool decide(const o2::framework::DataRef& dataRef) override;

private:
uint8_t mThreshold;
};

} // namespace o2::quality_control_modules::example

#endif //QC_MODULE_EXAMPLE_EXAMPLECONDITION_H
2 changes: 2 additions & 0 deletions Modules/Example/include/Example/LinkDef.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@
#pragma link C++ class o2::quality_control_modules::example::ExampleTask + ;
#pragma link C++ class o2::quality_control_modules::example::FakeCheck + ;
#pragma link C++ class o2::quality_control_modules::example::BenchmarkTask + ;
#pragma link C++ class o2::quality_control_modules::example::ExampleCondition + ;

#endif
31 changes: 31 additions & 0 deletions Modules/Example/src/ExampleCondition.cxx
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
// Copyright CERN and copyright holders of ALICE O2. This software is
// distributed under the terms of the GNU General Public License v3 (GPL
// Version 3), copied verbatim in the file "COPYING".
//
// See http://alice-o2.web.cern.ch/license for full licensing information.
//
// In applying this license CERN does not waive the privileges and immunities
// granted to it by virtue of its status as an Intergovernmental Organization
// or submit itself to any jurisdiction.

///
/// \file ExampleCondition.cxx
/// \author Piotr Konopka
///

#include "Example/ExampleCondition.h"

namespace o2::quality_control_modules::example
{

void ExampleCondition::configure(const boost::property_tree::ptree& config)
{
mThreshold = config.get<uint8_t>("threshold");
}

bool ExampleCondition::decide(const o2::framework::DataRef& dataRef)
{
return dataRef.payload != nullptr ? dataRef.payload[0] > mThreshold : false;
}

} // namespace o2::quality_control_modules::example
8 changes: 7 additions & 1 deletion doc/ModulesDevelopment.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ An example of a workflow definition which describes the processing steps (_Data

### Data Sampling

The Data Sampling provides the possibility to sample data in DPL workflows, based on certain conditions ( 5% randomly, when payload is greater than 4234 bytes, etc.). The job of passing the right data is done by a data processor called `Dispatcher`. A desired data stream is specified in the form of Data Sampling Policies, defined in the QC JSON configuration file. Please refer to the main [Data Sampling readme](https://github.com/AliceO2Group/AliceO2/blob/dev/Framework/Core/README.md#data-sampling) for more details.
The Data Sampling provides the possibility to sample data in DPL workflows, based on certain conditions ( 5% randomly, when payload is greater than 4234 bytes or others, including custom conditions). The job of passing the right data is done by a data processor called `Dispatcher`. A desired data stream is specified in the form of Data Sampling Policies, defined in the QC JSON configuration file. Please refer to the main [Data Sampling readme](https://github.com/AliceO2Group/AliceO2/blob/dev/Framework/Core/README.md#data-sampling) for more details.

Data Sampling is used by Quality Control to feed the tasks with data. Below we present an example of a configuration file. It instructs Data Sampling to provide a QC task with 10% randomly selected data that has the header `{"ITS", "RAWDATA", 0}`. The data will be accessible inside the QC task by the binding `"raw"`.
```json
Expand Down Expand Up @@ -90,6 +90,12 @@ Data Sampling is used by Quality Control to feed the tasks with data. Below we p

An example of using the data sampling in a DPL workflow is visible in [runAdvanced.cxx](https://github.com/AliceO2Group/QualityControl/blob/master/Framework/runAdvanced.cxx).

#### Custom Data Sampling Condition

If needed, a custom data selection can be performed by inheriting the `DataSamplingCondition` class and implementing the `configure` and `decide` methods. Then, to use it, one needs to specify the library and class names in the config file.

The class [ExampleCondition](https://github.com/AliceO2Group/QualityControl/blob/master/Modules/Example/include/Example/ExampleCondition.h) presents the how to write one's own condition, while in [example-default.json](https://github.com/AliceO2Group/QualityControl/blob/master/Framework/example-default.json) the policy `ex1` shows how it should be configured.

#### Bypassing the Data Sampling

In case one needs to sample at a very high rate, or even monitor 100% of the data, the Data Sampling can be omitted altogether. As a result the task is connected directly to the the Device producing the data to be monitored. To do so, change the _dataSource's_ type in the config file from `dataSamplingPolicy` to `direct`. In addition, add the information about the type of data that is expected (dataOrigin, binding, etc...) and remove the dataSamplingPolicies :
Expand Down

0 comments on commit 4f9ff2f

Please sign in to comment.