Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of TRestDataSet #345

Merged
merged 44 commits into from
Dec 13, 2022
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f968878
TRestDataSet metadata class added
vicctorps May 18, 2022
274782b
dset.rml added RML TRestDataSet example
vicctorps May 18, 2022
6c7208e
TRestDataSet::SiderealTime added
vicctorps May 25, 2022
f33bb65
dset.rml fixing minor bug
vicctorps May 25, 2022
1857613
dset.rml renamed to dataset.rml
vicctorps May 25, 2022
a8f42f2
TRestDataSet fixing compilation
vicctorps May 25, 2022
459d43d
TRestDataSet updates
vicctorps Jun 9, 2022
3e0ab07
Merge branch 'master' into vicctorps_dataset
jgalan Sep 14, 2022
ec0b9ee
Merge branch 'master' into vicctorps_dataset
jgalan Sep 14, 2022
7625cea
Merge branch 'master' into vicctorps_dataset
jgalan Nov 25, 2022
6f9279d
TRestDataSet. Fixing compilation issues
jgalan Nov 25, 2022
14b7c79
Reviewing TRestDataSet
jgalan Nov 29, 2022
8ce86f2
Merge branch 'jgalan_units_fix' into vicctorps_dataset
jgalan Nov 29, 2022
7ece330
TRestDataSet implemented metadata filters
jgalan Nov 30, 2022
c2677b0
TRestMetadata::GetParameter will return empty instead of
jgalan Nov 30, 2022
684c20b
TRestMetadataPlot. Replacing TString by std::string to use .empty()
jgalan Nov 30, 2022
6e4de7f
TRestDataSet. Adding observables list
jgalan Dec 1, 2022
0744f12
Merge branch 'master' into vicctorps_dataset
jgalan Dec 2, 2022
7f1fb5f
TRestDataSet::PrintMetadata now prints out metadata filters
jgalan Dec 7, 2022
8ba6de4
TRestAnalysisTree. Refurbished GetObservableNames method
jgalan Dec 7, 2022
cb2a3d7
TRestDataSet. Implemented initialization of DataFrame and TTree
jgalan Dec 7, 2022
fb5d84a
TRestDataSet::Export method has been implemented
jgalan Dec 8, 2022
f39851d
TRestDataSet::Export adding header to csv format
jgalan Dec 8, 2022
e3fbbea
Adding dataset.rml example
jgalan Dec 8, 2022
356c759
Revert "TRestMetadataPlot. Replacing TString by std::string to use .e…
jgalan Nov 30, 2022
c064c1a
Revert "TRestMetadata::GetParameter will return empty instead of"
jgalan Dec 8, 2022
bf8ac30
TRestDataSet. Adding documentation
jgalan Dec 8, 2022
bfb8963
TRestDataSet. Trying to fix indenting issue
jgalan Dec 8, 2022
deef9a1
TRestDataSet. Trying to fix indenting issue
jgalan Dec 8, 2022
35cca0a
TRestDataSet. Fixing typos on documentation
jgalan Dec 8, 2022
a95ea39
TRestDataSet::fTotalDuration added to keep control of the accumulated…
jgalan Dec 9, 2022
4709494
use nullptr
lobis Dec 9, 2022
2659311
made methods `const` when possible, unified style with framework (imp…
lobis Dec 9, 2022
12b1c49
TRestDataSet. Removing new/delete scheme
jgalan Dec 10, 2022
dc930eb
Merge branch 'jgalan_dataset' of github.com:rest-for-physics/framewor…
jgalan Dec 10, 2022
f7b3527
dataset.rml Updating dataset example
jgalan Dec 10, 2022
7426c5c
TRestDataSet. Implementing quantity metadata members
jgalan Dec 10, 2022
ff87945
TRestDataSet. Quantity data member is now encapsulated inside Relevan…
jgalan Dec 10, 2022
4945af4
TRestDataSet::GetFileSelection method added
jgalan Dec 10, 2022
6973bea
TRestDataSet. Implemeting readout of relevant quantities
jgalan Dec 10, 2022
15222f3
TRestDataSet. Adding relevant quantities documentation
jgalan Dec 10, 2022
467a25c
TRestDataSet::Export. Txt output now includes relevant quantities
jgalan Dec 10, 2022
b388349
TRestDataSet. Now observables separator is a comma
jgalan Dec 12, 2022
17229cc
Merge branch 'master' into jgalan_dataset
jgalan Dec 13, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions examples/dataset.rml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<?xml version="1.0"?>
<TRestDataSet name="BabyIAXO">
<parameter name="startTime" value="2022/04/28 00:00"/>
<parameter name="endTime" value="2022/11/25 13:20"/>
<parameter name="filePattern" value="/home/jgalan/rest-framework/source/libraries/axion/examples/full-ray-tracing/BabyIAXO_TrueWolterMicromegasTest*.root"/>
<filter metadata="TRestRun::fRunNumber" greaterThan="370" lowerThan="375"/>
<filter metadata="TRestRun::fRunTag" contains="Test"/>
<observables list="window_transmission:optics_efficiency:axionPhoton_probability:axionPhoton_transmission:boreExitGate_transmission"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why use : as the separator? If it's correlated with a vector we can use rml auto parsing, like: {window_transmission,optics_efficiency,axionPhoton_probability,axionPhoton_transmission,boreExitGate_transmission}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, , separator is probably much better. I thought about it, perhaps we could use a Split automatic separator identifier?

Such as we find on string, if we find only ,, we use it.

We could use a wildcard Split separator to try to find the separator.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just updated it at b388349

<processObservables list="final"/>
</TRestDataSet>
2 changes: 1 addition & 1 deletion source/framework/core/inc/TRestAnalysisTree.h
Original file line number Diff line number Diff line change
Expand Up @@ -266,7 +266,7 @@ class TRestAnalysisTree : public TTree {
Bool_t EvaluateCuts(const std::string& expression);
Bool_t EvaluateCut(const std::string& expression);

TString GetStringWithObservableNames();
std::vector<std::string> GetObservableNames();

std::vector<std::string> GetCutObservables(const std::string& cut_str);

Expand Down
103 changes: 103 additions & 0 deletions source/framework/core/inc/TRestDataSet.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
/*************************************************************************
* This file is part of the REST software framework. *
* *
* Copyright (C) 2016 GIFNA/TREX (University of Zaragoza) *
* For more information see https://gifna.unizar.es/trex *
* *
* REST is free software: you can redistribute it and/or modify *
* it under the terms of the GNU General Public License as published by *
* the Free Software Foundation, either version 3 of the License, or *
* (at your option) any later version. *
* *
* REST is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* GNU General Public License for more details. *
* *
* You should have a copy of the GNU General Public License along with *
* REST in $REST_PATH/LICENSE. *
* If not, see https://www.gnu.org/licenses/. *
* For the list of contributors see $REST_PATH/CREDITS. *
*************************************************************************/

#ifndef REST_TRestDataSet
#define REST_TRestDataSet

#include "ROOT/RDataFrame.hxx"
#include "TRestMetadata.h"
#include "TTimeStamp.h"

/// It allows to group a number of runs that satisfy given metadata conditions
class TRestDataSet : public TRestMetadata {
private:
/// All the selected runs will have a starting date after fStartTime
std::string fStartTime = "2000/01/01"; //<

/// All the selected runs will have an ending date before fEndTime
std::string fEndTime = "3000/12/31"; //<

/// A glob file pattern that must be satisfied by all files
std::string fFilePattern = ""; //<

/// It contains a list of the observables that will be added to the final tree or exported file
std::vector<std::string> fObservablesList; //<

/// It contains a list of the process where all observables should be added
std::vector<std::string> fProcessObservablesList; //<

/// A list of metadata members where filters will be applied
std::vector<std::string> fFilterMetadata; //<

/// If not empty it will check if the metadata member contains the value
std::vector<std::string> fFilterContains; //<

/// If the corresponding element is not empty it will check if the metadata member is greater
std::vector<Double_t> fFilterGreaterThan; //<

/// If the corresponding element is not empty it will check if the metadata member is lower
std::vector<Double_t> fFilterLowerThan; //<

/// The total integrated run time of selected files
Double_t fTotalDuration = 0; //<

/// The resulting RDataFrame object after initialization
ROOT::RDataFrame fDataSet = 0; //!

/// A pointer to the generated tree
TTree* fTree = 0; //!

/// A list populated by the FileSelection method using the conditions of the dataset
std::vector<std::string> fFileSelection; //!

void InitFromConfigFile() override;

protected:
virtual std::vector<std::string> FileSelection();

public:
ROOT::RDataFrame GetDataFrame() {
jgalan marked this conversation as resolved.
Show resolved Hide resolved
if (fTree == nullptr) RESTWarning << "DataFrame has not been yet initialized" << RESTendl;
return fDataSet;
}

TTree* GetTree() {
if (fTree == nullptr) RESTWarning << "Tree has not been yet initialized" << RESTendl;
return fTree;
}

Int_t GetNumberOfColumns() { return fDataSet.GetColumnNames().size(); }
Int_t GetNumberOfBranches() { return GetNumberOfColumns(); }

Double_t GetTotalTimeInSeconds() { return fTotalDuration; }
jgalan marked this conversation as resolved.
Show resolved Hide resolved

void Export(std::string fname);

void PrintMetadata() override;
void Initialize() override;
TRestDataSet();
TRestDataSet(const char* cfgFileName, std::string name = "");
~TRestDataSet();

ClassDefOverride(TRestDataSet, 1);
};
#endif
15 changes: 8 additions & 7 deletions source/framework/core/src/TRestAnalysisTree.cxx
Original file line number Diff line number Diff line change
Expand Up @@ -1022,18 +1022,19 @@ Double_t TRestAnalysisTree::GetObservableMinimum(const TString& obsName, Double_
}

///////////////////////////////////////////////
/// \brief It returns a string containing all the observables that exist in the analysis tree.
/// \brief It returns a vector with strings containing all the observables that exist in
/// the analysis tree.
///
TString TRestAnalysisTree::GetStringWithObservableNames() {
Int_t nEntries = GetEntries();
std::vector<std::string> TRestAnalysisTree::GetObservableNames() {
std::vector<std::string> names;

// Int_t nEntries = GetEntries();
auto branches = GetListOfBranches();
std::string branchNames = "";
for (int i = 0; i < branches->GetEntries(); i++) {
if (i > 0) branchNames += " ";
branchNames += (string)branches->At(i)->GetName();
names.push_back((string)branches->At(i)->GetName());
}

return (TString)branchNames;
return names;
}

///////////////////////////////////////////////
Expand Down
Loading