Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of TRestDataSet #345

Merged
merged 44 commits into from
Dec 13, 2022
Merged

Implementation of TRestDataSet #345

merged 44 commits into from
Dec 13, 2022

Conversation

jgalan
Copy link
Member

@jgalan jgalan commented Dec 8, 2022

jgalan Large: 819

This PR is related to issue #13.

A class allowing to create datasets from an official REST data repository.

The class documentation has been pre-rendered at the following location: https://sultan.unizar.es/test/classTRestDataSet.html have a look!

RML setup would look as follows:

    <TRestDataSet name="BabyIAXO_Test_DataSet_00">
        <parameter name="startTime" value="2022/04/28 00:00" />
        <parameter name="endTime" value="2022/11/25 13:20" />
        <parameter name="filePattern" value="/home/jgalan/rest-framework/source/libraries/axion/examples/full-ray-tracing/BabyIAXO_TrueWolterMicromegasTest*.root"/>

        <filter metadata="TRestRun::fRunNumber" greaterThan="370" lowerThan="375" />
        <filter metadata="TRestRun::fRunTag" contains="Test" />

        <observables list="window_transmission:optics_efficiency:axionPhoton_probability:axionPhoton_transmission:boreExitGate_transmission" />

        <processObservables list="final" />
    </TRestDataSet>

The class will be capable to generate a final txt file with the selected data, and a header with the origin of the data:

### TRestDataSet generated file
###
### StartTime : 2022/04/28 00:00
### EndTime : 2022/11/25 13:20
###
### Data path : /home/jgalan/rest-framework/source/libraries/axion/examples/full-ray-tracing
### File pattern : BabyIAXO_TrueWolterMicromegasTest*.root
###
### Metadata filters :
### - TRestRun::fRunNumber. Greater than: 370.000. Lower than: 375.000.
### - TRestRun::fRunTag. Contains: Test.
###
### Observables list:  runOrigin eventID timeStamp window_transmission optics_efficiency axionPhoton_probability axionPhoton_transmission boreExitGate_transmission final_energy final_posX final_posY final_posZ final_R final_thetaAngle final_phiAngle
###
### Data starts here
000373  000003  0000000000  9.945e-01   2.828e-01   4.034e-19   1.000e+00   1.000e+00   7.021e+00   5.524e+00   -5.612e-01  1.454e+04   5.553e+00   4.188e-02   1.313e+00
000373  000004  0000000000  0.000e+00   6.521e-01   7.624e-19   1.000e+00   1.000e+00   2.744e+00   2.122e+00   -1.710e+00  1.454e+04   2.726e+00   2.012e-02   2.235e+00

vicctorps and others added 30 commits May 18, 2022 12:45
@lobis
Copy link
Member

lobis commented Dec 9, 2022

I think it's important that unit tests are added for this class, now that we have the possibility to do it from the ground up.

@lobis lobis added the development To define issues with development proposals label Dec 9, 2022
@jgalan
Copy link
Member Author

jgalan commented Dec 10, 2022

I think it's important that unit tests are added for this class, now that we have the possibility to do it from the ground up.

I agree, just waiting to see what happens with the files I prepared for testing the class.

See https://root-forum.cern.ch/t/checkbytecount-and-streamer-error/52716/7

<parameter name="filePattern" value="../../framework-data/pipeline/dataset/BabyIAXO_TrueWolterMicromegasTest*.root"/>
<filter metadata="TRestRun::fRunNumber" greaterThan="370" lowerThan="375"/>
<filter metadata="TRestRun::fRunTag" contains="Test"/>
<observables list="window_transmission:optics_efficiency:axionPhoton_probability:axionPhoton_transmission:boreExitGate_transmission"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why use : as the separator? If it's correlated with a vector we can use rml auto parsing, like: {window_transmission,optics_efficiency,axionPhoton_probability,axionPhoton_transmission,boreExitGate_transmission}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, , separator is probably much better. I thought about it, perhaps we could use a Split automatic separator identifier?

Such as we find on string, if we find only ,, we use it.

We could use a wildcard Split separator to try to find the separator.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just updated it at b388349

@jgalan jgalan merged commit 7877c37 into master Dec 13, 2022
@jgalan jgalan deleted the jgalan_dataset branch December 13, 2022 21:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development To define issues with development proposals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants