RML 2.0

[Atraxus]

To properly support RAYX's new features, we are looking into creating a new format to load and save beamlines with.

• Example rml 2.0 file

[memoryleak47]

Here is a starting point on how we could do our beamline format. Instead of .rml we would then have a .toml suffix.

# we need the elements to be ordered, as we use their element-id all over the place.
# Thus we use an array of elements using `[[element]]`

[[source]]
name = "a-pointy-source"
numberRays = 200
type = "PointSource"

[source.distribution]
type = "gaussian"
energy = 100 # in eV? or do we want to be explicit about units?
energySpread = 30

[[element]]
name = "miroir"
# We store the out-transform to express position & orientation.
# I prefer having a singular transform instead, as this answers the question in which order we apply position and orientation.
# We can easily calculate the in-transform by inverting this matrix.
transform = [
    [1.0, 0.0, 0.0, 0.0],
    [0.0, 1.0, 0.0, 0.0],
    [0.0, 0.0, 1.0, 0.0],
    [0.0, 0.0, 0.0, 1.0],
]

[element.behaviour]
type = "Mirror"
material = "Au"

[element.surface]
type = "Plane"

[element.cutout]
type = "Rectangle"
width = 20.0
length = 20.0

[[element]]
name = "the-imaginary-plane"
transform = [
    [1.0, 0.0, 0.0, 0.0],
    [0.0, 1.0, 0.0, 0.0],
    [0.0, 0.0, 1.0, 100.0],
    [0.0, 0.0, 0.0, 1.0],
]

# this is a shorter notation for tables.
behaviour = { type = "ImagePlane" }
surface = { type = "Plane" }
cutout = { type = "Unlimited" }

My thoughts about this:
We need to use [[element]] so that the elements are stored in an array. We need this, as we mostly refer to elements by their index. And everything that's not in an array is unordered in toml.

If we wanted to have objects that are both light-sources and optical-elements, or if we wanted to have concrete indices to reference both light-sources or optical-elements (eg. the Ray::m_lastElement field might want that), we would need to use [[element]] even for light-sources too. In the model I proposed there is no defined order between the light-sources and the elements, as they are stored in different arrays.

[memoryleak47]

We made some more progress on our new TOML-based format, In the following I want to document our thought process and our decisions.

1. We write [[object]] instead of [[source]] or [[element]] so that the toml serializer won't put the light-sources after the elements.
   Also, @Atraxus is looking for order- (and comment-) preserving toml-libraries to solve these kinds of problems generally.

2. About multiple calculation methods:
   Generally, we want to provide the user to express the parameters in a way that is intuitive to the user - we want to allow for design parameters.
   Hence all of the following ways might be valid methods to define eg. the transform

• transform = [...] # the matrix is given explicitly
• transform = { position = [...], orientation = [...] } # transform is derived from position and orientation
• transform = { distanceToPreceding = 1000, azimuthalAngle = ... }

One big problem with the current RML model is that it's implicit which of the three calculation-method is used,
because it just lists position, orientation, distanceToPreceding, azimuthalAngle and all others - so we don't know which of them the parser actually uses, and which it discards.

Thus, our first thought was making this explicit by writing the required parameters as tables inside of transform (as seen above)

The problem with defining stuff like this is that it is possible that multiple params depend on the same design parameters.

transform = { position = [...], orientation = [..] }
other-param = { position = [...], ...}

This would effectively force us to write the position twice (and thus create redundancy and sources of errors & confusion).

Thus, we decided to use "calculation-strings" instead of nested tables:

transform = "by-position-and-orientation"
other-param = "by-position"
position = [...]
orientation = [...]

This is a simple example file for the new state:

[[object]]
name = "MatrixSource1"
type = "Matrix Source"
numberRays = 100
transform = "by-position-orientation"
position = [...]
orientation = [...]

distribution = "gaussian"
energy = 100 # in eV? or do we want to be explicit about units?
energySpread = 30

I'll now work on separating the current DesignObject from the XML parser so that we can use it for toml as well.

[memoryleak47]

Before we can reasonably implement a parser for a new format, it makes - in my opinion - sense to have a "DesignElement" definition. A "DesignElement" is like an Element, but with all the design parameters still in there.

Because, if both the RML and the TOML parser simply produce "Elements" as outputs, both of these parsers need to resolve all of the the design parameters separately. This would cause a huge amount of code duplication.

Hence, I claim that it makes more sense that both parsers produce a DesignElement as intermediate output.
And then there is a ".resolveDesignParams()" function that converts the DesignElement to Element.

The relevant issue for this DesignElement idea should be #227.

I'm hence closing this issue for now, as we first need to solve the DesignElement problem, which itself might take some time.

[Atraxus]

We can continue this discussion since the design element has been implemented (#227), which adds an abstraction in the code base to the parameters in the rml files.

[Atraxus]

Updated Work Outline:

1. Add group support to the core library (Implement Groups for the Core Library #301).
2. Implement an interface to support both the XML and TOML parsers (this may involve simply updating the importBeamline function).
3. Implement the TOML parser.

Additionally, we might want to reconsider using JSON. While TOML is easier to read, JSON is more widely used and still offers improved readability compared to the current RML format.