[FEATURE] Exposing core definitions in the include folder under repository root as public header

[Jerry-Jinfeng-Guo]

The power grid model provides a stable API that contains mainly functionalities. This has many benefits. However, the current way of having everything in core and only expose function calls makes it impossible to access core definitions such as Input, Node, Dataset etc. Applications that would like to interact with PGM could only rely on API calls. Other interactions such as creating dataset could only be done by either mimicking what is defined in the core, or resort to the de/serializer.

In this request, a public header approach is proposed to expose the very essential definitions in public headers. This will lead to major change in the structure of the library and is therefore not of high priority.

[mgovers]

without going into details or working it out entirely, a possible approach could be:

• an SDK like our meta_data in the core, like in:
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\meta_data.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\meta_gen\gen_getters.hpp
    • these getters should
      • for each dataset type that is registered
      • for each component type that is registered
      • for each attribute that is registered
      • ask the C API (at runtime):
        • if it is exists, and,
        • if yes, what the PGM_MetaAttribute* is
• pre-defined component data structs, like in:
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\input.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\output.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\update.hpp
• those pre-defined component data structs can then be registered, like in:
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\meta_gen\input.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\meta_gen\output.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\meta_gen\update.hpp
• an interface that exposes all of the above, like in:
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\meta_data_gen.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\dataset_fwd.hpp
  • power_grid_model_c\power_grid_model\include\power_grid_model\auxiliary\dataset.hpp

[TonyXiang8787]

The C boundary has to remain with opaque structs. Otherwise, there is no guarantee for ABI compatibility. All the offsets, alignments, sizes need to be dynamically retrieved at runtime.

[mgovers]

The C boundary has to remain with opaque structs. Otherwise, there is no guarantee for ABI compatibility. All the offsets, alignments, sizes need to be dynamically retrieved at runtime.

yes, hence this explicit step in my comment:

• ask the C API (at runtime):
  • if it is exists, and,
  • if yes, what the PGM_MetaAttribute* is

[TonyXiang8787]

I am thinking like to extend Buffer class in the C++ wrapper with concrete types and you can use proxy to do attribute stuff. Something like this:

template<class StructType>
class StructBuffer: public Buffer {
  public:
    StructBuffer(Idx size): Buffer{StructType::meta_component(), size} {}
    
    // getter, may need to optimize the handle
    StructType operator[] (Idx pos) { 
        return StructBuffer{(void*)((char*)data_ + pos * MetaData::component_size(StructType::meta_component()))};
    }
    StructType const operator[] (Idx pos) const { 
        return StructBuffer{(void*)((char*)data_ + pos * MetaData::component_size(StructType::meta_component()))};
    }
};

The StructType should be automatically generated via all the getters of attributes. For example, for NodeInput we can have:

class NodeInput {
  public:
    static MetaComponent const* meta_component() { return PGM_def_input_node; }
    NodeInput(void* data): data_{data} {}
    
    // getter for attributes
    int32_t& id() { 
        return *(int32_t*)((char*)data_ + MetaData::attribute_offset(PGM_def_input_node_id)); 
    }
    int32_t const& id() const { 
        return *(int32_t const*)((char*)data_ + MetaData::attribute_offset(PGM_def_input_node_id)); 
    }
    // generate other attribute getters

  private:
    void* data_;

};

Then the user should be able to do

StructBuffer<NodeInput> node_input{5};
node_input[0].id() = 10;

[TonyXiang8787]

After some thinking, I guess the user also wants to be able to define a single owned struct (not proxy from buffer) in the stack. Like below.

NodeInput single_node_input{};
single_node_input.id() = 5;

Also user would like to grow the buffer dynamically, like below. We might need to implement many std::vector like functions in Buffer class.

StructBuffer<NodeInput> node_input{};
node_input.push_back(single_node_input);