Refactor: Modeld: Extract model running logic to separate classes

[devtekve]

Description

The model running logic has been refactored into separate classes for TICI and non-TICI hardware: TinygradRunner and ONNXRunner. These classes are located in the new file model_runner.py. The runners handle all tasks related to model execution, including preparing inputs, running the model, and slicing the outputs. This makes modeld.py cleaner, and easier to manage and debug.

Sequence diagram for model inference process

sequenceDiagram
    participant M as Modeld
    participant R as ModelRunner
    participant T as TinygradRunner/ONNXRunner

    M->>R: prepare_inputs(imgs_cl, numpy_inputs)
    R->>T: prepare_inputs(imgs_cl, numpy_inputs)
    T-->>M: prepared inputs

    M->>R: run_model()
    R->>T: run_model()
    T-->>R: model outputs
    R->>R: slice_outputs()
    R-->>M: parsed outputs

Loading

Class diagram for ModelRunner hierarchy

classDiagram
    class ModelRunner {
        <<abstract>>
        +model_metadata
        +input_shapes
        +output_slices
        +inputs: dict
        +__init__()
        +slice_outputs(model_outputs)
        +prepare_inputs(imgs_cl, numpy_inputs)*
        +run_model()*
    }

    class TinygradRunner {
        +model_run
        +__init__()
        +prepare_inputs(imgs_cl, numpy_inputs)
        +run_model()
    }

    class ONNXRunner {
        +runner
        +frames
        +__init__(frames)
        +prepare_inputs(imgs_cl, numpy_inputs)
        +run_model()
    }

    ModelRunner <|-- TinygradRunner
    ModelRunner <|-- ONNXRunner

    note for ModelRunner "Abstract base class for model runners"
    note for TinygradRunner "Implementation for TICI hardware"
    note for ONNXRunner "Implementation for non-TICI hardware"

Loading

Verification

• Ran the pipelines and drove the code changes

[devtekve]

@YassineYousfi FYI

[YassineYousfi]

Thanks!

This is a good cleanup in theory. But the short term goal is to only have tinygrad runner for all backends, and remove onnxruntime as a dependency. In which case this adds an unneeded ABC.

Closing for now.

[devtekve]

Thanks for checking it out @YassineYousfi !