MLP benchmarks

[adam-smnk]

Usage: ./tools/mlir_bench/mlp_bench.sh

TODO:

• test on cluster
• add support for matmul without transpose
• investigate broadcast error when type is not f32

[adam-smnk]

Works fine on cluster, I was able to gather initial numbers for f32.
There's some lowering issue for other data types. I'll look into it next.

[slyalin]

investigate broadcast error when type is not f32

@adam-smnk, have you unlocked non f32 types in MLIR conversion?

I noticed issues with accuracy in Pytorch layer tests when running min_max tests. Reproduced when OV_MLIR is enabled only.

[adam-smnk]

have you unlocked non f32 types in MLIR conversion?

I think it was primarily a mistake in my testing setup. Otherwise, I just need to relax matchers to accept any types.

[slyalin]

Just FYI: ov.save_model will partially compress weights from f32 to f16. "Partially" is because for each constant it is decided individually based on the values range in the constant. In the IR it will be represented as two operations: Constant(f16) -> Convert(f32). This is our default behavior for all models in ov.save_model to save space on the disk, and it shouldn't affect final inference because this combination of operations will be constant folded during model compilation.

[slyalin]

It doesn't affect inference precision, it is just a way of weights compression.