FP8 GPU implementation

[umangyadav]

Tested on MI300.

Added verify tests for Math ops, reduce ops and a couple/few pointwise ops.

Some FP8 tests are commented out because they run into compile errors. Need to fix and enable them

[pfultz2]

Why do you need void here?

[umangyadav]

to make it call numeric_max() always. I had to specialize numeric_max for fp8 to avoid redefinition by setting second template parmaeter to void. by adding void this would work for all the types.

[pfultz2]

I think you can add class Enable=void parameter to avoid this. The void parameter is an implementation detail that consumers shouldn't touch as its just there for enable_if.

[umangyadav]

Made changes.

[pfultz2]

This should just be a templated function instead of using a macro:

template <class T, MIGRAPHX_REQUIRES(is_same<T, fp8::fp8e4m3fnuz>{} or
                                     is_same<T, fp8::fp8e5m2fnuz>{} or
                                     is_same<T, fp8::fp8e4m3fn>{} or
                                     is_same<T, fp8::fp8e5m2>{})>
constexpr T numeric_max<T>()
{
    return fp8::numeric_limits<T>::max();
}

[umangyadav]

Doesn't work it, It would be considered as redefinition of numeric_max(). I need to speicalize numeric_max template that is defined in type_traits.hpp.

[pfultz2]

Ok then you should be able to specialize it like this:

template <migraphx::fp8::f8_type T = migraphx::fp8::f8_type::fp8, bool FNUZ, class Enable=void>
constexpr float8<T, FNUZ> numeric_max<float8<T, FNUZ>, Enable>()
{
    return fp8::numeric_limits<float8<T, FNUZ>>::max();
}

That should work for all fp8 types. Plus the class Enable=void wont require us to pass void into the function.

[umangyadav]

Made changes.

[pfultz2]

Is this overload necessary?

[umangyadav]

Yes i think it's used inside ops::lowest{}.

[umangyadav]

removed it. It it taken care of by numeric_lowest inside type_traits.

[pfultz2]

Ok, you can specialize this using float8<T, FNUZ> template.

[umangyadav]

float8<T, FNUZ> Partial specialization that way is not allowed. Runs into compilation errors.

[umangyadav]

Made changes.

[CharlieL7]

What's the usage of implicit_conversion?
Looks like a lot of the kernels were changed to be more explicit about the types.

[umangyadav]

What's the usage of implicit_conversion?
Looks like a lot of the kernels were changed to be more explicit about the types.

From what i understand, some of the internal functions/kernels return output in different type compared to what is desired from JIT kernel's output type. and implicit conversion takes care of such types mismatches.

Float8 constructors are marked as explicit to avoid implict conversions and therefore had to call constructor explicitly in many places explicitly.

Other reason is to avoid warnings about narrowing conversion from Float to __Float16 or fp8.

[TedThemistokleous]

LGTM. No more questions/concerns and looks like you've handled all comments

[pfultz2]

Looks good! We should probably look into having an env variable to enable fp8 emulation since its probably much slower on unsupported hardware then just using fp32 or fp16 directly.

[TedThemistokleous]

Looks good! We should probably look into having an env variable to enable fp8 emulation since its probably much slower on unsupported hardware then just using fp32 or fp16 directly.

like this idea too but lets get this in so we can hammer the others out. Good for a smaller final PR