what is the purpose of ggml_half2 dm in quantization structure?

[PenutChen]

I notice that there is ggml_half2 dm in many k-quant structures, e.g. block_q2_K and block_q4_K , but I'm not sure of its purpose. I didn't see it being used in quantization or dequantization. What is the purpose of this value? Is it for memory alignment or something?

[ggerganov]

It's used in GPU code that supports half2 ops. For example in CUDA:

ggml/src/ggml-cuda/fattn-common.cuh

Line 120 in a3c0188

const half2 d4d8_m4s8 = K_q4_1[ib].dm * Q_ds[k_KQ_0/WARP_SIZE];

Since this is a union, the dm member is basically an alias for the d and dmin factors

[PenutChen]

Thanks for explaining, really cool tips!