[FEATURE] Kolmogorov–Arnold Transformer

[Adamdad]

Model description

The Kolmogorov–Arnold Transformer (KAT) replaces the standard MLP layers in transformers with Kolmogorov-Arnold Network (KAN) layers, improving the model's expressiveness and overall performance.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

The code is at https://github.com/Adamdad/kat and paper at https://arxiv.org/abs/2409.10594