Kolmogorov-Arnold Transformer #24
Description
KAN was strong but faced scalability issues. We tackled this with 3 simple tricks. By combining KAN with Transformers, we've built a much stronger and more scalable model. 💪
📄 Paper: https://arxiv.org/abs/2409.10594
💻 Code: https://github.com/Adamdad/kat