v2.6.9
What's Changed
- 🔥🔥[TurboAttention] TURBOATTENTION: EFFICIENT ATTENTION APPROXIMATION FOR HIGH THROUGHPUTS LLMS by @DefTruth in https://github.com/DefTruth/Awesome-LLM-Inference/pull/105
- 🔥🔥[NITRO] NITRO: LLM INFERENCE ON INTEL® LAPTOP NPUS by @DefTruth in https://github.com/DefTruth/Awesome-LLM-Inference/pull/106
- 🔥[DynamicKV] DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs by @DefTruth in https://github.com/DefTruth/Awesome-LLM-Inference/pull/107
- 🔥🔥[HADACORE] HADACORE: TENSOR CORE ACCELERATED HADAMARD TRANSFORM KERNEL by @DefTruth in https://github.com/DefTruth/Awesome-LLM-Inference/pull/108
Full Changelog: DefTruth/Awesome-LLM-Inference@v2.6.8...v2.6.9