Tensor-parallel like FeedForward to lower memory requirements#10623

Closed

a-r-r-o-w wants to merge 2 commits intomainfrom

memory-optimizations/tensor-parallel-like-ff

Commits on Jan 21, 2025

memory-optimized ff
a-r-r-o-w
committed

Commits on Jan 25, 2025

fix geglu and swiglu gate computation
a-r-r-o-w
committed