Where's the gain come from？

[CrashBugger]

Hello, could you please explain the difference between your work and GRACE? Does the gain come from replacing the hidden state with LoRA?

[HichemAK]

https://arxiv.org/pdf/2312.11795