Llmlingua can help with multiple docs rerank and reduce tokens https://github.com/microsoft/LLMLingua
