server: process prompt fairly accross slots

[jakexcosme]

Note: This issue was copied from ggml-org#6607

Original Author: @phymbert
Original Issue Number: ggml-org#6607
Created: 2024-04-11T10:23:54Z

---

Context

At the moment we implement a FIFO approach to batch prompt tokens. So if a large prompt is to be processed it blocks all other slots.

Proposal: implement a fair batch usage of prompt processing accross all pending slots.

References:

• server : improvements and maintenance ggml-org/llama.cpp#4216 (comment)
• server: main loop blocked, server stuck ggml-org/llama.cpp#5851 (comment)