Implement waiting loras metrics

[mayabar]

What would you like to be added:
Currently waiting loras always sends an empty string

Why is this needed:
Real list of loras in waiting requests are required for Lora based scorer

[mayabar]

Possible solution

Add storage that contains mapping from lora name to number of running/waiting requests.
Update the running loras map when worker pulls request from the channel (increase the counter) and when request processing finished (decrease the counter)
Update the waiting map when request was added to the channel. ---> there is a problem with adding the request to the waiting map for requests that will be immediately passed to the running state (this will add irrelevant waiting loras permutations)
To fix this problem we will add an additional map from requestId to its state (new, waiting, running)
When request is received and pushed to the channel, set a timer for a short delay (need to measure which time to use) and added to the requests map with state New
When worker is pulling the request from the channel, if its state is New -> updates the state to Running + increase running lora's counter; if its state is Waiting -> updates the state to Running + increase running lora's counter + decrease waiting lora's counter.
In addition, we don't want to report lora as waiting when request is added for "a moment" to the channel and immediately pulled by a worker. To solve this problem, the actions, done on request arrival as done after a short delay. If after the delay the state of the request is still New - proceed as described, if the state is Running - don't update as waiting lora

[mayabar]

Selected approach:

Use simple solution which will produce not required metrics reports for requests that enters to the channel and immediately processed as waiting.
This behavior will be fixed when requests queue will be re-implemented #172

[mayabar]

Implemented by #175