[Enhancement]: add tracing/observer support (direct or indirect)

[stdweird]

What features would you like to see added?

From the perspective of a team giving support to end users, it is necessary to have detailed traces collected by an external tool and references to those traces, so when end users complain about the quality of a response, this can be investigated.

LibreChat needs 2 features:

• send trace data (direct or indirect)
• provide feedback interface

More details

W.r.t. sending the trace data in a direct way: example of such external tool is (in our case) OSS Phoenix, javascript bindings documentation (for the neutral openinference protocol) see https://docs.arize.com/phoenix/tracing/how-to-tracing/setup-tracing/javascript

As you can see from the code, this is a lot of work. (I am not familiar with javascript enough, but e.g. in python, much of this complexity is hidden in decorators, making the code changes a lot lighter, and easier to implement different sorts of tracing backends; but even then it's still quite some work).

Anyway, as i assume this is way too much work and/or too complicated to setup, there is an indirect way that would help in our case: add details of chat session via eg headers to the endpoints. In our case, we control the endpoints, it's easier for us to use the extra details to do the tracing from the endpoints to e.g. phoenix.
The details we would need are e.g. some uuid for each chat session and response counter in that chat session. One can configure custom headers for the endpoints, but i cannot find from the docs what dynamic values one can pass on.
These headers would also be required to be send to the rag_api (not configured as a separate endpoint), so also the rag details can be traced via langchain builtin support (so we can couple the rag_api lookup to the correct chat session, and then see that eg. a response had low quality because the wrong chunks were picked up).

Wrt feedback interface: at the very least an enduser can access the session uuid, so they can contact support and discuss the exact chat session using that id.

Giving feedback to a response (0-5 stars; or simple thumbs up/down (0-1 response) is ofcourse even better. The feedback value could be send with the session id and response counter to some custom url, that can convert this value to whatever format and store it somehow (in our case, we could lookup the spanid (an openinference concept) from the session id; and add the score as an evaluation to the span).

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct