-
-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Open
Labels
possible bugBug was reported but is not confirmed or is unable to be replicated.Bug was reported but is not confirmed or is unable to be replicated.
Description
How are you running AnythingLLM?
Docker (local)
What happened?
I'm setting up analytics in my fork and noticed tps/duration not lining up with expectations (especially when not streaming). In my tests with Phi4:14b streaming was about 60tps, but non-streaming was about 15tps. In either case, metrics are capturing duration and using that (with network latency + time to first token) to calculate outputTps, which is misleading.
Instead of using duration for the calculation, it should be done with eval_duration, as this is how output speed is generally calculated.
Metadata
Metadata
Assignees
Labels
possible bugBug was reported but is not confirmed or is unable to be replicated.Bug was reported but is not confirmed or is unable to be replicated.