Skip to content

Commit c60b3c3

Browse files
authored
Fix CPU overhead in LLM Obs eval processor (#9765)
* fix MLOS-268 * change poll timeout to 100ms
1 parent 62c5de9 commit c60b3c3

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

dd-java-agent/agent-llmobs/src/main/java/datadog/trace/llmobs/EvalProcessingWorker.java

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,11 @@ public void run() {
145145
private void runDutyCycle() throws InterruptedException {
146146
Thread thread = Thread.currentThread();
147147
while (!thread.isInterrupted()) {
148-
consumeBatch();
148+
LLMObsEval eval = queue.poll(100, TimeUnit.MILLISECONDS);
149+
if (eval != null) {
150+
buffer.add(eval);
151+
consumeBatch();
152+
}
149153
flushIfNecessary();
150154
}
151155
}

0 commit comments

Comments
 (0)