Skip to content

Commit 99f7140

Browse files
committed
Improved flaky test.
The tool-calls-with-prompt-stop-test was falky in CI/CD. Likely because it relied on sleep times. Added promises to the test to coordinate between the concurrent tool calls and the prompt stop. Hopefully, this works better. Changed tool-calls-with-prompt-stop to tool-calls-with-prompt-stop-test for consistency. Fixed a reflection warning in deep-sleep by casting the sleep arg to long.
1 parent e60b208 commit 99f7140

File tree

1 file changed

+14
-4
lines changed

1 file changed

+14
-4
lines changed

test/eca/features/chat_test.clj

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
(loop [remaining (- deadline (System/currentTimeMillis))]
3030
(when (pos? remaining)
3131
(try
32-
(Thread/sleep remaining)
32+
(Thread/sleep (long remaining))
3333
(catch InterruptedException _))
3434
(recur (- deadline (System/currentTimeMillis)))))))
3535

@@ -329,10 +329,13 @@
329329
{:role :system :content {:type :progress :state :finished}}]}
330330
(h/messages))))))
331331

332-
(deftest tool-calls-with-prompt-stop
332+
(deftest tool-calls-with-prompt-stop-test
333333
(testing "Three concurrent tool calls. Stopped before they all finished. Tool call 3 finishes. Calls 1,2 reject."
334334
(h/reset-components!)
335-
(let [{:keys [chat-id]}
335+
(let [wait-for-tool3 (promise)
336+
wait-for-tool2 (promise)
337+
wait-for-stop (promise)
338+
{:keys [chat-id]}
336339
(complete!
337340
{:message "Run 3 read-only tool calls simultaneously."}
338341
{:api-mock
@@ -345,7 +348,10 @@
345348
(on-prepare-tool-call {:id "call-2" :name "ro_tool_2" :arguments-text ""})
346349
(on-prepare-tool-call {:id "call-3" :name "ro_tool_3" :arguments-text ""})
347350
(future (Thread/sleep 200)
348-
(f.chat/prompt-stop {:chat-id chat-id} (h/db*) (h/messenger) (h/metrics)))
351+
(deref wait-for-tool3)
352+
(Thread/sleep 50)
353+
(f.chat/prompt-stop {:chat-id chat-id} (h/db*) (h/messenger) (h/metrics))
354+
(deliver wait-for-stop true))
349355
(on-tools-called [{:id "call-1" :name "ro_tool_1" :arguments {}}
350356
{:id "call-2" :name "ro_tool_2" :arguments {}}
351357
{:id "call-3" :name "ro_tool_3" :arguments {}}])))
@@ -355,16 +361,20 @@
355361

356362
"ro_tool_1"
357363
(do (deep-sleep 350)
364+
(deref wait-for-tool2)
358365
{:error false
359366
:contents [{:type :text :content "RO tool call 1 result"}]})
360367

361368
"ro_tool_2"
362369
(do (deep-sleep 300)
370+
(deref wait-for-stop)
371+
(deliver wait-for-tool2 true)
363372
{:error false
364373
:contents [{:type :text :content "RO tool call 2 result"}]})
365374

366375
"ro_tool_3"
367376
(do (deep-sleep 100)
377+
(deliver wait-for-tool3 true)
368378
{:error false
369379
:contents [{:type :text :content "RO tool call 3 result"}]})))})]
370380
(is (match? {chat-id

0 commit comments

Comments
 (0)