Skip to content

Commit d65c8ce

Browse files
authored
Fix text generation endpoint (ikawrakow#654)
1 parent 4e9c78c commit d65c8ce

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

examples/server/server.cpp

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1615,10 +1615,10 @@ struct server_context {
16151615
std::vector<ik_chat_msg_diff> oaicompat_msg_diffs;
16161616
slot.update_chat_msg(oaicompat_msg_diffs);
16171617

1618-
// Following original llama.cpp pattern: send empty content in streaming mode
1619-
// Clean content comes through oaicompat_msg_diffs instead of raw tokens
1618+
// For text completion endpoints, send actual content; for chat completion, use diffs
1619+
// OpenAI-compatible chat endpoints use empty content with diffs for tool calls
16201620
res.data = json {
1621-
{"content", ""}, // Empty - clean content provided via diffs
1621+
{"content", slot.oaicompat ? "" : tkn.text_to_send}, // Text completion needs actual content
16221622
{"stop", false},
16231623
{"id_slot", slot.id},
16241624
{"multimodal", false}

0 commit comments

Comments
 (0)