server: Fix has_next_line in JSON response

[MichelleTanPY]

Make sure to read the contributing guidelines before submitting a PR

Description

Fix #10728
Update the JSON response for has_next_line and model

Changes Made

• Replaced int32_t with bool for has_next_line
• Update the DEFAULT_OAICOMPAT_MODEL value to gpt-3.5-turbo (Should be OpenAI compatible based on OpenAI Docs)

Tests

curl -Ss --data '{"prompt": "Alice: Ask me any question.\nBob: What color is the sky on", "n_predict": 8, "cache_prompt": true, "stop": ["?\n"], "seed": 42, "stream": true}' http://127.0.0.1:8080/completion
Response:

data: {"index":0,"content":" Pl","stop":false,"id_slot":-1,"tokens_predicted":1,"tokens_evaluated":18}

data: {"index":0,"content":"uto","stop":false,"id_slot":-1,"tokens_predicted":2,"tokens_evaluated":18}

data: {"index":0,"content":"","stop":false,"id_slot":-1,"tokens_predicted":3,"tokens_evaluated":18}

data: {"index":0,"content":"","stop":false,"id_slot":-1,"tokens_predicted":4,"tokens_evaluated":18,"timings":{"prompt_n":18,"prompt_ms":1864.694,"prompt_per_token_ms":103.5941111111111,"prompt_per_second":9.653058357028018,"predicted_n":4,"predicted_ms":706.629,"predicted_per_token_ms":176.65725,"predicted_per_second":5.660679083366236}}

data: {"index":0,"content":"","id_slot":0,"stop":true,"model":"gpt-3.5-turbo","tokens_predicted":4,"tokens_evaluated":18,"generation_settings":{"n_predict":8,"seed":42,"temperature":0.800000011920929,"dynatemp_range":0.0,"dynatemp_exponent":1.0,"top_k":40,"top_p":0.949999988079071,"min_p":0.05000000074505806,"xtc_probability":0.0,"xtc_threshold":0.10000000149011612,"typical_p":1.0,"repeat_last_n":64,"repeat_penalty":1.0,"presence_penalty":0.0,"frequency_penalty":0.0,"dry_multiplier":0.0,"dry_base":1.75,"dry_allowed_length":2,"dry_penalty_last_n":-1,"dry_sequence_breakers":["\n",":","\"","*"],"mirostat":0,"mirostat_tau":5.0,"mirostat_eta":0.10000000149011612,"penalize_nl":false,"stop":["?\n"],"max_tokens":8,"n_keep":0,"n_discard":0,"ignore_eos":false,"stream":true,"logit_bias":[],"n_probs":0,"min_keep":0,"grammar":"","samplers":["dry","top_k","typ_p","top_p","min_p","xtc","temperature"],"speculative.n_max":16,"speculative.n_min":5,"speculative.p_min":0.8999999761581421,"timings_per_token":false},"prompt":"<s> Alice: Ask me any question.\nBob: What color is the sky on","has_new_line":false,"truncated":false,"stop_type":"word","stopping_word":"?\n","tokens_cached":21,"timings":{"prompt_n":18,"prompt_ms":1864.694,"prompt_per_token_ms":103.5941111111111,"prompt_per_second":9.653058357028018,"predicted_n":4,"predicted_ms":706.671,"predicted_per_token_ms":176.66775,"predicted_per_second":5.6603426488422475}}

[ngxson]

We can merge as soon as the CI pass

[ngxson]

There is a test case you haven't updated: https://github.com/ggerganov/llama.cpp/actions/runs/12323491767/job/34400288305?pr=10818

[MichelleTanPY]

I removed my unit test changes. Correct me if I'm wrong, I think has_new_line attribute isn't a fixed value until the text generation is finished. I'm still figuring out why my local unit tests is failing on other asserts.

[ngxson]

You can also add a test case like type(res.body["has_new_line"]) == bool, not necessary check its value

[MichelleTanPY]

Updated, thanks for reviewing. 😊

[ngxson]

LGTM now. Having proper test cases is always appreciated 👍