Misc. bug: [json.exception.type_error.316] invalid UTF-8 byte at index 145: 0x27 #11738
Description
Name and Version
Client side
ramalama serve tinyllama --host 0.0.0.0
curl -X POST "http://127.0.0.1:8080/v1/completions" -H "Content-Type: application/json" -d '{
"model": "tinyllama",
"prompt": "Explain artificial intelligence in simple words.",
"max_tokens": 100,
"temperature": 1.5,
"top_k": 100,
"top_p": 1.0,
"repeat_penalty": 1.2
}'
curl: (52) Empty reply from server
Server side
srv update_slots: all slots are idle
got exception: {"code":500,"message":"[json.exception.parse_error.101] parse error at line 1, column 1: attempting to parse an empty input; check that your input string or stream contains the expected JSON","type":"server_error"}
request: POST /completion 192.168.82.25 500
got exception: {"code":500,"message":"[json.exception.parse_error.101] parse error at line 1, column 1: attempting to parse an empty input; check that your input string or stream contains the expected JSON","type":"server_error"}
slot launch_slot_: id 0 | task 25 | processing task
slot update_slots: id 0 | task 25 | new prompt, n_ctx_slot = 2048, n_keep = 0, n_prompt_tokens = 9
slot update_slots: id 0 | task 25 | kv cache rm [1, end)
slot update_slots: id 0 | task 25 | prompt processing progress, n_past = 9, n_tokens = 8, progress = 0.888889
slot update_slots: id 0 | task 25 | prompt done, n_past = 9, n_tokens = 8
slot release: id 0 | task 25 | stop processing: n_past = 108, truncated = 0
slot print_timing: id 0 | task 25 |
prompt eval time = 43.39 ms / 8 tokens ( 5.42 ms per token, 184.38 tokens per second)
eval time = 1458.28 ms / 100 tokens ( 14.58 ms per token, 68.57 tokens per second)
total time = 1501.67 ms / 108 tokens
srv update_slots: all slots are idle
request: GET /health 192.168.82.25 200
terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_3::detail::type_error'
what(): [json.exception.type_error.316] invalid UTF-8 byte at index 145: 0x27
Operating systems
No response
Which llama.cpp modules do you know to be affected?
No response
Command line
Problem description & steps to reproduce
http request got terminate the server side with:
terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_3::detail::type_error'
what(): [json.exception.type_error.316] invalid UTF-8 byte at index 145: 0x27
First Bad Commit
No response