Skip to content

Commit 178be59

Browse files
authored
feat(genapi): update recommandations in troubleshooting
1 parent 2360cbd commit 178be59

File tree

1 file changed

+12
-1
lines changed

1 file changed

+12
-1
lines changed

pages/generative-apis/troubleshooting/fixing-common-issues.mdx

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,13 +53,24 @@ Below are common issues that you may encounter when using Generative APIs, their
5353

5454
## Structured output (e.g., JSON) is not working correctly
5555

56-
### Cause
56+
### Description
57+
- Structured output response contains invalid JSON
58+
- Structured output response is valid JSON but content is less relevant
59+
60+
### Causes
5761
- Incorrect field naming in the request, such as using `"format"` instead of the correct `"response_format"` field.
5862
- Lack of a JSON schema, which can lead to ambiguity in the output structure.
63+
- Maximum tokens is lower than what the model response needs to be complete.
64+
- Temperature is not set or set too high.
65+
5966

6067
### Solution
6168
- Ensure the proper field `"response_format"` is used in the query.
6269
- Provide a JSON schema in the request to guide the model's structured output.
70+
- Ensure the `max_tokens` value is higher than the response `completion_tokens` value. If this is not the case, the model answer may be stripped down before it can finish the proper JSON structure (and lack closing JSON brackets `}` for example). Note that if the `max_tokens` value is not set in the query, [default values apply for each models](/generative-apis/reference-content/supported-models/).
71+
- Ensure the `temperature` value is set with a lower range value for the model. If this is not the case, the model answer may output invalid JSON characters. Note that if the `temperature` value is not set in the query, [default values apply for each models](/generative-apis/reference-content/supported-models/). As examples:
72+
- for `llama-3.3-70b-instruct`, `temperature` should be set lower than `0.6`
73+
- for `mistral-nemo-instruct-2407 `, `temperature` should be set lower than `0.3`
6374
- Refer to the [documentation on structured outputs](/generative-apis/how-to/use-structured-outputs/) for examples and additional guidance.
6475

6576

0 commit comments

Comments
 (0)