llama-3.2-3b-instruct does not actually support JSON Mode #21316
Description
Existing documentation URL(s)
- https://developers.cloudflare.com/workers-ai/models/llama-3.2-3b-instruct/
- https://developers.cloudflare.com/workers-ai/json-mode/#supported-models
What changes are you suggesting?
While building out a Workflow and testing various AI models, I attempted to try llama-3.2-3b-instruct as I might need the context window. The model documentation shows JSON Mode under the Input parameters for both prompts and messages.
Unfortunately the API responds with the following error:
AiError: 5025: This model doesn't suuport JSON Schema.
Referencing the JSON Mode Supported Models reveals it is not enabled for this model.
Ideally the functionality would be supported but in lieu of this it should be removed from the input parameters on the model page.
Additional information
Example (incomplete) JS snippet:
async function extractDetailFromVTT(transcript, env) {
console.log('Using AI to extract details from transcript...');
const detected = await env.AI.run( "@cf/meta/llama-3.2-3b-instruct", {
stream: false,
max_tokens: 512,
messages: [
{
role: 'system',
content: "You have been provided an automatically generated WEBVTT file which is the transcript of a speaker telling their story. Your job is to extract key details from the transcript."
+ " You should find the speaker's name, the date of the talk, their group, and how many years they have. It is critically important that you include all the keys even if the values are not detected."
+ " Do not guess at dates, do not include an invalid date, do not include a two digit year it must be a four digit year. It is better to be an incomplete date than an invalid date.",
},
{ role: 'user', content: transcript },
],
response_format: {
type: "json_schema",
json_schema: {
type: "object",
properties: {
name: { type: "string" },
date: { type: "string" },
group: { type: "string" },
years: { type: "integer" }
},
required: ["name"]
}
}
});
return detected;
}