postprocess: switch from gemini-3-pro-preview to gemini-3-flash-preview for the default
#1535
Conversation
…eview` for the default `flash` is cheaper and faster and seems to still be sufficiently capable. I did have to adjust the prompt slightly to explicitly say to include the surrounding Rust Markdown code block. This isn't actually used right now, but is in #1534, so it seems worth preserving.
thedataking
left a comment
thedataking
left a comment
There was a problem hiding this comment.
I don't think we should require the markdown code blocks.
Sometimes you get them, sometimes you don't (even for same model), so the way to round-trip less is to just remove them when they're present rather than forcing the model to potentially re-do its work when they're not present.
kkysen
left a comment
kkysen
left a comment
There was a problem hiding this comment.
I don't think we should require the markdown code blocks.
Sometimes you get them, sometimes you don't (even for same model), so the way to round-trip less is to just remove them when they're present rather than forcing the model to potentially re-do its work when they're not present.
I was always getting them with gemini-3-pro-preview, including on all of json-c, although not with gemini-3-flash-preview anymore. But with the direction to include it, it seems to always do it now. The remove_backticks logic isn't changed in this PR, though. But my thinking in general is that if we want the LLM to ever respond with anything else (which I found super useful in asking it why certain comments were omitted or emitted in certain weird ways), we need a way to distinguish that extra commentary/reasoning from the Rust code, and the Markdown code block seems like the perfect way to do that.
To keep our code more model-agnostic, don't ask for or enforce that the Rust code be in a Rust Markdown code block. Weaker models sometimes don't do this consistently, and we don't want extra LLM queries just to fix backticks.
flashis cheaper and faster and seems to still be sufficiently capable.I did have to adjust the prompt slightly to explicitly say to include the surrounding Rust Markdown code block. This isn't actually used right now, but it is in #1534, so it seems worth preserving.Edit: The prompt is kept the same now.