The reasoning from the LLM is not consistent across the calls

[gbergengruen]

While seeing the page reports and the LLM calls, it seems that the reasoning is not asked to be consistent.

Example:
Book in Spanish - Cuaderno 3:
input_language: es

reasoning | "Diseñé una página tipo actividad, inspirada en el estilo del libro: banda superior púrpura/claros, etiquetas naranjas y un globo verde para la consigna. Respeté estrictamente los ids de texto, incluí una sola sección con role="activity" y data-section-type="other", usé solo la imagen provista y no añadí textos adicionales fuera de los dados. Los campos vacíos se modelan con inputs y un gran textarea sin placeholder para que el estudiante escriba."

But what if you don't understand Spanish at all, and you are converting that material with the plate in English. This is not synched.

My take is that the reasoning should be in the plate language configured in the config, which is currently defaulted to the input language.

In my development branch I'm defaulting that to English, but I think we should pass the plate language parameter to every prompt and ask the reasoning in that language.

[nicpottier]

That's a good idea. I wonder if there are any knock on effects of that language "bleeding over" to the content if we include multiple, but agree that reasoning being in a plate language makes sense.