Performance improvment for open source models

[Auth0rM0rgan]

Hi,

I'm using the data formulator with the open source models like qwen2.5-coder:32b or codellama:34b however 95% of the time, the models fail even with the simple and easy queries. The models fail to generate the code block, cannot find the data, or don't follow the code template. Is there any way to improve the performance of the agents? I tried enhancing the prompt, providing more examples to system prompts, but still failing.

Thanks

[Chenglong-MS]

It seems like its the burden of refining user goals is causing issue with code generation.

We can create a simplified version asking the model to only produce code, without doing the refinement. You can try it out a little bit? I could test it out later after finishing some dev branch works lately ---