Eval suites for AI features

[thibaudcolas]

All the prompts / features currently implemented have been created with trial and error across different models. I think it’s time we create evals that allow us to iterate and see gradual improvements over time. I expect this would help us to:

• Get higher-quality results via prompt engineering
• Move towards smaller open source models while maintaining output quality

[thibaudcolas]

In the guide website prompts folder, we now have a corpus of 35k-ish tokens of real-world site content to test things with. And eval suites for existing integrations in the package.

We need to now move this over to the wagtail-ai repository, and extend for other current and future features,

• Rich text content generation
• R&D to use AI for image metadata (tags, collections) #151
• Support multiple content quality checker prompts #160
• Style guide / tone of voice field to be shared between multiple AI features #159

[thibaudcolas]

In the guide website prompts folder, we now have a corpus of 35k-ish tokens of real-world site content to test things with. And eval suites for existing integrations in the package.

We need to now move this over to the wagtail-ai repository, and extend for other current and future features,

• Rich text content generation
• R&D to use AI for image metadata (tags, collections) #151
• Support multiple content quality checker prompts #160
• Style guide / tone of voice field to be shared between multiple AI features #159