Idea: Log metadata for prompts (e.g. version and evaluation)

[minthemiddle]

I iterate a lot on my prompts. And I would like to log my experiences with these prompts in llm, alongside the existing logs. The infos I'm keeping are version (Which version of a prompt, e.g. 0.3, rationale, i.e. why I used exactly this prompt, feedback on what worked as expected and what I might improve on still.

Right now, I save this extra information in a separate SQLite database.
As llm already logs prompts, this might be a good addition. Or maybe to keep the core tight, it might be better as plugin.

As other people will save feedback differently, maybe a more general metadata with simple key:value might be a good format.

What do you think?