content_type non-enforcement for Document

[bglearning]

We don't seem to enforce content_type to be from a pre-defined set anywhere in Document creation.

As such, Document(content="...", content_type="random-type") is also valid.

Though we do have ContentTypes Literal (here) as part of the type hint for content_type in Document and so type-checkers can complain. Of course, ContentTypes specifically is only meant for type hinting but can also imagine having some normal list specified to be enforced.

Just wanted to confirm: is it a design choice? i.e. to allow for greater flexibility to users at the schema level.

[ZanSara]

Hey @bglearning ! I don't recall ever having a proper design discussion over this topic, so thank you for raising it up!

I'd say this is a proper use for a Proposal, although first-of-its-kind (so far we used it only for new features). Could you draft one according to the guidelines? https://github.com/deepset-ai/haystack/tree/main/proposals I'm looking forward to see how it plays out 🙂

[bglearning]

I'm actually not sure if there is a change needed for this. The more I think about it the more it seems it's probably fine to leave it as-is. 🤔

The thought being: You can use any content_type in the schema though we officially acknowledge only these ones defined in the Literal. So, all official nodes will work with a subset of those types but you're free to build your own that uses custom content_type.

Only wanted to start a discussion on this so that it's something we're conscious about as a design choice.

All in all, not sure if there is a possible Proposal to be made unless we want to house this discussion in a "what if we check and/or enforce the content_type" proposal rather than here 😄 .