DEV: Use structured responses for summaries

[romanrizzi]

System personas:

---

Regular personas:

---

Schema editor:

[keegangeorge]

Two minor suggestions, otherwise LGTM

[SamSaffron]

this is a bit surprising imo... it should be: {key: "Hello!"} or perhaps a special object that inherits off hash eg: StructuredOutput , then nobody needs to worry about parsing and we can enforce structure at the boundary.

[xfalcox]

What about we use JSON.parse(result, object_class: OpenStruct) so we always have a proper object to work with?

[romanrizzi]

I don't think that will work. During streaming, we only get chunks of the JSON so we cannot use JSON#parse.

I believe Sam's proposal is to have a class that handles the buffering and parsing while decoding the response. This class reads the chunk, keeps track of which key/value are we at, and updates the state:

Chunk 1: "{\"message\": \"Line 1\\"
Partial: {"message"=>"Line 1"}

Chunk 2: "nLine 2\", \"status\": \"o"
Partial: {"message"=>"Line 1\nLine 2", "status"=>"o"}

Chunk 3: "k\"}"
Partial: {"message"=>"Line 1\nLine 2", "status"=>"ok"}

By taking care of it internally, the burden of doing this doesn't fall on the modules using structured outputs, like in this case, FoldContent.