JSON output, store result as entity, and call tools

[widewing]

I would like to use LLM to generate an entity represent the information in the image. My specific use case is to track my garbage carts location (if they are in curbside) but I believe this should be a generic ask.

For doing that, I would like the LLM to generate JSON response, and store the result accordingly.

Furthermore, if it can call some tools to aid its decision it would be even better. For example, the "think" tool will have great improvement to the final result.

[andrewfry]

I have been trying to figure this out as well. Will definitely respond if I figure it out.

[eMeF1]

I managed to get a response with multiple keys in JSON. This works well, and I can parse the variables and use them further in automation, but then, if I enable the Remember parameter, I get the whole JSON as a description in the timeline.

I'm trying to figure out the best way to solve this.

Any recommendations, @valentinfrlch?

[valentinfrlch]

For now you could parse the json response in a Home Assistant automation and then create a custom event on the timeline via llmvision.create_event.
Feel free to create a feature request to correctly parse the event description if response_format is set to json. This way the parsing could be handled internally.

[eMeF1]

@valentinfrlch, am I correct in understanding that in 1.6.0, this is implemented to the extent that the integration parses the JSON, and I then select the key for the event title but not the description? If so, should I create a feature request to include configuration for the description?