Skip to content

Conversation

@gspencergoog
Copy link
Collaborator

@gspencergoog gspencergoog commented Oct 28, 2025

Description

This PR updates the A2UI schema and examples to replace and deprecate valueList with valueMap for representing map-like data structures within the data model. It updates the samples as well as the evals and web examples to match.

Changes:

  • Schema Update (a2ui_schema.py):
    • Replaced the valueList property in the DataModelUpdate contents definition with valueMap.
    • valueMap is defined as an array of objects, where each object has a key (string) and one of the value* properties (valueString, valueNumber, valueBoolean).
    • Updated the description for path to clarify that / can be used to replace the entire data model.
  • Example Update (a2ui_examples.py):
    • Modified the CONTACT_UI_EXAMPLES to use valueMap to represent the list of contacts.
    • Each contact is now an entry in the valueMap array, keyed by a unique identifier (e.g., "contact1", "contact2").
    • The properties of each contact (name, phone, email, etc.) are represented as entries in a nested valueMap.
  • Type Update (types.ts):
    • Updated the DataModelUpdate interface in the web library to reflect the schema change from valueList to valueMap.

Reasoning:

Using valueMap provides a more explicit and structured way to represent key-value pairs, improving clarity and consistency in how map-like data is handled within the A2UI framework. This change aligns the examples and types with the updated schema definition. Also, LLMS suck at dealing with list indices, so this helps them use string keys instead. They can always still use "1" and "2" as keys, but the examples help encourage strings.

Eval results with new schema (N=2)

Evaluation Summary

Model: gemini-2.5-flash (thinking: 0)

Prompt Name Avg Latency (ms) Failed Runs Tool Error Runs
deleteSurface 2254
dogBreedGenerator 8144
loginForm 2912
productGallery 4578
productGalleryData 2889
settingsPage 5730
dataModelUpdate 2370
uiRoot 2305
animalKingdomExplorer 19135 1 / 2
recipeCard 4805 2 / 2
musicPlayer 4409 2 / 2
weatherForecast 7828
surveyForm 3590
flightBooker 3448
dashboard 4687
contactCard 5565
calendarEventCreator 3731
checkoutPage 3783
socialMediaPost 4485
eCommerceProductPage 6854
interactiveDashboard 5017
travelItinerary 8719

Total failed runs: 5 / 44

Model: gemini-2.5-flash-lite (thinking: 0)

Prompt Name Avg Latency (ms) Failed Runs Tool Error Runs
deleteSurface 2115
dogBreedGenerator 5591 2 / 2
loginForm 3210
productGallery 3054
productGalleryData 2619
settingsPage 3983
dataModelUpdate 2346
uiRoot 2253
animalKingdomExplorer 12091
recipeCard 4606 2 / 2
musicPlayer 3922
weatherForecast 3734
surveyForm 4025
flightBooker 3918
dashboard 3733 1 / 2
contactCard 3049
calendarEventCreator 3905
checkoutPage 4146
socialMediaPost 4275
eCommerceProductPage 5567
interactiveDashboard 4819
travelItinerary 8552

Total failed runs: 5 / 44


Overall Summary

  • Number of tool error runs: 0 / 88
  • Number of runs with any failure (tool error or validation): 10 / 88
  • Models with at least one failure: gemini-2.5-flash-lite (thinking: 0), gemini-2.5-flash (thinking: 0)

Copy link
Collaborator

@dmandar dmandar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL at comment

@gspencergoog gspencergoog merged commit eadd79a into google:main Oct 28, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants