Update v0.8 spec with updated catalog negotiation style #136
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -62,13 +62,12 @@ Communication occurs via a JSON Lines (JSONL) stream. The client parses each lin | |
|
|
||
| - `surfaceUpdate`: Provides a list of component definitions to be added to or updated in a specific UI area called a "surface." | ||
| - `dataModelUpdate`: Provides new data to be inserted into or to replace a surface's data model. Each surface has its own data model. | ||
| - `beginRendering`: Signals to the client that it has enough information to perform the initial render, specifying the ID of the root component and, optionally, the component catalog to use. | ||
|
Collaborator
There was a problem hiding this comment. I'd require it in begin render, or you could default it to the standard one
Collaborator
There was a problem hiding this comment. Why is the agent card option not sufficient here?
Collaborator
Author
There was a problem hiding this comment. @wrenj Re requiring this: I want to add it as an optional parameter to beginRender in v0.8 and default to standard catalog, to make this change non-breaking. Otherwise all our agents theoretically need updating right away. In v0.9, let's make this required and explicit. I filed #193 for this. @dmandar Re the switch I've made here to having the client advertise multiple catalogs and have the server choose, instead of the inverse - see https://docs.google.com/document/d/1SppqNwHytVIUp7o8x5py5Z7AIiMdz-pzgC0SWh9qfkE/edit?tab=t.lwpqe4lgroyz#bookmark=id.p3dk9n24f9ja which was prompted by James's request above. I think it's important that the agent choose the catalog, because in a multi-agent system, the client may not know which subagent is going to reply, so it can't know which catalog to choose. |
||
| - `deleteSurface`: Explicitly removes a surface and its contents from the UI. | ||
|
|
||
| Client-to-server communication for user interactions is handled separately via an A2A message. This message can be one of two types: | ||
|
|
||
| - `userAction`: Reports a user-initiated action from a component. | ||
| - `error`: Reports a client-side error. | ||
| This keeps the primary data stream unidirectional. | ||
|
|
||
|
|
@@ -167,13 +166,124 @@ The following is a complete, minimal example of a JSONL stream that renders a us | |
|
|
||
| A2UI's component model is designed for flexibility, separating the protocol from the component set. | ||
|
|
||
| ### 2.1. Catalog Negotiation | ||
|
|
||
| A **Catalog** defines the contract between the server and the client for the UI that can be rendered. It contains a list of supported component types (e.g., `Row`, `Text`), their properties, and available styles. A catalog is defined by a **Catalog Definition Document**. | ||
|
|
||
| There is a **Standard Catalog** associated with each version of the A2UI protocol. For v0.8, its identifier is `https://github.com/google/A2UI/blob/main/specification/0.8/json/standard_catalog_definition.json`. | ||
|
|
||
| Catalog IDs are simple string identifiers. While they can be anything, it is conventional to use a URI within a domain that you own, to simplify debugging, avoid confusion, and avoid name collisions. Furthermore, if any changes are made to a catalog that could break compatibility between an agent and renderer, a new `catalogId` **must** be assigned. This ensures clear versioning and prevents unexpected behavior if an agent has changes but the client does not, or vice versa. | ||
|
|
||
| The negotiation process allows the client and server to agree on which catalog to use for a given UI surface. This process is designed to be flexible, supporting standard, custom, and even dynamically-defined catalogs. | ||
|
|
||
| The flow is as follows: | ||
|
|
||
| #### 1. Server Advertises Capabilities | ||
|
|
||
| The server (agent) advertises its capabilities in its Agent Card as part of the A2A protocol. For A2UI, this includes which catalogs it supports and whether it can handle catalogs defined inline by the client. | ||
|
|
||
| - `supportedCatalogIds` (array of strings, optional): A list of IDs for all pre-defined catalogs the agent is known to support. | ||
| - `acceptsInlineCatalogs` (boolean, optional): If `true`, the server can process `inlineCatalogs` sent by the client. Defaults to `false`. | ||
|
|
||
| **Example Server Agent Card Snippet:** | ||
| ```json | ||
| { | ||
| "name": "Restaurant Finder", | ||
| "capabilities": { | ||
| "extensions": [ | ||
| { | ||
| "uri": "https://a2ui.org/ext/a2a-ui/v0.8", | ||
| "params": { | ||
| "supportedCatalogIds": [ | ||
| "https://github.com/google/A2UI/blob/main/specification/0.8/json/standard_catalog_definition.json", | ||
| "https://my-company.com/a2ui/v0.8/my_custom_catalog.json" | ||
| ], | ||
| "acceptsInlineCatalogs": true | ||
| } | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Note that this is not a strict contract and purely included as a signal to help orchestrators and clients identify agents with matching UI capabilities. At runtime, orchestrating agents may dynamically delegate tasks to subagents which support additional catalogs that the orchestrating agent did not advertise. Thus, clients should consider the advertised supportedCatalogIds as a subset of the true catalogs that the agent or its subagents may support. | ||
|
|
||
| #### 2. Client Declares Supported Catalogs | ||
|
|
||
| In **every** message sent to the server, the client includes an `a2uiClientCapabilities` object within the A2A `Message` metadata. This object informs the agent server of all catalogs the client can render. | ||
|
|
||
| - `supportedCatalogIds` (array of strings, required): A list of identifiers for all pre-defined catalogs the client supports. The client must explicitly include the standard catalog ID here if it supports the standard catalog. The contents of these catalogs are expected to be compiled into the agent server and not downloaded at runtime, in order to prevent malicious content being injected into the prompt dynamically, and ensure predictable results. | ||
| - `inlineCatalogs` (array of objects, optional): An array of full Catalog Definition Documents. This allows a client to provide custom, on-the-fly catalogs, typically for use in local development workflows where it is faster to update a catalog in one place on the client. This may only be provided if the server has advertised `acceptsInlineCatalogs: true`. | ||
|
|
||
| **Example A2A Message with Client Capabilities:** | ||
| ```json | ||
| { | ||
| "metadata": { | ||
| "a2uiClientCapabilities": { | ||
| "supportedCatalogIds": [ | ||
| "https://github.com/google/A2UI/blob/main/specification/0.8/json/standard_catalog_definition.json", | ||
| "https://my-company.com/a2ui_catalogs/custom-reporting-catalog-1.2" | ||
| ], | ||
| "inlineCatalogs": [ | ||
| { | ||
| "catalogId": "https://my-company.com/inline_catalogs/temp-signature-pad-catalog", | ||
| "components": { | ||
| "SignaturePad": { | ||
| "type": "object", | ||
| "properties": { "penColor": { "type": "string" } } | ||
| } | ||
| }, | ||
| "styles": {} | ||
| } | ||
| ] | ||
| } | ||
| }, | ||
| "message": { | ||
| "prompt": { | ||
| "text": "Find me a good restaurant" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| #### 3. Server Chooses Catalog and Renders | ||
|
|
||
| The server receives the client's capabilities and chooses a catalog to use for a specific UI surface. The server specifies its choice in the `beginRendering` message using the `catalogId` field. | ||
jacobsimionato marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| - `catalogId` (string, optional): The identifier of the chosen catalog. This ID must be one of the `supportedCatalogIds` or the `catalogId` from one of the `inlineCatalogs` provided by the client. | ||
|
Collaborator
There was a problem hiding this comment. should we require this? in your example if its not present, what does the client do? just assume the default catalog?
Collaborator
Author
There was a problem hiding this comment. I want to add it as an optional parameter to beginRender in v0.8 and default to standard catalog, to make this change non-breaking. Otherwise all our agents theoretically need updating right away. Filed #193 to make it required for v0.9.
Collaborator
There was a problem hiding this comment. tbh breaking changes are fine, nobody is really using in 1P, and updating the agent samples is easy, not sure about for other teams you work with
Collaborator
Author
There was a problem hiding this comment. Hey that's fair. I guess I want to avoid creating more work trying to get all the renderers in sync for v0.8 too! |
||
|
|
||
| If the `catalogId` is omitted, the client **MUST** default to the standard catalog for the protocol version (`https://github.com/google/A2UI/blob/main/specification/0.8/json/standard_catalog_definition.json`). | ||
|
|
||
| **Example `beginRendering` Message:** | ||
| ```json | ||
| { | ||
| "beginRendering": { | ||
| "surfaceId": "unique-surface-1", | ||
| "catalogId": "https://my-company.com/inline_catalogs/temp-signature-pad-catalog", | ||
| "root": "root-component-id" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Each surface can use a different catalog, providing a high degree of flexibility, particularly in multi-agent systems where different agents may support different catalogs. | ||
|
|
||
| #### Schemas for Developers | ||
|
|
||
| When building an agent, it is recommended to use a resolved schema that includes the specific component catalog you are targeting (e.g., a custom schema combining `server_to_client.json` with your `https://my-company.com/a2ui_catalogs/custom-reporting-catalog-1.2` definition). This provides the LLM with a strict definition of all available components and their properties, as well as the catalog-specific styles, leading to more reliable UI generation. The generic `server_to_client.json` is the abstract wire protocol, while the resolved schema is the concrete tool for generation. | ||
|
|
||
| In order to do the substitution, based on the standard `server_to_client_schema` and a `custom_catalog_definition` object, you can use JSON manipulation logic similar to: | ||
|
|
||
| ```py | ||
| component_properties = custom_catalog_definition["components"] | ||
| style_properties = custom_catalog_definition["components"] | ||
| resolved_schema = copy.deepcopy(server_to_client_schema) | ||
|
|
||
| resolved_schema["properties"]["surfaceUpdate"]["properties"]["components"]["items"]["properties"]["component"]["properties"] = component_properties | ||
| resolved_schema["properties"]["beginRendering"]["properties"]["styles"]["properties"] = style_properties | ||
| ``` | ||
|
|
||
| See `server_to_client_with_standard_catalog.json` for an example of a resolved | ||
| schema which has the components substituted in. | ||
|
|
||
| ### 2.2. The `surfaceUpdate` Message | ||
|
|
||
|
|
@@ -414,7 +524,7 @@ While the server-to-client UI definition is a one-way stream (e.g., over SSE), u | |
|
|
||
| ### 5.1. The Client Event Message | ||
|
|
||
| The client sends a single JSON object that acts as a wrapper. It must contain exactly one of the following keys: `userAction` or `error`. | ||
|
|
||
| ### 5.2. The `userAction` Message | ||
|
|
||
|
|
@@ -430,60 +540,11 @@ The `userAction` object has the following structure: | |
|
|
||
| The process for resolving the `action.context` remains the same: the client iterates over the `context` array, resolves all literal or data-bound values, and constructs the `context` object. | ||
|
|
||
| ### 5.3. The `error` Message | ||
|
|
||
| This message provides a feedback mechanism for the server. It is sent when the client encounters an error, for instance, during UI rendering or data binding. The content of the object is flexible and can contain any relevant error information. | ||
|
|
||
| ### 5.4. Event Flow Example (`userAction`) | ||
|
|
||
| 1. **Component Definition** (from `surfaceUpdate`): | ||
|
|
||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
single inline catalog right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i see multiple below
whats the usecase you are envisioning here? not saying having an array is a problem just curious if it solves any problems not solved by a single inline catalog?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can totally see an ecosystem of inline catalogs evolving, available to client developers to include in their clients. You could concatenate them, I suppose, but why not have a list?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the agent picks a single catalog to use, the inline catalogs aren't concatenated, so the client would say here are 3 inline catalogs pick one which I don't understand
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My take 1. InlineCatalogs hopefully are only used in non-prod environments. The server agent permitting which catalog(s) in its agent card is the primary mechanism we should push for.
2. For inline catalogs, one can imagine client developers keeping separate catalog files for each of their components. If we support that, then both the server and client need to have a long list of catalogs to manage. The client could collate components off of each catalog and send the list inline.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I put multiple inline catalogs here because I figured it's consistent with advertising multiple well-known catalogs which an agent can pick from. If a client supports 5 different catalogs, shouldn't it also be able to advertise them all inline? Or should it have to pick only one of them to advertise inline, and advertise the rest only via ID?
Thinking about it a bit more, I can see how this is not especially likely to be used in practice, because how will agents know which catalog to pick? But I kind of still want to leave it here, because it doesn't add much complexity to the API, and I think it might even make the client SDKs simpler - you will just have a boolean like "advertiseCatalogsInline" rather than a setting which specific catalog of your many catalogs you'd like to advertise inline.
I agree with Mandar that this is mostly a mechanism for local development, so we don't have to think too hard about it.
How strongly does everyone feel? I'll remove this if there is consensus that it's overly complex!
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like it as a
list.dictionary.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
multiple inline catalogs sgtm
its not a list though its a dictionary right? of catalogId: inlineCatalogJson ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you're right, it's a dict.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The catalog_description_schema contains catalogId already, which I think makes sense - a catalog should declare it's Id. Now, inlineCatalogs references this schema. So I think it makes sense to just have an array of these, and each of them has an ID in it, rather than redundantly including the ID as a key as well. Yes, there could be duplicate Ids, and that would be an error.
WDYT?