Reintroduce createSurface, add evolution guide (#226)

Author: gspencergoog

specification/0.9/docs/a2ui_protocol.md

@@ -0,0 +1,276 @@
+# A2UI Protocol Evolution Guide: v0.8 to v0.9
+
+This document serves as a comprehensive guide to the changes between A2UI version 0.8 and version 0.9. It details the shifts in philosophy, architecture, and implementation, providing a reference for stakeholders and developers migrating between versions.
+
+## 1. Executive Summary
+
+**Version 0.9 represents a fundamental philosophical shift from "Structured Output First" to "Prompt First."**
+
+- **v0.8** was designed to be generated by an LLM using Structured Output above all else, optimized for LLMs that support strict JSON mode or function calling (which is also a form of Structured Output). It relied on deep nesting and specific wrapper structures that were definable in the limited schema features but often confusing for an LLM to generate.
+- **v0.9** is designed to be **embedded directly in an LLM's system prompt**. The schema is refactored to be more human-readable and "token-efficient" for the model to understand. It prioritizes patterns that LLMs naturally excel at (like standard JSON objects for maps) over strict structured output-friendly structures (like arrays of key-value pairs).
+
+### Summary Table
+
+| Feature               | v0.8                                   | v0.9                                                 |
+| :-------------------- | :------------------------------------- | :--------------------------------------------------- |
+| **Philosophy**        | Structured Output / Function Calling   | Prompt-First / In-Context Schema                     |
+| **Message Types**     | `beginRendering`, `surfaceUpdate`, ... | `createSurface`, `updateComponents`, ...             |
+| **Surface Creation**  | Explicit `beginRendering`              | Explicit `createSurface`                             |
+| **Component Type**    | Key-based wrapper (`{"Text": ...}`)    | Property-based discriminator (`"component": "Text"`) |
+| **Data Model Update** | Array of Key-Value Pairs               | Standard JSON Object                                 |
+| **Data Binding**      | `dataBinding` / `literalString`        | `path` / Native JSON types                           |
+| **Button Context**    | Array of Key-Value pairs               | Standard JSON Object                                 |
+| **Auxiliary Rules**   | N/A                                    | `standard_catalog_rules.txt`                         |
+| **Validation**        | Basic Schema                           | Strict `ValidationFailed` feedback loop              |
+
+## 2. Architectural & Schema Changes
+
+### 2.1. Modular Schema Architecture
+
+**v0.8:**
+
+- Monolithic tendencies. `server_to_client.json` often contained deep definitions or relied on complex `oneOf` structures that were hard to decompose.
+- `standard_catalog_definition.json` existed but was often implicitly coupled.
+
+**v0.9:**
+
+- **Modularization**: The schema is strictly split into:
+  - `common_types.json`: Reusable primitives (IDs, paths, weights).
+  - `server_to_client.json`: The "envelope" defining the message types.
+  - `standard_catalog_definition.json`: The specific UI components.
+- **Benefit**: This allows developers to swap out the `standard_catalog_definition.json` for a `custom_catalog.json` without touching the core protocol envelope.
+
+### 2.2. Strict Message Typing
+
+**v0.8:**
+
+- Messages were objects where properties like `surfaceUpdate` were optional keys.
+- Validation often relied on "minProperties: 1" constraints.
+
+**v0.9:**
+
+- Uses a top-level **`oneOf`** constraint in `server_to_client.json`.
+- **Reason**: This is a more natural way to express the schema to an LLM, and easier for the LLM to reason about. It's also a more natural form for the developer to read.
+
+### 2.3. Auxiliary Rules File
+
+**v0.9:**
+
+- **New Artifact**: `standard_catalog_rules.txt`.
+- **Purpose**: A plain-text prompt fragment containing rules for using the catalog schema (e.g., "MUST provide 'action' for Button").
+- **Usage**: Designed to be included in the system prompt alongside the catalog schema.
+- **Reason**: Some constraints (like conditional requirements or specific property combinations) are difficult or verbose to express in JSON Schema but easy to express in natural language rules for an LLM, and it can be packaged with the catalog schema for ease of customizing the prompt for a particular catalog.
+
+## 3. Protocol Lifecycle Changes
+
+### 3.1. `beginRendering` Replaced by `createSurface`
+
+**v0.8 (`beginRendering`):**
+
+- **Explicit Signal**: The server sent a `beginRendering` message to tell the client "I am done sending the initial batch of components, you can draw now."
+- **Root Definition**: The root component ID was defined in this message.
+- **Style Information**: The message included style information for the surface.
+
+**v0.9 (`createSurface`):**
+
+- **Replacement**: `beginRendering` is **REPLACED** by `createSurface`.
+- **Purpose**: `createSurface` signals the client to create a new surface and prepare for rendering.
+- **Style Information Removed**: `createSurface` does **NOT** contain style information. Theming is now handled via the client styles, decoupling it from the message stream.
+- **Root Rule**: The rule is: "There must be exactly one component with the ID `root`." The "root" attribute that `beginRendering` had has been removed. The client is expected to render as soon as it has a valid tree with a root component.
+- **New Requirement**: `createSurface` now requires a **`catalogId`** (URI) to explicitly state which component set is being used.
+
+**Example:**
+
+**v0.8 (`beginRendering`)**:
+```json
+{
+  "beginRendering": {
+    "surfaceId": "user_profile_card",
+    "root": "root",
+    "styles": {
+      "primaryColor": "#007bff",
+    }
+  }
+}
+```
+
+**v0.9 (`createSurface`)**:
+```json
+{
+  "createSurface": {
+    "surfaceId": "user_profile_card",
+    "catalogId": "https://a2ui.dev/specification/0.9/standard_catalog_definition.json"
+  }
+}
+```
+
+## 4. Message Structure Comparison
+
+### 4.1. Component Updates
+
+**v0.8 (`surfaceUpdate`)**:
+
+- Components were wrapped in an object where the **key** was the component type.
+- **Structure**: `{ "id": "...", "component": { "Text": { "text": "..." } } }`
+
+**v0.9 (`updateComponents`)**:
+
+- **Renamed**: `surfaceUpdate` -> `updateComponents`.
+- **Refactored**: Components use a flattened structure with a const discriminator property `component`.
+- **Structure**: `{ "id": "...", "component": "Text", "text": "..." }`
+- **Reason**: This "flat" structure with a discriminator field (`component: "Text"`) is much easier for LLMs to generate consistently than a dynamic key (`"Text": {...}`). It also simplifies polymorphism in many JSON parsers.
+
+Specifying an unknown surfaceId will cause an error. It is recommended that clients implement a namespacing scheme internally to prevent separate agents from creating surfaces with the same ID, and to prevent agents from modifying surfaces created by other agents.
+
+#### Side-by-Side Example
+
+**v0.8:**
+
+```json
+{
+  "surfaceUpdate": {
+    "surfaceId": "main",
+    "components": [
+      {
+        "id": "title",
+        "component": {
+          "Text": { "text": { "literalString": "Hello" } }
+        }
+      }
+    ]
+  }
+}
+```
+
+**v0.9:**
+
+```json
+{
+  "updateComponents": {
+    "surfaceId": "main",
+    "components": [
+      {
+        "id": "root",
+        "component": "Column",
+        "children": ["title"]
+      },
+      {
+        "id": "title",
+        "component": "Text",
+        "text": "Hello"
+      }
+    ]
+  }
+}
+```
+
+### 4.2. Data Model Updates
+
+**v0.8 (`dataModelUpdate`)**:
+
+- **Adjacency List**: The `contents` property was an **array** of key-value pair objects.
+- **Typed Values**: Each entry required explicit typing like `valueString`, `valueNumber`, `valueBoolean`.
+- **Structure**: `[{ "key": "name", "valueString": "Alice" }]`
+
+**v0.9 (`updateDataModel`)**:
+
+- **Renamed**: `dataModelUpdate` -> `updateDataModel`.
+- **Standard JSON**: The `value` property is now a standard **JSON object**.
+- **Op**: The `op` property is added to allow for more complex updates (e.g., `replace`, `remove`).
+- **Structure**: `{ "name": "Alice" }`
+- **Reason**: LLMs are trained to generate JSON objects. Forcing them to generate an "adjacency list" representation of a map was inefficient and error-prone.
+
+## 5. Data Binding & State
+
+### 5.1. Standardization of `path`
+
+**v0.8:**
+
+- Used `dataBinding` in `childrenProperty` templates.
+- Used `path` in `BoundValue` objects.
+- Inconsistent terminology.
+
+**v0.9:**
+
+- **Unified**: Everything is now **`path`**.
+- **Reason**: Reduces cognitive load for the LLM. "Path" always means "JSON Pointer to data."
+
+### 5.2. Simplified Bound Values
+
+**v0.8:**
+
+- `{ "literalString": "foo" }` or `{ "path": "/foo" }`.
+- Explicit typing in keys (`literalNumber`, `literalBoolean`).
+
+**v0.9:**
+
+- **Implicit Typing**: `stringOrPath`, `numberOrPath`, etc. are defined in `common_types.json`.
+- **Structure**: The schema allows `string` OR `{ "path": "..." }`.
+- **Reason**: Much more natural JSON. `{ "text": "Hello" }` is valid. `{ "text": { "path": "/msg" } }` is valid. No need for `{ "text": { "literalString": "Hello" } }`.
+
+## 6. Component-Specific Changes
+
+### 6.1. Button Context
+
+**v0.8:**
+
+- **Array of Pairs**: `context: [{ "key": "id", "value": "123" }]`
+- **Reason**: Easy to parse, hard to generate.
+
+**v0.9:**
+
+- **Standard Map**: `context: { "id": "123" }`
+- **Reason**: Token efficiency. LLMs understand JSON objects as maps natively.
+
+### 6.2. TextField
+
+**v0.8:**
+
+- Property: `textFieldType` (e.g., "email", "password").
+
+**v0.9:**
+
+- Property: **`usageHint`**.
+- **Reason**: Consistency with `Text` and `Image` components which already used `usageHint`.
+
+### 6.3. ChoicePicker (vs MultipleChoice)
+
+**v0.8:**
+
+- Component: **`MultipleChoice`**.
+- Properties: `selections` (array), `maxAllowedSelections` (integer).
+
+**v0.9:**
+
+- Component: **`ChoicePicker`**.
+- Properties: **`value`** (array), **`usageHint`** (enum: `multipleSelection`, `mutuallyExclusive`). The `maxAllowedSelections` property was removed.
+- **Reason**: `ChoicePicker` is a more generic name that covers both radio buttons (mutually exclusive) and checkboxes (multiple selection). The `usageHint` controls the behavior, simplifying the component surface area.
+
+### 6.4. Slider
+
+**v0.8:**
+
+- Properties: `minValue`, `maxValue`.
+
+**v0.9:**
+
+- Properties: **`min`**, **`max`**.
+- **Reason**: Standardizing on shorter, more common property names.
+
+## 7. Error Handling
+
+**v0.9** introduces a strict **`ValidationFailed`** error format in `client_to_server.json`.
+
+- **Purpose**: To allow the "Prompt-Generate-Validate" loop to work effectively.
+- **Mechanism**: If the LLM generates invalid JSON, the system sends back a structured error:
+  ```json
+  {
+    "error": {
+      "code": "VALIDATION_FAILED",
+      "surfaceId": "...",
+      "path": "/components/0/text",
+      "message": "Expected string, got number"
+    }
+  }
+  ```
+- **Result**: The LLM sees this and can "self-correct" in the next turn.

specification/0.9/docs/evolution_guide.md

@@ -93,6 +93,7 @@ Instructions:
 - If label text is similar but not exact, you can still pass the test as long as the meaning is the same. (e.g. "Cancel" vs "Cancel Order")
 - If the generated output is missing a component that is specified in the user request, it is required to exist in the output in order to pass the test. If it is not specified, it is not required.
 - If the request is vague about the contents of a label or other property, you can still pass the test as long as it can be construed as matching the intent.
+- Unless explicitly required to be absent by the user request, extra components or attributes are allowed.
 
 Severity Definitions:
 - Minor: Merely cosmetic or a slight deviation from the request.

specification/0.9/eval/src/evaluation_flow.ts

@@ -41,19 +41,17 @@ export const componentGeneratorFlow = ai.defineFlow(
 The output MUST be a series of JSON objects, each enclosed in a markdown code block (or a single block with multiple objects).
 
 Standard Instructions:
-1. Generate a 'updateComponents' message with surfaceId 'main' and catalogId 'https://a2ui.dev/specification/0.9/standard_catalog_definition.json' containing the requested UI.
-2. Ensure all component children are referenced by ID (using the 'children' or 'child' property with IDs), NOT nested inline as objects.
-3. If the request involves data binding, you may also generate 'updateDataModel' messages.
-4. Among the 'updateComponents' messages in the output, there MUST be one root component with id: 'root'.
-5. Components need to be nested within a root layout container (Column, Row). No need to add an extra container if the root is already a layout container.
-6. There shouldn't be any orphaned components: no components should be generated which don't have a parent, except for the root component.
-7. Do NOT output a list of lists (e.g. [[...]]). Output individual JSON objects separated by newlines.
-8. STRICTLY follow the JSON Schemas. Do NOT add any properties that are not defined in the schema. Ensure ALL required properties are present.
-9. Do NOT invent data bindings or action contexts. Only use them if the prompt explicitly asks for them.
-10. Read the 'description' field of each component in the schema carefully. It contains critical usage instructions (e.g. regarding labels, single child limits, and layout behavior) that you MUST follow.
-11. **CRITICAL: 'weight' is a TOP-LEVEL property.** Do NOT put 'weight' inside 'props'. It must be a sibling of 'id' and 'props'.
-    INCORRECT: { "id": "...", "props": { "component": "...", "weight": 1 } }
-    CORRECT:   { "id": "...", "weight": 1, "props": { "component": "..." } }
+1. Generate a 'createSurface' message with surfaceId 'main' and catalogId 'https://a2ui.dev/specification/0.9/standard_catalog_definition.json'.
+2. Generate a 'updateComponents' message with surfaceId 'main' containing the requested UI.
+3. Ensure all component children are referenced by ID (using the 'children' or 'child' property with IDs), NOT nested inline as objects.
+4. If the request involves data binding, you may also generate 'updateDataModel' messages.
+5. Among the 'updateComponents' messages in the output, there MUST be one root component with id: 'root'.
+6. Components need to be nested within a root layout container (Column, Row). No need to add an extra container if the root is already a layout container.
+7. There shouldn't be any orphaned components: no components should be generated which don't have a parent, except for the root component.
+8. Do NOT output a list of lists (e.g. [[...]]). Output individual JSON objects separated by newlines.
+9. STRICTLY follow the JSON Schemas. Do NOT add any properties that are not defined in the schema. Ensure ALL required properties are present.
+10. Do NOT invent data bindings or action contexts. Only use them if the prompt explicitly asks for them.
+11. Read the 'description' field of each component in the schema carefully. It contains critical usage instructions (e.g. regarding labels, single child limits, and layout behavior) that you MUST follow.
 12. Do NOT define components inline inside 'child' or 'children'. Always use a string ID referencing a separate component definition.
 13. Do NOT use a 'style' property. Use standard properties like 'alignment', 'distribution', 'usageHint', etc.
 14. Do NOT invent properties that are not in the schema. Check the 'properties' list for each component type.