You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This document provides context for the Gemini Code Assistant to understand the `genui` project.
4
4
5
-
A CatalogItem is an object which represents a widget that can be instantiated by an LLM. It centralizes the widgetBuilder function for the widget along with the data schema and name.
5
+
## Project Overview
6
6
7
-
## Structure of a CatalogItem
7
+
This is a monorepo for a Generative UI SDK (`flutter_genui`). The SDK allows developers to add interactive, dynamic, and graphical UI to their applications, generated by a Large Language Model (LLM). Instead of rendering static text responses from an LLM, this SDK allows the LLM to compose UIs from a developer-provided widget catalog.
8
8
9
-
- Only the variable that defines the CatalogItem should be public. Everything else should be private.
10
-
- The dataSchema must be defined as a Schema object. Look at `packages/flutter_genui/lib/src/catalog/core_widgets/elevated_button.dart` as a guide.
11
-
- Remember to use `Schema.object` and `Schema.enumString` etc.
12
-
- The Schema should _not_ have a "props" member or an "id" member, as those will be injected at a higher level. The schema should just be an object that includes all the properties that are specific to this widget, e.g. content to display.
13
-
- The only imports should be 'package:flutter/material.dart' and the import for CatalogItem - this will be '../../model/catalog_item.dart' for the SDK, or 'package:flutter_genui/flutter_genui.dart' for the example apps.
14
-
- The name of the CatalogItem variable should be in lowerCamelCase (e.g. `myWidgetName`).
15
-
- The name of the CatalogItem itself (the "name" parameter) should be in PascalCase (e.g. "MyWidgetName").
16
-
- Widgets that require controllers or state in order to instantly reflect user inputs (e.g. Radio buttons, checkboxes, text fields) should be implemented as StatefulWidgets which maintain internal UI state. Look at `packages/flutter_genui/lib/src/catalog/core_widgets/radio_group.dart` as an example. There should be a private StatefulWidget class definition in the file.
17
-
- The input schema for CatalogItems should not include parallel lists of data - instead use a single list of object where it makes sense.
18
-
- If you are implementing a CatalogItem that needs to compose other CatalogItems, include a "child" or "children" parameter of String type, which will contain the ID of the child to reference. When building the child or children, use the buildChild function and pass in the string. See `packages/flutter_genui/lib/src/catalog/core_widgets/elevated_button.dart` and `packages/flutter_genui/lib/src/catalog/core_widgets/column.dart` as examples.
19
-
- All interactive UI elements where the user can input data e.g. by typing or selecting from options, or take action by clicking buttons etc, should handle those actions by calling the `dispatchEvent` callback, and including all the context needed for the LLM to understand what the user has done. E.g. if they have selected a specific option, the handler call should include the name of the selected option as the value.
20
-
- All properties in a schema are optional by default. Use the `required` list to specify required properties.
21
-
- When parsing a catalog item in the widgetBuilder, use extension types for the data required by the catalog item, similar to `examples/travel_app/lib/src/catalog/travel_carousel.dart`.
9
+
The project is structured as a monorepo containing several Dart and Flutter packages, along with example applications.
22
10
23
-
##How to create a new a CatalogItem
11
+
### Key Packages
24
12
25
-
1. Understand the structure of a CatalogItem by looking at `packages/flutter_genui/lib/src/model/catalog_item.dart` for the type, and `packages/flutter_genui/lib/src/catalog/core_widgets/elevated_button.dart` as an example to follow.
26
-
2. Create a new file in the relevant app, which contains a declaration of a global variable with a CatalogItem. The location of the file should be:
|`packages/flutter_genui`| The core framework for employing Generative UI. |
16
+
|`packages/flutter_genui_firebase_ai`| Firebase AI integration for `flutter_genui`. |
17
+
|`packages/dart_schema_builder`| A Dart JSON Schema package with validation, used by the core framework. |
27
18
28
-
- For the generic flutter_genui SDK: `packages/flutter_genui/lib/src/catalog/`
29
-
- For an example app, at `examples/[MY_APP]/lib/src/catalog/` e.g. for the travel_app: `examples/travel_app/lib/src/catalog/`
19
+
### Example Applications
30
20
31
-
3. Review your implementation and compare it to the examples, to ensure that the use of parameters matches and the structure of the code is similar. Fix any mistakes.
32
-
4. Write a test for the CatalogItem and run it, fixing any mistakes.
33
-
5. Update the Catalog definition for the app or SDK to include the new item.
21
+
The `examples` directory contains sample applications demonstrating the usage of the `flutter_genui` SDK.
|`simple_chat`| A minimal example of a conversational chat application. It demonstrates the fundamental concepts of `flutter_genui`, such as initializing the `UiAgent`, sending user messages, and rendering the AI-generated UI surfaces using the default core widget catalog. |
26
+
|`travel_app`| A more advanced example of a travel planning assistant. It showcases dynamic UI generation, the use of a custom, domain-specific widget catalog, and how user interactions with the UI can be fed back to the AI to refine the conversation. |
27
+
|`catalog_gallery`| A simple application that displays the widgets available in the catalog. It's a useful tool for developers to visualize the components that the AI can use to build UIs. |
36
28
37
-
When the CatalogItem API changes, it is necessary to update all the existing catalog items to match.
29
+
The `simple_chat` and `travel_app` examples are good starting points for understanding the library's capabilities.
38
30
39
-
1. Understand the change that has been requested, and carefully update the code at `packages/flutter_genui/lib/src/model/catalog_item.dart`.
40
-
2. Update all places that create and use CatalogItems e.g. `packages/flutter_genui/lib/src/model/catalog.dart`. Consider all the code at `packages/flutter_genui/lib/*` when doing this.
41
-
3. Find all CatalogItem implementations which will be at `examples/travel_app/lib/src/catalog/*`, `packages/flutter_genui/lib/src/catalog/*` etc. Update each of them to match the changes in the API.
31
+
## Implementation Details
42
32
43
-
## How to update every CatalogItem
33
+
For a deeper understanding of the project's architecture and data flow, refer to the following documents:
44
34
45
-
1. Find all CatalogItem implementations which will be at `examples/travel_app/lib/src/catalog/*`, `packages/flutter_genui/lib/src/catalog/*` etc. Update each of them.
35
+
-**`packages/flutter_genui/IMPLEMENTATION.md`**: Provides a comprehensive overview of the core `flutter_genui` package's architecture, purpose, and implementation.
36
+
-**`examples/travel_app/IMPLEMENTATION.md`**: Describes the architecture and implementation of the `travel_app` example, showcasing how the `flutter_genui` package is used to build a dynamic, conversational UI.
37
+
38
+
## Building and Running
39
+
40
+
The project uses standard `flutter` and `dart` commands. A comprehensive script is provided to automate fixes, formatting, analysis, and testing.
41
+
42
+
### Key Commands
43
+
44
+
-**Run all checks and tests:**
45
+
46
+
```bash
47
+
./tool/run_all_tests_and_fixes.sh
48
+
```
49
+
50
+
This is a script used by CI for ensuring code quality. It runs `dart fix`, `dart format`, `flutter test`, and `flutter analyze` for all packages and examples in the repository.
51
+
52
+
## Development Conventions
53
+
54
+
### Code Style and Formatting
55
+
56
+
- The project follows the `dart_flutter_team_lints` linting rules, which is the combination of this [analysis_options.yaml](https://raw.githubusercontent.com/dart-lang/ecosystem/refs/heads/main/pkgs/dart_flutter_team_lints/lib/analysis_options.yaml) file and the [recommended](https://raw.githubusercontent.com/dart-lang/core/refs/heads/main/pkgs/lints/lib/recommended.yaml) and [core](https://raw.githubusercontent.com/dart-lang/core/refs/heads/main/pkgs/lints/lib/core.yaml) lints.
57
+
- Code formatting is enforced using the `dart_format` tool.
58
+
- The `tool/run_all_tests_and_fixes.sh` script should be run before committing to ensure all files are correctly formatted and analyzed.
59
+
60
+
### Testing
61
+
62
+
- Widget and unit tests are located in the `test` directory of each package/example.
63
+
- Tests are run using `flutter test`.
64
+
- The CI pipeline, defined in `.github/workflows/flutter_packages.yaml`, runs tests for all packages on every push and pull request to the `main` branch.
65
+
66
+
### Copyright Headers
67
+
68
+
- All files must have a copyright header.
69
+
- The `tool/fix_copyright.sh` script (which is called by `run_all_tests_and_fixes.sh`) can be used to automatically add or update copyright headers.
70
+
71
+
### Firebase Integration
72
+
73
+
- The examples and the `flutter_genui_firebase_ai` package use Firebase.
74
+
- A script at `tool/stub_firebase_options.sh` is used in CI to create a stub `firebase_options.dart` file. For local development, developers need to configure their own Firebase project by following the instructions in `packages/flutter_genui/USAGE.md`.
46
75
47
76
## Folder `spikes`
48
77
@@ -53,5 +82,4 @@ Skip this folder when reviewing code.
53
82
54
83
## Draft pull requests
55
84
56
-
Do not review pull requests, when they are in draft state.
57
-
Wait them to be ready for review.
85
+
Do not review pull requests when they are in draft state. Wait them to be ready for review.
Copy file name to clipboardExpand all lines: examples/travel_app/IMPLEMENTATION.md
+34-19Lines changed: 34 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -67,34 +67,49 @@ classDiagram
67
67
68
68
### 1. UI Layer (`main.dart`)
69
69
70
-
This is the main entry point and the visible part of the application, primarily managed by the `TravelPlannerPage` widget. Its responsibilities are:
70
+
This is the main entry point for the application. Its responsibilities are:
71
71
72
-
-**Initialization**: It initializes Firebase, the `GenUiManager`, and the `AiClient`.
73
-
-**UI Scaffolding**: It provides the basic structure of the app, which consists of an app bar, a `ConversationWidget` to render the conversation history, and a custom chat input field.
74
-
-**State Management & App Logic**: It holds the core application logic. It manages the `_conversation` history list, sends prompts to the AI via `_triggerInference`, and handles UI events returned from the `GenUiSurface` widgets.
72
+
-**Initialization**: It initializes Firebase and loads the asset image catalog.
73
+
-**UI Scaffolding**: It sets up the root `MaterialApp` and a `Scaffold` with a `TabBar`. The app has two main tabs:
74
+
-**"Travel"**: This tab contains the `TravelPlannerPage`, which is the primary interface for the conversational travel agent.
75
+
-**"Widget Catalog"**: This tab displays a `CatalogView` from the `flutter_genui_dev` package, allowing developers to browse and inspect all the available UI components in the app's catalog.
75
76
76
-
### 2. UI State Management Layer ([`package:flutter_genui`](../../packages/flutter_genui/IMPLEMENTATION.md))
The core application logic resides in the `TravelPlannerPage` widget. Its responsibilities are:
80
+
81
+
-**Initialization**: It initializes the `GenUiManager` and the `AiClient`.
82
+
-**State Management**: It manages the `_conversation` history list, sends prompts to the AI via `_triggerInference`, and handles UI events returned from the `GenUiSurface` widgets.
83
+
-**UI Rendering**: It provides the structure for the chat interface, which consists of a `Conversation` widget to render the conversation history and a custom chat input field.
84
+
85
+
### 3. UI State Management Layer ([`package:flutter_genui`](../../packages/flutter_genui/IMPLEMENTATION.md))
77
86
78
87
The components from the `flutter_genui` package are the central orchestrators of the dynamic UI state.
79
88
80
89
-**`GenUiManager`**: The core state manager. It maintains the definitions for all active UI "surfaces", provides the UI manipulation tools (`addOrUpdateSurface`, `deleteSurface`) to the app logic, and notifies listening widgets (like `TravelPlannerPage`) of changes via a stream.
81
-
-**`ConversationWidget`**: A facade widget that renders a list of `ChatMessage`s. It is responsible for displaying the conversation history, including text messages and dynamically rendered UI surfaces via `GenUiSurface`.
90
+
-**`Conversation` widget**: A widget defined in the app (`lib/src/widgets/conversation.dart`) that renders a list of `ChatMessage`s. It is responsible for displaying the conversation history, including text messages and dynamically rendered UI surfaces via `GenUiSurface`.
The `GeminiAiClient` class, also part of `flutter_genui`, abstracts the communication with the underlying generative AI model (a Google Gemini model via Firebase). It handles the API calls to the model, sending prompts and receiving the model's responses, including the tool calls that drive the UI generation.
94
+
The `FirebaseAiClient` class, from the `flutter_genui_firebase_ai` package, abstracts the communication with the underlying generative AI model (a Google Gemini model via Firebase). It handles the API calls to the model, sending prompts and receiving the model's responses, including the tool calls that drive the UI generation.
86
95
87
-
### 4. Widget Catalog (The "Tools")
96
+
### 5. Widget Catalog (The "Tools")
88
97
89
98
This is the collection of predefined UI components that the AI can use to construct the interface. It acts as the "API" that the AI targets.
90
99
91
100
-**Definition**: The catalog is defined in `lib/src/catalog.dart` as a `Catalog` instance, which is a list of `CatalogItem`s.
92
-
-**Custom Components**: The travel app defines several custom `CatalogItem`s in the `lib/src/catalog/` directory, such as `travel_carousel`, `itinerary_with_details`, and `filter_chip_group`.
93
-
-**Standard Components**: It also uses standard, pre-built components from `flutter_genui` like `column`, `text`, `elevatedButton`, etc.
101
+
-**Custom Components**: The travel app defines several custom `CatalogItem`s in the `lib/src/catalog/` directory. Key components include:
102
+
-`TravelCarousel`: For displaying a horizontal list of selectable options.
103
+
-`ItineraryWithDetails`, `ItineraryDay`, `ItineraryEntry`: For building structured travel plans.
104
+
-`InputGroup`: A container for grouping various input widgets.
105
+
-`OptionsFilterChipInput`, `CheckboxFilterChipsInput`, `TextInputChip`: Different types of input chips for user selections.
106
+
-`InformationCard`: For displaying detailed information about a topic.
107
+
-`Trailhead`: For suggesting follow-up prompts to the user.
108
+
-**Standard Components**: It also uses standard, pre-built components from `flutter_genui` like `column`, `text`, `image`, etc.
94
109
95
110
## Data Flow: The Generative UI Cycle
96
111
97
-
The diagram below shows the sequence of events from user input to UI rendering. The application logic in `main.dart` is responsible for driving this cycle.
112
+
The diagram below shows the sequence of events from user input to UI rendering. The application logic in `lib/src/travel_planner_page.dart` is responsible for driving this cycle.
98
113
99
114
```mermaid
100
115
sequenceDiagram
@@ -121,14 +136,14 @@ sequenceDiagram
121
136
deactivate AiClient
122
137
123
138
note right of AppLogic: The AiClient invokes the tool, which calls a method on GenUiManager.
124
-
139
+
125
140
activate GenUiManager
126
141
GenUiManager->>GenUiManager: Updates internal state
127
142
GenUiManager-->>AppLogic: Notifies of state change via Stream
128
143
deactivate GenUiManager
129
144
130
145
AppLogic->>AppLogic: Updates _conversation list with AiUiMessage
131
-
146
+
132
147
activate ConversationWidget
133
148
AppLogic->>ConversationWidget: Rebuilds with new message list
134
149
ConversationWidget->>ConversationWidget: Renders new UI surface
@@ -141,7 +156,7 @@ sequenceDiagram
141
156
142
157
### The System Prompt
143
158
144
-
The interaction with the model is heavily guided by a detailed **system prompt** defined in `main.dart`. This prompt is critical to the application's success. It instructs the model on:
159
+
The interaction with the model is heavily guided by a detailed **system prompt** defined in `lib/src/travel_planner_page.dart`. This prompt is critical to the application's success. It instructs the model on:
145
160
146
161
-**Persona**: To act as a helpful travel agent.
147
162
-**Conversation Flow**: To first ask clarifying questions and then present results.
@@ -208,8 +223,8 @@ graph TD
208
223
209
224
### User Interaction and Events
210
225
211
-
The UI is not just for display; it's interactive. Widgets like `optionsFilterChip`and `filterChipGroup` can capture user input.
226
+
The UI is not just for display; it's interactive. Widgets like `InputGroup`, `OptionsFilterChipInput`, and `TravelCarousel` can capture user input.
212
227
213
-
-**Event Dispatching**: The `widgetBuilder` receives a `dispatchEvent` function. This function is called in response to user actions (like a button press or item selection), creating a `UiEvent` (`UiActionEvent` for submissions, `UiChangeEvent` for value changes).
214
-
-**Event Handling**: The `onEvent` callback on the `GenUiSurface` widget is triggered. The application logic in `main.dart` receives the event via a `UiEventManager`.
215
-
-**Conversation Update**: The app logic processes the event, wraps the information into a new `UserMessage`, adds it to the conversation history, and sends it to the model on the next turn. This informs the model of the user's actions, allowing it to respond accordingly (e.g., refining a search based on a selected filter).
228
+
-**Event Dispatching**: The `widgetBuilder`for each catalog item receives a `dispatchEvent` function. This function is called in response to user actions (like a button press or item selection), creating a `UiEvent` (e.g., `UiActionEvent`).
229
+
-**Event Handling**: The `TravelPlannerPage` listens to the `genUiManager.onSubmit` stream. When an event is dispatched from a widget, the `GenUiManager` processes it and emits a `UserMessage` on this stream.
230
+
-**Conversation Update**: The `TravelPlannerPage`'s stream listener (`_handleUserMessageFromUi`) receives the `UserMessage`, adds it to the conversation history, and triggers a new inference call to the model. This informs the model of the user's actions, allowing it to respond accordingly (e.g., refining a search based on a selected filter).
0 commit comments