Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 16 additions & 17 deletions .gemini/GEMINI.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,41 +3,40 @@
A CatalogItem is an object which represents a widget that can be instantiated by an LLM. It centralizes the widgetBuilder function for the widget along with the data schema and name.

## Structure of a CatalogItem

- Only the variable that defines the CatalogItem should be public. Everything else should be private.
- The dataSchema must be defined as a Schema object. Look at pkgs/flutter_genui/lib/src/catalog/elevated_button.dart as a guide. Remember to use `Schema.object` and `Schema.enumString` etc.
- The Schema should *not* have a "props" member or an "id" member, as those will be injected at a higher level. The schema should just be an object that includes all the properties that are specific to this widget, e.g. content to display.
- The dataSchema must be defined as a Schema object. Look at `pkgs/flutter_genui/lib/src/catalog/core_widgets/elevated_button.dart` as a guide.
- Remember to use `Schema.object` and `Schema.enumString` etc.
- The Schema should _not_ have a "props" member or an "id" member, as those will be injected at a higher level. The schema should just be an object that includes all the properties that are specific to this widget, e.g. content to display.
- The only imports should be 'package:firebase_ai/firebase_ai.dart', 'package:flutter/material.dart' and the import for CatalogItem - this will be '../../model/catalog_item.dart' for the SDK, or 'package:flutter_genui/flutter_genui.dart' for the example apps. Any other utilities etc that are needed should be inlined into the file.
- The name of the CatalogItem should be in lower camel case.
- Widgets that require controllers or state in order to instantly reflect user inputs (e.g. Radio buttons, checkboxes, text fields) should be implemented as StatefulWidgets which maintain internal UI state. Look at libs/src/widgets/radio_group.dart as an example. There should be a private StatefulWidget class definition in the file.
- Widgets that require controllers or state in order to instantly reflect user inputs (e.g. Radio buttons, checkboxes, text fields) should be implemented as StatefulWidgets which maintain internal UI state. Look at `libs/src/widgets/radio_group.dart` as an example. There should be a private StatefulWidget class definition in the file.
- The input schema for CatalogItems should not include parallel lists of data - instead use a single list of object where it makes sense.
- If you are implementing a CatalogItem that needs to compose other CatalogItems, include a "child" or "children" parameter of String type, which will contain the ID of the child to reference. When building the child or children, use the buildChild function and pass in the string. See pkgs/flutter_genui/lib/src/catalog/elevated_button.dart and pkgs/flutter_genui/lib/src/catalog/column.dart as examples.
- If you are implementing a CatalogItem that needs to compose other CatalogItems, include a "child" or "children" parameter of String type, which will contain the ID of the child to reference. When building the child or children, use the buildChild function and pass in the string. See `pkgs/flutter_genui/lib/src/catalog/elevated_button.dart` and `pkgs/flutter_genui/lib/src/catalog/column.dart` as examples.
- All interactive UI elements where the user can input data e.g. by typing or selecting from options, or take action by clicking buttons etc, should handle those actions by calling the `dispatchEvent` callback, and including all the context needed for the LLM to understand what the user has done. E.g. if they have selected a specific option, the handler call should include the name of the selected option as the value.
- IMPORTANT: `required` is *not* a parameter of Schema.object! For this Schema class, all parameters are required by default. Instead, there is an `optionalProperties` which you specify if there are any optional parameters.
- IMPORTANT: `required` is _not_ a parameter of Schema.object! For this Schema class, all parameters are required by default. Instead, there is an `optionalProperties` which you specify if there are any optional parameters.
- When parsing a catalog item in the widgetBuilder, use extension types for the data required by the catalog item, similar to `examples/travel_app/lib/src/catalog/travel_carousel.dart`.

## How to create a new a CatalogItem

1. Understand the structure of a CatalogItem by looking at pkgs/flutter_genui/lib/src/model/catalog_item.dart for the type, and pkgs/flutter_genui/lib/src/catalog/elevated_button.dart as an example to follow.

1. Understand the structure of a CatalogItem by looking at `pkgs/flutter_genui/lib/src/model/catalog_item.dart` for the type, and `pkgs/flutter_genui/lib/src/catalog/elevated_button.dart` as an example to follow.
2. Create a new file in the relevant app, which contains a declaration of a global variable with a CatalogItem. The location of the file should be:
- For the generic flutter_genui SDK: pkgs/flutter_genui/lib/src/catalog/
- For an example app, at examples/<MY_APP>/lib/src/catalog/ e.g. for the travel_app: examples/travel_app/lib/src/catalog/

3. Review your implementation and compare it to the examples, to ensure that the use of parameters matchces and the structure of the code is similar. Fix any mistakes.
- For the generic flutter_genui SDK: `pkgs/flutter_genui/lib/src/catalog/`
- For an example app, at `examples/[MY_APP]/lib/src/catalog/` e.g. for the travel_app: `examples/travel_app/lib/src/catalog/`

3. Review your implementation and compare it to the examples, to ensure that the use of parameters matches and the structure of the code is similar. Fix any mistakes.
4. Write a test for the CatalogItem and run it, fixing any mistakes.

5. Update the Catalog definition for the app or SDK to include the new item.

## How to update the CatalogItem API

When the CatalogItem API changes, it is necessary to update all the existing catalog items to match.

1. Understand the change that has been requested, and carefully update the code at pkgs/flutter_genui/lib/src/model/catalog_item.dart

2. Update all places that create and use CatalogItems e.g. pkgs/flutter_genui/lib/src/model/catalog.dart. Consider all the code at pkgs/flutter_genui/lib/* when doing this.

3. Find all CatalogItem implementations which will be at examples/travel_app/lib/src/catalog/*, pkgs/flutter_genui/lib/src/catalog/* etc. Update each of them to match the changes in the API.
1. Understand the change that has been requested, and carefully update the code at `pkgs/flutter_genui/lib/src/model/catalog_item.dart`.
2. Update all places that create and use CatalogItems e.g. `pkgs/flutter_genui/lib/src/model/catalog.dart`. Consider all the code at `pkgs/flutter_genui/lib/*` when doing this.
3. Find all CatalogItem implementations which will be at `examples/travel_app/lib/src/catalog/*`, `pkgs/flutter_genui/lib/src/catalog/*` etc. Update each of them to match the changes in the API.

## How to update every CatalogItem

1. Find all CatalogItem implementations which will be at examples/travel_app/lib/src/catalog/*, pkgs/flutter_genui/lib/src/catalog/* etc. Update each of them.
1. Find all CatalogItem implementations which will be at `examples/travel_app/lib/src/catalog/*`, `pkgs/flutter_genui/lib/src/catalog/*` etc. Update each of them.
17 changes: 17 additions & 0 deletions .gemini/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Gemini Configuration

This directory (`.gemini/`) contains configuration and context files for Google's Gemini tools, including:

- **`gemini-cli`**: The command-line interface for Gemini.
- **Gemini Code Assist**: An AI-powered assistant that integrates with GitHub and IDEs and provides for automated code reviews, among other features.

The files in this directory are used to customize the behavior of these tools for this specific repository.

- **`GEMINI.md`**: Provides project-specific context, instructions, and guidelines that are included in the context when using Gemini CLI and Code Assist. This helps the AI understand the project's conventions and requirements.
- **`config.yaml`**: Configuration for the Gemini for GitHub tools, such as settings for code review.
- **`styleguide.md`**: Contains the project's style guide, which is used by the Gemini for Github tools to ensure that generated reviews adhere to the project's conventions.

## Documentation

- [Gemini CLI Documentation](https://github.com/google-gemini/gemini-cli)
- [Gemini Code Assist Documentation](https://cloud.google.com/products/gemini/code-assist)
86 changes: 86 additions & 0 deletions examples/travel_app/IMPLEMENTATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Travel App Implementation

This document provides a comprehensive overview of the architecture, purpose, and implementation of the Travel App example.

## Purpose

The Travel App is a demonstration of a generative UI application built using the `flutter_genui` package. It functions as a conversational travel agent assistant. Instead of a static, predefined user interface, the app's UI is dynamically generated by a large language model (LLM) in response to the user's prompts. This allows for a highly flexible and context-aware user experience, where the UI adapts to the flow of the conversation.

## Architecture

The application is structured into several distinct layers, each with a specific responsibility.

### 1. UI Layer (`main.dart`)

This is the main entry point and the visible part of the application. Its responsibilities are:

- **Initialization**: It initializes Firebase and Firebase App Check.
- **UI Scaffolding**: It provides the basic structure of the app, which consists of an app bar, a text input field for user prompts, a send button, and a main content area where the dynamic UI is rendered.
- **State Management**: It holds and manages the lifecycle of the `ConversationManager`, which is the core of the app's logic.

### 2. Conversation Management Layer ([`package:flutter_genui`](../../pkgs/flutter_genui/IMPLEMENTATION.md))

The `ConversationManager` from the `flutter_genui` package is the central orchestrator of the application.

- **Conversation Flow**: It manages the back-and-forth between the user and the AI model.
- **Prompt Handling**: It takes the user's text prompt, combines it with the conversation history and the system prompt, and sends it to the AI Client.
- **UI Generation**: It receives tool-call requests from the model, which are essentially instructions to render specific UI widgets.
- **Widget Rendering**: It looks up the requested widget in the `Catalog`, validates the provided data against the widget's schema, and uses the widget's builder function to create and display the Flutter widget.
- **State Tracking**: It maintains the state of the rendered UI, including the history of the conversation.

### 3. AI/Model Layer (`package:flutter_genui`)

The `AiClient` class, also part of `flutter_genui`, abstracts the communication with the underlying generative AI model (e.g., a Google Gemini model via Firebase). It handles the API calls to the model, sending prompts and receiving the model's responses, including the tool calls that drive the UI generation.

### 4. Widget Catalog (The "Tools")

This is the collection of predefined UI components that the AI can use to construct the interface. It acts as the "API" that the AI targets.

- **Definition**: The catalog is defined in `lib/src/catalog.dart` as a `Catalog` instance, which is a list of `CatalogItem`s.
- **Custom Components**: The travel app defines several custom `CatalogItem`s in the `lib/src/catalog/` directory, such as `travel_carousel`, `itinerary_with_details`, and `filter_chip_group`.
- **Standard Components**: It also uses standard, pre-built components from `flutter_genui` like `column`, `text`, `elevatedButton`, etc.

## Implementation Details

### The System Prompt

The interaction with the model is heavily guided by a detailed **system prompt** defined in `main.dart`. This prompt is critical to the application's success. It instructs the model on:

- **Persona**: To act as a helpful travel agent.
- **Conversation Flow**: To first ask clarifying questions and then present results.
- **Tool Usage**: Which widgets (tools) to use in different situations (e.g., use `travel_carousel` for options, `itinerary_with_details` for final results).
- **UI Style**: How to compose widgets for a good user experience (e.g., breaking up itineraries with other content).
- **Available Data**: It includes a JSON string (`imagesJson`) containing a list of available image URLs and descriptions, ensuring the model uses relevant and high-quality images.

### The Widget Catalog

The `catalog` is the cornerstone of the dynamic UI generation. It's a registry of all possible UI components the model can render.

Each component in the catalog is a `CatalogItem` with three key parts:

1. **`name`**: A unique string that the AI uses to identify the widget (e.g., `'travel_carousel'`).
2. **`dataSchema`**: A `Schema` object that defines the structure and types of data the widget expects. This schema is provided to the model so it knows what parameters to generate for the tool call. The descriptions within the schema are crucial for the model to understand the purpose of each parameter.
3. **`widgetBuilder`**: A Dart function that takes the data generated by the model (as a `Map<String, Object?>`) and returns a Flutter `Widget`.

### Catalog Item Definition

Each custom widget in `lib/src/catalog/` follows a consistent pattern:

1. **Schema Definition**: A `_schema` variable defines the data structure using `Schema.object`, `Schema.string`, `Schema.array`, etc.
2. **Data Accessor**: A Dart `extension type` is defined to provide type-safe access to the `Map<String, Object?>` data received from the model. This avoids manual casting and error-prone map access.
3. **`CatalogItem` Instance**: A final variable creates the `CatalogItem`, passing the `name`, `dataSchema`, and `widgetBuilder`.
4. **Widget Implementation**: A standard Flutter `StatelessWidget` or `StatefulWidget` that contains the actual UI code for the component.

### Dynamic UI Composition

Widgets can be nested to create complex layouts. This is achieved by having a widget's schema accept the ID of another widget as a parameter (e.g., the `child` property of `itinerary_with_details`).

The `widgetBuilder` for a container-like widget receives a `buildChild` function as an argument. It can call `buildChild(childId)` to recursively ask the `ConversationManager` to build and return the child widget, which is then placed in the parent's widget tree.

### User Interaction and Events

The UI is not just for display; it's interactive. Widgets like `optionsFilterChip` and `filterChipGroup` can capture user input.

- **Event Dispatching**: The `widgetBuilder` receives a `dispatchEvent` function. This function can be called in response to user actions (like a button press or item selection).
- **Event Data**: `dispatchEvent` sends an event type and a value back to the `ConversationManager`.
- **Conversation Update**: The `ConversationManager` adds this event information to the conversation history and sends it to the model on the next turn. This informs the model of the user's actions, allowing it to respond accordingly (e.g., refining a search based on a selected filter).
Loading
Loading