feat(genui): support functions in prompts and verify rendering#921
Open
gspencergoog wants to merge 20 commits into
Open
feat(genui): support functions in prompts and verify rendering#921gspencergoog wants to merge 20 commits into
gspencergoog wants to merge 20 commits into
Conversation
- Update PromptBuilder to include available functions in the prompt when present in the catalog. - Add a widget test to verify that function output (specifically pluralize) renders correctly. - Add a test to verify that PromptBuilder includes the functions section. Follow-up for flutter#874 and flutter#873.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
- Refactor `PromptBuilder` to load `server_to_client.json` and `common_types.json` as assets and use `$refs` in the generated prompt. - Make `PromptBuilder` creation asynchronous to support asset loading. - Fix examples (`composer`, `simple_chat`, `travel_app`) to use the new async API. - Add mock asset handlers in tests to support loading schemas. - Update golden files for prompt builder tests. Resolves flutter#873 and flutter#874.
gspencergoog
force-pushed
the
add_prompt_functions
branch
from
1a6de3d to
4134cb3
Compare
- Use a `Set` to avoid duplicate entries when generating the `required` list for components in the catalog schema. - Update golden files to reflect the fix. - Add mock asset handler to `travel_app` tests to fix failing test. Follow-up for flutter#873.
gspencergoog
force-pushed
the
add_prompt_functions
branch
from
c689ca3 to
1affaab
Compare
- Update `A2uiSchemas` to use `Schema.combined($ref: '...')` for `DynamicString`, `DynamicNumber`, `DynamicBoolean`, `Action`, and `DynamicStringList`. - Update golden files to reflect the use of refs in the generated prompt. - Add a new test group for custom components in `prompt_builder_test.dart`. Follow-up for flutter#873.
… button test - Modify `a2ui_schemas.dart` to use absolute `file://` paths for schema `$refs` pointing to `common_types.json` to allow resolution in memory-based validation in tests. - Post-process the generated prompt in `prompt_builder.dart` to strip absolute paths, ensuring compact AI prompts. - Avoid async hangs in `button_test.dart` by returning `Stream.error` directly instead of manual `StreamController` timing hooks. - Fix `StateError` in `SurfaceController.reportError` by ensuring it does not add messages to a closed `_onSubmit` stream after dispose. - Clean up remaining debug prints in `catalog_item.dart`, `button.dart`, and `surface_controller.dart` to keep test output clean.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…lution in prompt generation and tests - Remove VM-only `dart:io` import and local file URI resolution in production `prompt_builder.dart` and `a2ui_schemas.dart`. - Use a constant, platform-agnostic URI for `common_types.json` schema refs, improving compatibility with Flutter for web. - Preload the local `common_types.json` schema into `SchemaRegistry` in test utilities (`validation.dart` and `validation_test_utils.dart`) to allow offline, VM-safe reference resolution in tests.
…roperties in prompt_builder - Replace unsafe dynamic cast `as List?` with type checks using `is` when building required properties list in `prompt_builder.dart` for catalog schema generation.
…o eliminate code duplication - Refactor `stringReference` in `a2ui_schemas.dart` to compose `enumValues` constraints with the referenced `DynamicString` schema using `allOf`. - Eliminates redundant specification of literal string, DataBinding, and FunctionCall structures, resolving code duplication and easing schema maintenance.
…k trace and state leakage to AI server - Create platform-agnostic `A2uiFunctionException` to represent sanitized client-side function failures. - Implement Layer 3 Boundary Handlers in `button.dart` to intercept dynamic action VM crashes, log the full traceback locally to stdout/logs, and wrap/re-throw them under a generic `A2uiFunctionException`. - Implement Layer 2 Egress Gateway in `SurfaceController.reportError` to strictly type-check exceptions, mask unexpected system crashes as generic `INTERNAL_ERROR`, and elide VM stack traces entirely from serialized JSON outgoing messages. - Add comprehensive unit and integration test suite in `test/error_boundary_test.dart` to verify clean error reporting and masking.
gspencergoog
force-pushed
the
add_prompt_functions
branch
from
5dae301 to
8aff417
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…og defs - Populate `$defs` section of generated catalog JSON in `prompt_builder.dart` with complete definitions for `theme`, `anyComponent`, and `anyFunction`. - Resolves broken external `$ref` references within prompt system schemas and provides a valid, complete spec mapping for AI models. - Regenerate facade prompt builder test golden files to align with the new definitions inside catalog schema definitions.
…c helper method - Extract duplicated `rootBundle.loadString` calls for loading `common_types.json` and `server_to_client.json` inside `createChat` and `createCustom` methods. - Introduce `_loadSchemas` private static helper method returning a modern Dart Record `(String, String)`, improving code deduplication and maintainability.
…aths inside constants - Define `commonTypesSchemaId`, `commonTypesAssetKey`, `serverToClientAssetKey`, and `commonTypesLocalPath` inside `constants.dart` as a single source of truth. - Refactor `a2ui_schemas.dart` and `prompt_builder.dart` to import `constants.dart` and reference these centralized strings instead of duplicating raw literals. - Cleans up all hardcoded path references, completely preventing schema URI mismatch issues.
…ostic path testing - Create individual symbolic links `assets/schemas/common_types.json` and `assets/schemas/server_to_client.json` pointing to the repository root submodule `submodules/a2ui` to sync with `upstream/main`'s relocation. - Remove obsolete `packages/genui/submodules` folder to eliminate nested submodule naming confusion. - Reference the new local symlinks inside `pubspec.yaml` assets block and `constants.dart` asset keys. - Support robust, path-agnostic loading of `common_types.json` inside `validation.dart` and `validation_test_utils.dart` to find files under both package-relative and workspace-root-relative contexts. - Exclude `packages/genui/submodules/**` inside root `analysis_options.yaml` to prevent static warnings inside nested symlink paths.
…alysis_options.yaml - Remove the `'packages/genui/submodules/**'` exclusion from `analysis_options.yaml` since the directory was deleted. - Keeps the root configuration perfectly aligned and clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request enhances the GenUI system by enabling the AI to recognize and utilize client-side functions defined in the catalog. It achieves this by updating the
PromptBuilderto inject function definitions into the system prompt and by providing the necessary JSON schemas for the AI to understand the expected interaction patterns. Additionally, the changes improve system robustness by introducing structured error handling for function execution and masking sensitive internal error details, while also adding new tests to validate these capabilities.Highlights
Fixes #874 and related to #873.