Integrate Kotlin SDK with A2UI conformance tests#1129
Integrate Kotlin SDK with A2UI conformance tests#1129jiahaog wants to merge 1 commit intogoogle:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request enhances the Kotlin A2UI validator by adding support for schema mappings, refactoring root ID discovery to handle multiple surfaces, and implementing a YAML-based conformance test suite. Key feedback includes addressing a potential O(N^2) performance bottleneck in the validation loop, removing debug print statements from the build script, and optimizing memory usage by moving heuristic reference sets to a companion object. Additionally, the repository root discovery logic in the new conformance tests should be updated to match the build script's directory-based check for consistency.
agent_sdks/kotlin/src/main/kotlin/com/google/a2ui/core/schema/Validator.kt
Show resolved
Hide resolved
agent_sdks/kotlin/src/main/kotlin/com/google/a2ui/core/schema/Validator.kt
Show resolved
Hide resolved
agent_sdks/kotlin/src/main/kotlin/com/google/a2ui/core/schema/Validator.kt
Show resolved
Hide resolved
agent_sdks/kotlin/src/test/kotlin/com/google/a2ui/core/schema/ConformanceTest.kt
Show resolved
Hide resolved
This commit groups the changes made to Validator.kt into 4 themes, each necessitated by specific failing tests in the conformance suite:
1. Heuristic Component Reference Resolution
- Changes: Added hardcoded field names ('child', 'children', 'explicitList') to extractComponentRefFields to identify references when not explicitly declared in schema.
- Necessity: Complex schemas (like in v0.9) didn't explicitly mark these as references, but Python validator assumed them.
- Failing Test: test_custom_catalog_0_9 (failed with "Component 'c1' is not reachable from 'root'")
2. Multi-Surface Validation Support
- Changes: Refactored validate to instantiate a new A2uiTopologyValidator for each message batch based on surfaceId, and updated findRootId to look up rootId per surface.
- Necessity: Conformance tests contain batches with messages for multiple surfaces, each needing independent root validation.
- Failing Test: test_validate_multi_surface_v08 (failed with "Missing root component: No component has id='root-a'")
3. Incremental Update Support (Nullable Root)
- Changes: Allowed findRootId to return null when no root-defining message is found, and updated topology validator to skip root check and traverse all nodes for cycles.
- Necessity: Incremental updates might not contain the root component in the current batch.
- Failing Test: test_incremental_update_no_root_v08 (failed with "Missing root component: No component has id='root'")
4. v0.9 createSurface Support
- Changes: Added recognition of createSurface messages in findRootId and defaulted root ID to 'root' for v0.9.
- Necessity: In v0.9, root is assumed to be 'root' when createSurface is used.
- Failing Test: test_validate_missing_root_v09 (failed because it expected failure but passed due to fallback to null root)
Also fixed findRepoRoot in build.gradle.kts to look for 'specification' directory instead of '.git', fixing resource loading in A2uiSchemaManagerTest.
cdff4e4 to
178a7c2
Compare
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request implements a data-driven conformance test suite for the Kotlin SDK validator and updates the A2uiValidator to support custom schema mappings. The validation logic was enhanced to handle multiple surfaces by filtering rootId lookups by surfaceId. Review feedback highlights opportunities to optimize performance by avoiding O(N^2) complexity in the validation loop and reducing object allocations in recursive reference extraction. Additionally, it is recommended to remove a debug println from the build script and improve null safety in the test assertions.
|
|
||
| components?.let { topologyValidator.validate(it) } | ||
| components?.let { | ||
| val rootId = findRootId(messages, surfaceId) |
There was a problem hiding this comment.
Calling findRootId(messages, surfaceId) inside the loop results in findRootId performs a full scan of the messages array for each message in the batch. Consider pre-calculating a mapping of surfaceId to rootId or using a cache to ensure
|
|
||
| val copySpecs by tasks.registering(Copy::class) { | ||
| val repoRoot = findRepoRoot() | ||
| println("REPO ROOT: ${repoRoot.absolutePath}") |
| val heuristicSingle = setOf("child", "contentChild", "entryPointChild", "detail", "summary", "root") | ||
| val heuristicList = setOf("children", "explicitList", "template") |
There was a problem hiding this comment.
| assertTrue( | ||
| regex.containsMatchIn(exception.message!!) || | ||
| exception.message!!.contains("Validation failed") || | ||
| exception.message!!.contains("Invalid JSON Pointer syntax"), | ||
| "Expected error matching '$expectError' or containing 'Validation failed', but got: ${exception.message}" | ||
| ) |
There was a problem hiding this comment.
Using the non-null assertion operator !! on exception.message multiple times is unsafe and can lead to a NullPointerException if the message is missing. It's better to capture the message safely and use it for assertions.
| assertTrue( | |
| regex.containsMatchIn(exception.message!!) || | |
| exception.message!!.contains("Validation failed") || | |
| exception.message!!.contains("Invalid JSON Pointer syntax"), | |
| "Expected error matching '$expectError' or containing 'Validation failed', but got: ${exception.message}" | |
| ) | |
| val actualMessage = exception.message ?: "" | |
| assertTrue( | |
| regex.containsMatchIn(actualMessage) || | |
| actualMessage.contains("Validation failed") || | |
| actualMessage.contains("Invalid JSON Pointer syntax"), | |
| "Expected error matching '$expectError' or containing 'Validation failed', but got: $actualMessage" | |
| ) |
This commit groups the changes made to Validator.kt into 4 themes, each necessitated by specific failing tests in the conformance suite:
Also fixed findRepoRoot in build.gradle.kts to look for 'specification' directory instead of '.git', fixing resource loading in A2uiSchemaManagerTest.
Description
Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.
List which issues are fixed by this PR. For larger changes, raising an issue first helps reduce redundant work.
Pre-launch Checklist
If you need help, consider asking for advice on the discussion board.