feat(extensions): validate providers declared in extension.yaml (part 1/2)#9033
Open
JeffreyCA wants to merge 2 commits into
Open
feat(extensions): validate providers declared in extension.yaml (part 1/2)#9033JeffreyCA wants to merge 2 commits into
JeffreyCA wants to merge 2 commits into
Conversation
github-actions
Bot
added
area/extensions
Contributor
There was a problem hiding this comment.
Pull request overview
Adds validation and CI coverage to ensure extension providers declared in extension.yaml are well-formed, capability-consistent, uniquely owned, and (for first-party extensions) kept in sync with the providers actually registered by the extension code.
Changes:
- Add provider validation to registry validation (
providers[]type/name uniqueness and required capability checks) plus a registry-level “unique provider ownership” check. - Introduce
azdext.VerifyProvidersMatchManifestand add per-extension tests to verify runtime registrations matchextension.yaml. - Add CI/workflow coverage to validate first-party manifests and keep provider-type schema enum in sync with Go constants.
Show a summary per file
| File | Description |
|---|---|
| cli/azd/pkg/extensions/validate_registry.go | Adds per-version provider validation and cross-extension provider ownership validation. |
| cli/azd/pkg/extensions/validate_registry_test.go | Adds unit tests covering provider validation and ownership rules. |
| cli/azd/pkg/extensions/registry.go | Defines provisioning provider type, valid provider-type enum, and required-capability mapping. |
| cli/azd/pkg/extensions/extension_manifest_files_test.go | Adds tests to validate first-party extension.yaml providers and enforce presence of manifest-vs-code verification tests. |
| cli/azd/pkg/azdext/provider_manifest.go | Adds helper to compare registered providers against extension.yaml. |
| cli/azd/pkg/azdext/provider_manifest_test.go | Adds unit tests for VerifyProvidersMatchManifest. |
| cli/azd/pkg/azdext/extension_host.go | Adds introspection helpers for tests/tools to read registered providers. |
| cli/azd/extensions/microsoft.azd.demo/internal/cmd/providers_manifest_test.go | Adds manifest-vs-code provider verification test for the demo extension. |
| cli/azd/extensions/microsoft.azd.demo/internal/cmd/listen.go | Refactors host wiring into a shared configureExtensionHost for reuse by tests. |
| cli/azd/extensions/microsoft.azd.demo/ci-test.ps1 | Replaces placeholder with a real unit-test runner script. |
| cli/azd/extensions/extension.schema.json | Extends schema enum to include provisioning-provider. |
| cli/azd/extensions/azure.ai.toolboxes/internal/cmd/providers_manifest_test.go | Adds manifest-vs-code provider verification test. |
| cli/azd/extensions/azure.ai.routines/internal/cmd/providers_manifest_test.go | Adds manifest-vs-code provider verification test. |
| cli/azd/extensions/azure.ai.connections/internal/cmd/providers_manifest_test.go | Adds manifest-vs-code provider verification test. |
| cli/azd/extensions/azure.ai.agents/internal/cmd/providers_manifest_test.go | Adds manifest-vs-code provider verification test. |
| .github/workflows/verify-ext-providers.yml | Adds reusable workflow to run provider manifest verification tests for extensions. |
| .github/workflows/lint-ext-azure-ai-toolboxes.yml | Adds provider verification job for the extension. |
| .github/workflows/lint-ext-azure-ai-routines.yml | Adds provider verification job for the extension. |
| .github/workflows/lint-ext-azure-ai-connections.yml | Adds provider verification job for the extension. |
| .github/workflows/lint-ext-azure-ai-agents.yml | Adds provider verification job for the extension. |
| .github/workflows/ext-registry-ci.yml | Expands triggers and adds manifest/schema validation to registry CI. |
Review details
- Files reviewed: 21/21 changed files
- Comments generated: 4
- Review effort level: Low
JeffreyCA
force-pushed
the
jeffreyca/ext-provider-linting
branch
from
f248c17 to
326745c
Compare
github-actions
Bot
added
ext-connections
JeffreyCA
force-pushed
the
jeffreyca/ext-provider-linting
branch
2 times, most recently
from
db3245f to
909e71c
Compare
Contributor
Author
Provider validation CI exerciseTemporary regressions were pushed and reverted one at a time using normal pushes. Times are approximate completion times in UTC.
✅ All temporary edits were reverted. The final tree matches the pre-exercise baseline; latest-head checks are settling. |
JeffreyCA
requested review from
RickWinter,
danieljurek,
glharper,
tg-msft,
therealjohn,
trangevi,
trrwilson and
vhvb1989
as code owners
jongio
reviewed
Jul 10, 2026
jongio
left a comment
jongio
left a comment
Member
There was a problem hiding this comment.
One issue from the schema and capability changes:
validation-provideris added to GoValidCapabilitiesand tomicrosoft.azd.demo/extension.yaml, but it's missing from the capabilities enum in bothextension.schema.jsonandregistry.schema.json(inline). A schema-aware editor or validator will reject that first-party manifest for a capability azd accepts.provisioning-providerwas added to Go, both schemas, and the demo manifest together;validation-providerskipped the two schemas. The new sync test only covers provider types, not capabilities, so CI doesn't catch it.
jongio
approved these changes
Jul 10, 2026
jongio
left a comment
jongio
left a comment
Member
There was a problem hiding this comment.
My earlier note is resolved: validation-provider is now present in both extension.schema.json and registry.schema.json, and TestCapabilitySchemaTypesInSyncWithGo guards both capability enums against Go drift in CI. The scope-reduction commit builds and vets clean here, no dangling references to the removed helpers, and all five provider extensions carry the canonical internal/cmd/providers_manifest_test.go.
Add registry and manifest checks for provider metadata, capabilities, and unique ownership. Verify runtime registrations match extension.yaml and run the checks in extension CI.
JeffreyCA
force-pushed
the
jeffreyca/ext-provider-linting
branch
from
7c44faf to
8c5926c
Compare
jongio
approved these changes
Jul 13, 2026
jongio
left a comment
jongio
left a comment
Member
There was a problem hiding this comment.
Re-reviewed after the rebase and scope reduction. Deferring the cross-extension verification and per-extension tests until the SDK release removes the temporary replace directives, which was my main concern on the earlier iterations. The remaining change is self-contained: capability schemas stay in sync via TestCapabilitySchemaTypesInSyncWithGo, and VerifyProvidersMatchManifest is guarded by its own tests plus the demo manifest check. Provider tests, azdext tests, and go vet all pass locally.
JeffreyCA
changed the title
Contributor
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #8992
Summary
This PR adds the SDK prerequisite for verifying that provider declarations in
extension.yamlmatch the providers configured by extension code. It also corrects and protects the extension capability schemas.Issue
Provider metadata can drift from calls such as
WithServiceTargetandWithProvisioningProviderwithout being detected. The incident corrected by #8901 demonstrated the impact:azure.ai.agentsdeclared service-target providers that it did not register, allowing incorrect metadata to reach the extension registry.SDK verification helper
The extension SDK now provides
VerifyProvidersMatchManifest, which configures an extension host without connecting to azd or invoking provider factories, then compares the accumulated registrations withextension.yaml.providerslist.Demo integration
The demo extension uses the helper in a canonical verification test against its real configuration callback. Its existing test script now runs the Go test suite so the verification is exercised by the demo extension's CI.
Capability schema consistency
The extension and registry schemas now include
validation-provider, matching the capability already accepted by azd and declared by the demo extension. A synchronization test compares both schema enums with Go's supported capability list so future drift fails testing.Behavior at a glance
Scope and follow-ups
This PR intentionally lands the SDK prerequisite rather than closing the complete first-party enforcement gap in #8992.
VerifyProvidersMatchManifest. After an azd SDK release contains the helper, a follow-up will update those dependencies, add their canonical verification tests, and wire the tests into extension CI.extension.yamlJSON Schema validation are deferred; this PR synchronizes capability enums only.Testing
Testing covers matching registrations, dead manifest claims, undeclared runtime registrations, duplicate declarations and registrations, exact casing, ignored registration categories, invalid callback input, the demo extension's real configuration callback, and synchronization of both capability schema enums.