Skip to content

Commit 1609a81

Browse files
lesnik512claude
andauthored
Unify provider-graph traversal behind a DependencyGraph module (#308)
* docs(planning): spec + decision for DependencyGraph traversal unification Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test: characterize validate() and runtime-guard behavior before refactor Locks today's behavior of Container.validate() (cycle detection shape, collecting multiple error kinds in one pass) and the runtime resolve cycle guard, against the public API only, so a later DependencyGraph refactor can be checked for parity. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * feat: track validated_version on ProvidersRegistry Adds a `validated_version` marker (int | None, default None) reset to None on every mutation (register, add_providers, _remove_providers). Later work will let validate() set it on success and short-circuit re-validation when the registry hasn't changed since. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * feat: add redirect_target node hook for transparent redirects Lets a later DependencyGraph module follow alias chains without knowing about the concrete Alias type. Alias overrides it to return its source, or None when the source is unregistered (dangling). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * feat: DependencyGraph.walk emits an iterative event stream Add modern_di/dependency_graph.py with the NodeEntered/Edge/Cycle/ DependenciesError event NamedTuples, the Event union, and DependencyGraph.walk: a single explicit-stack (non-recursive) pre-order DFS of the static provider graph. Event order mirrors Container.validate's recursive _visit exactly (Edge before a possible Cycle; one NodeEntered per node; visiting/visited shared across roots), so a later task can reproduce validate() output byte-for-byte and reuse the walk inside a RecursionError handler near the stack limit. Imports only AbstractProvider and exceptions at runtime; Container stays behind TYPE_CHECKING to avoid an import cycle. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * feat: add DependencyGraph.find_cycle_from and terminal_scope find_cycle_from reuses the iterative walk to locate the first cycle reachable from a provider, staying safe near the stack limit for a later RecursionError handler. terminal_scope follows redirect_target hops to the terminal provider's scope, guarding against alias cycles via a seen-id set. Both will replace the hand-rolled chain walk in Alias.effective_scope in a later task. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * refactor: validate() folds over DependencyGraph; drop effective_scope Rewrite Container.validate() as a fold over the shared DependencyGraph.walk event stream instead of a bespoke recursive DFS, and short-circuit when the providers registry has already been validated at its current version. Remove AbstractProvider.effective_scope and Alias.effective_scope; the scope-ordering check now uses DependencyGraph.terminal_scope on both sides of each edge. Event emission order (NodeEntered -> DependenciesError -> Edge -> Cycle in registry order) reproduces the old _visit output byte-for-byte, so the characterization/parity suites stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * refactor: route runtime cycle guard through DependencyGraph; delete _find_reachable_cycle Rewire resolve_provider's RecursionError handler to reuse the shared DependencyGraph.find_cycle_from walker instead of a second explicit-stack DFS, and short-circuit (re-raise untouched) when the registry is already validated. Add dependency_graph.build_cycle_error so validate() and the guard construct CircularDependencyError from one place, removing the last verbatim duplication. Delete the now-redundant _find_reachable_cycle and _convert_recursion_error. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test: exclude never-run _explode guards from coverage Both `_explode` monkeypatch helpers intentionally never execute their body (they exist to fail the test loudly if the patched method IS called). Mark them `# pragma: no cover` per the convention already used in test_runtime_cycle_guard.py, so line coverage doesn't count dead-by-design guard bodies against the 100% gate. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * refactor: restore RecursionError call boundary for coverage tracer CPython suspends the coverage tracer for a few frames while unwinding a RecursionError, so a re-raise written inline in the except block is never recorded as covered. Route the handling back through a separate module-level function so the tracer gets a fresh call boundary to re-arm on, restoring the 100% coverage gate without changing any resolve behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * docs: promote DependencyGraph model into architecture and docs Replace the recursive-DFS/effective_scope narration with the shipped DependencyGraph model: validate() as a fold over one iterative walk, the runtime guard sharing that walk via find_cycle_from, terminal_scope following the redirect_target node hook, and the registry-level validated_version short-circuit. effective_scope no longer exists in code and is now absent from all architecture and docs prose. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> * test: rename stale effective_scope test; note match exhaustiveness Rename test_effective_scope_handles_mutual_alias_cycle to test_terminal_scope_handles_mutual_alias_cycle so it's grep-discoverable under the current terminal_scope API, and add a one-line comment noting the Event match in Container.validate() is exhaustive by construction. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
1 parent b3dfab0 commit 1609a81

16 files changed

Lines changed: 819 additions & 182 deletions

architecture/containers.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,10 +84,17 @@ dispatch) are the blessed integration seam — see
8484
`add_providers` is **root-only**: called on a child, it raises
8585
`ChildContainerRegistrationError` (`modern_di/exceptions.py`), since the
8686
registry it mutates is shared tree-wide. `Container` tracks a private
87-
`_validated` flag, set once `validate()` succeeds (construction or a manual
88-
call); `add_providers` on an already-validated container re-runs `validate()`
87+
`_validated` flag, set once `validate()` succeeds on **this** container
88+
(construction or a manual call); it is the per-container gate for
89+
`add_providers`, which on an already-validated container re-runs `validate()`
8990
after registering and, if that fails, removes the just-added batch again —
9091
the container ends up either fully registered and valid, or unchanged.
92+
Whether the graph is *currently* validation-clean is tracked separately and
93+
registry-level, by `ProvidersRegistry.validated_version` (see
94+
[validation.md](validation.md#what-validate-checks)). Because that registry is
95+
shared tree-wide, validating any container in the tree marks the whole graph
96+
clean, so a child's `resolve` benefits from the root's validation — its runtime
97+
cycle guard short-circuits — without the child ever calling `validate()` itself.
9198
`resolve_dependency` carries no such restriction; it is a resolve verb,
9299
callable on any container regardless of validation state.
93100

architecture/providers.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -153,8 +153,11 @@ providers.Alias(ConcreteDatabase, bound_type=DatabaseProtocol)
153153
accepts an optional `bound_type` override. See [docs/providers/alias.md](../docs/providers/alias.md) for the
154154
user-facing rationale and caching implications.
155155

156-
`Alias.effective_scope` follows alias chains transitively to the terminal non-alias provider and returns that
157-
provider's scope. This is what `Container.validate()` and scope-error reporting use — the alias's own `scope`
156+
`Alias` overrides the `redirect_target(container)` node hook to return its source provider (`None` when the
157+
source type is unregistered), marking the alias as a transparent redirect. `DependencyGraph.terminal_scope`
158+
follows that hook down an alias chain to the terminal non-alias provider and returns that provider's scope —
159+
which is what `Container.validate()` and scope-error reporting compare against (see
160+
[validation.md](validation.md#terminal-scope-and-alias-transparency)). The alias's own `scope`
158161
attribute is only a stored default.
159162

160163
### Deprecated `scope=` parameter

architecture/validation.md

Lines changed: 85 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -32,90 +32,120 @@ containers never emit this warning regardless of `validate` — see
3232

3333
## What validate() checks
3434

35-
`validate()` performs a depth-first search (DFS) over every provider in `providers_registry` (the walk itself is
36-
`Container.validate()` in `modern_di/container.py`). It collects **all errors** across the entire walk before
37-
raising, so a single call surfaces all wiring bugs at once rather than stopping at the first one. `validate()`
38-
raises `exceptions.ValidationFailedError` if any are found; its `.errors` attribute lists every collected
39-
exception, and its `__str__` groups them by class name so a report mixing several error kinds reads as one
40-
section per kind — see `ValidationFailedError.__str__` in `modern_di/exceptions.py`. Validation runs entirely
41-
against `providers_registry`, so a root APP-scope container validates deeper-scoped providers without building
35+
`validate()` is a **fold** over one depth-first walk of the provider graph:
36+
`DependencyGraph().walk(providers_registry, self)` (in `modern_di/dependency_graph.py`) runs a single
37+
iterative, explicit-stack traversal and yields an event stream — `NodeEntered`, `Edge`, `Cycle`,
38+
`DependenciesError` — while `validate()` applies one policy per event kind (`NodeEntered`
39+
`iter_validation_issues`; `Edge` → the scope-ordering check; `Cycle``CircularDependencyError`;
40+
`DependenciesError` → collect the raised exception). The walk emits *structure*; `validate()` supplies
41+
*policy*. That same walker also backs the runtime cycle guard (see the blockquote below), so the two
42+
share one traversal — a stance reversed, on the extraction axis only, from the "deliberate duplication"
43+
defense this file once carried; the reasoning is recorded in
44+
[decisions/2026-07-12-unify-graph-traversal.md](../planning/decisions/2026-07-12-unify-graph-traversal.md).
45+
46+
It collects **all errors** across the entire walk before raising, so a single call surfaces all wiring
47+
bugs at once rather than stopping at the first one. `validate()` raises `exceptions.ValidationFailedError`
48+
if any are found; its `.errors` attribute lists every collected exception, and its `__str__` groups them
49+
by class name so a report mixing several error kinds reads as one section per kind — see
50+
`ValidationFailedError.__str__` in `modern_di/exceptions.py`. Validation runs entirely against
51+
`providers_registry`, so a root APP-scope container validates deeper-scoped providers without building
4252
child containers.
4353

54+
**Validated-version short-circuit.** `ProvidersRegistry` carries a `validated_version: int | None`. On a
55+
successful walk `validate()` stamps `validated_version = version`; a later `validate()` whose
56+
`validated_version == version` returns immediately without re-walking, so a repeat `validate()` is free.
57+
Every registry mutation (`register` / `add_providers` / `_remove_providers`) bumps `version` and resets
58+
`validated_version = None`, so any change to the graph re-arms both `validate()` and the runtime guard.
59+
The stamp lives on the registry, which is shared tree-wide, so validating any one container marks the
60+
graph clean for every container in the tree.
61+
4462
### Circular dependencies
4563

46-
During the DFS, a provider encountered a second time while it is still on the active path (i.e., it appears in
47-
`visiting`) means a cycle exists. `validate()` records a `CircularDependencyError` with a `.cycle_path` list of
48-
type names showing the loop (e.g., `["A", "B", "A"]`). The recursive walk does **not** continue into the cycle,
49-
but the rest of the graph continues to be checked. `CircularDependencyError.__str__` renders `.cycle_path` as a
50-
multi-line arrow chain, not an inline `A -> B -> A` string — see `CircularDependencyError` in `exceptions.py`.
64+
When the walk follows an edge to a provider still on the active path (tracked in the walk's internal
65+
`visiting` set), it emits a `Cycle` event whose `providers` list closes the loop by repeating the first
66+
node last (e.g., `[A, B, A]`). `validate()` maps that event to a `CircularDependencyError` (built via
67+
`dependency_graph.build_cycle_error`) whose `.cycle_path` is the list of type names showing the loop
68+
(e.g., `["A", "B", "A"]`). The walk does **not** descend into the cycle, but the rest of the graph
69+
continues to be checked. `CircularDependencyError.__str__` renders `.cycle_path` as a multi-line arrow
70+
chain, not an inline `A -> B -> A` string — see `CircularDependencyError` in `exceptions.py`.
5171

5272
Each node in that chain may also carry an optional definition site — the creator's declaration
5373
point — rendered as a trailing `module:line` anchor alongside the provider name, using the same
5474
lazy, memoized, best-effort capture described for breadcrumb steps in
5575
[resolution.md](resolution.md#breadcrumb-definition-sites). The public `.cycle_path` stays the bare
56-
list of type names; the parallel locations live on a separate `.cycle_locations` attribute, kept in
57-
sync at both construction sites (the DFS above and the runtime `RecursionError` conversion below).
76+
list of type names; the parallel locations live on a separate `.cycle_locations` attribute. Both
77+
`validate()` and the runtime guard build the error through the one `dependency_graph.build_cycle_error`
78+
helper, so the two attributes stay in sync by construction rather than by parallel edit.
5879

5980
> **Runtime resolution has a cycle guard too — but `validate()` remains the way to see all errors up front.**
60-
> `Container.resolve_provider` wraps the final `provider.resolve(self)` in `try/except RecursionError`. When an
61-
> unvalidated circular graph's first resolve overflows the stack, the handler iteratively re-walks the static
62-
> graph from the failing provider (an explicit-stack DFS — the walk must stay flat, since it runs close to the
63-
> recursion limit) and, if a static cycle is reachable, raises `CircularDependencyError` with the cycle path,
64-
> `from` the original `RecursionError`. `resolve_provider` is re-entrant (`Factory`/`Alias` call it per dependency
65-
> edge), so it is the innermost frame that converts; outer frames see a `ResolutionError` and the existing
66-
> breadcrumb machinery (see [resolution.md](resolution.md)) prepends steps as it propagates back up, so the
67-
> converted error arrives with the dependency chain attached. A `RecursionError` from a creator that recurses on
68-
> its own (no static cycle in the graph) is re-raised untouched, not misreported as a circular dependency. This is
69-
> a deliberate duplication of cycle detection, not a refactor of the one DFS above into two callers: `validate()`
70-
> collects *all* errors of *all* kinds in one walk, while the runtime guard only answers "is a cycle reachable
71-
> from here" on an exhausted stack — unifying them would couple the resolve hot path to the all-errors walker for
72-
> no user benefit. Run with `validate=True` (or call `container.validate()`) in development to surface *every*
73-
> cycle (and other wiring bugs) before the first resolve, rather than only the one a particular resolve happens
74-
> to hit.
81+
> `Container.resolve_provider` wraps the final `provider.resolve(self)` in `try/except RecursionError`. The
82+
> handler first short-circuits: if the registry is already validated (`validated_version == version`), the static
83+
> graph is known acyclic, so the overflow is genuine self-recursion and the `RecursionError` re-raises untouched
84+
> without any walk. Otherwise, when an unvalidated circular graph's first resolve overflows the stack, the handler
85+
> re-walks the static graph from the failing provider via `DependencyGraph().find_cycle_from` — the same
86+
> iterative, explicit-stack `walk` that `validate()` uses (it must stay flat, since it runs close to the recursion
87+
> limit) — and, if a static cycle is reachable, raises `CircularDependencyError` (built by
88+
> `dependency_graph.build_cycle_error`) with the cycle path, `from` the original `RecursionError`.
89+
> `resolve_provider` is re-entrant (`Factory`/`Alias` call it per dependency edge), so it is the innermost frame
90+
> that converts; outer frames see a `ResolutionError` and the existing breadcrumb machinery (see
91+
> [resolution.md](resolution.md)) prepends steps as it propagates back up, so the converted error arrives with the
92+
> dependency chain attached. A `RecursionError` from a creator that recurses on its own (no static cycle in the
93+
> graph) is re-raised untouched, not misreported as a circular dependency. Both the guard and `validate()` consume
94+
> the one `DependencyGraph` walker: `validate()` collects *all* errors of *all* kinds up front, while the guard
95+
> only answers "is a cycle reachable from here" on an already-exhausted stack. Run with `validate=True` (or call
96+
> `container.validate()`) in development to surface *every* cycle (and other wiring bugs) before the first resolve,
97+
> rather than only the one a particular resolve happens to hit.
7598
7699
### Inverted scope dependencies
77100

78-
For every dependency edge `provider → dep`, `validate()` compares their **effective scopes** (see below). If
79-
`dep`'s effective scope is strictly deeper than `provider`'s effective scope, the dependency is inverted: a
80-
shallower-lived provider cannot hold a reference to a deeper-lived one. The error is recorded as
81-
`InvalidScopeDependencyError` (see `exceptions.py` for the exact message), which names the provider, the
82-
parameter, the dependent provider, and the offending scopes. The walk continues into the dependency so further
83-
issues in that subtree are also surfaced.
101+
For every dependency edge `provider → dep` (the walk's `Edge` event), `validate()` compares their
102+
**terminal scopes** (see below). If `dep`'s terminal scope is strictly deeper than `provider`'s terminal
103+
scope, the dependency is inverted: a shallower-lived provider cannot hold a reference to a deeper-lived
104+
one. The error is recorded as `InvalidScopeDependencyError` (see `exceptions.py` for the exact message),
105+
which names the provider, the parameter, the dependent provider, and the offending scopes. The walk
106+
continues into the dependency so further issues in that subtree are also surfaced.
84107

85108
### Missing required dependencies
86109

87-
Before recursing into a provider's dependencies, `validate()` calls `provider.iter_validation_issues(container)`
88-
and appends any returned exceptions to the error list. `Factory` implements this hook to yield
89-
`ArgumentResolutionError` for each constructor parameter that has no matching provider in `providers_registry`,
90-
no default value, and no static `kwargs` entry.
110+
When the walk enters a provider (the `NodeEntered` event, emitted before that provider's dependencies are
111+
read), `validate()` calls `provider.iter_validation_issues(container)` and appends any returned exceptions
112+
to the error list. `Factory` implements this hook to yield `ArgumentResolutionError` for each constructor
113+
parameter that has no matching provider in `providers_registry`, no default value, and no static `kwargs`
114+
entry.
91115

92-
## Effective scope and alias transparency
116+
## Terminal scope and alias transparency
93117

94-
`validate()`'s scope-ordering check uses `provider.effective_scope(container)` on both sides of every dependency
95-
edge — not `provider.scope` directly.
118+
`validate()`'s scope-ordering check uses `DependencyGraph.terminal_scope(provider, container)` on both
119+
sides of every dependency edge — not `provider.scope` directly.
96120

97-
For most providers, `effective_scope` simply returns `self.scope`. `Alias` overrides it to follow the alias chain
98-
to its terminal non-alias target and return **that provider's scope**. This makes validation transitive through
99-
aliases. Consider:
121+
`terminal_scope` follows the `AbstractProvider.redirect_target(container)` node hook from provider to
122+
provider until it reaches one whose resolution terminates there (`redirect_target` returns `None`), then
123+
returns that terminal provider's `.scope`. The hook defaults to `None` on `AbstractProvider`, so for most
124+
providers `terminal_scope` returns `self.scope` in a single step. `Alias` overrides `redirect_target` to
125+
return its source provider (and `None` when the source type is unregistered), so `terminal_scope` follows
126+
an alias chain to its terminal non-alias target and reports **that provider's scope**. This makes
127+
validation transitive through aliases. Consider:
100128

101129
```
102130
Factory(scope=APP, creator=Caller) # depends on IFace
103131
Alias(source_type=Impl, bound_type=IFace) # no scope parameter
104132
Factory(scope=REQUEST, creator=Impl)
105133
```
106134

107-
The alias's effective scope is `REQUEST` (the scope of `Impl`). When `validate()` checks the `Caller → IFace`
108-
edge, it compares `APP` against `REQUEST` and raises `InvalidScopeDependencyError`. Without `effective_scope`,
109-
the alias's internal `scope` attribute (defaulting to `APP`) would mask the true depth of the dependency.
135+
The alias's terminal scope is `REQUEST` (the scope of `Impl`). When `validate()` checks the `Caller → IFace`
136+
edge, it compares `APP` against `REQUEST` and raises `InvalidScopeDependencyError`. Without `terminal_scope`,
137+
the alias's own `scope` attribute (defaulting to `APP`) would mask the true depth of the dependency.
110138

111-
Two edge cases in `Alias.effective_scope` are handled safely:
139+
Two edge cases in `terminal_scope` are handled safely:
112140

113-
- **Alias cycle**: if the same alias is encountered twice during the chain walk, the method falls back to
114-
`self.scope` and returns immediately. The cycle itself is separately detected and reported by the DFS cycle
115-
check.
116-
- **Dangling source**: if the alias's source type is not registered, the method falls back to `self.scope`. The
117-
dangling source is separately detected and reported by `iter_validation_issues` or by the dependency lookup
118-
raising `AliasSourceNotRegisteredError` during the walk.
141+
- **Redirect cycle**: if the chain revisits a provider (tracked in `terminal_scope`'s `seen` set), it
142+
breaks out and falls back to the starting provider's own `.scope` instead of looping forever. The cycle
143+
itself is separately detected and reported as a `Cycle` event by the walk, which traverses the same
144+
`source` edge.
145+
- **Dangling source**: if an alias's source type is not registered, `redirect_target` returns `None`, so
146+
`terminal_scope` stops at the alias and falls back to its `.scope`. The dangling source is separately
147+
reported by the alias's dependency lookup raising `AliasSourceNotRegisteredError` during the walk
148+
(a `ResolutionError`, surfaced as a `DependenciesError` event).
119149

120150
## Exception types
121151

docs/providers/advanced-api.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,11 @@ To implement a custom provider, subclass `AbstractProvider` and implement:
2121
traversal.
2222
- **`iter_validation_issues(container)`** *(optional)* — yields `Exception` instances for
2323
validation-time problems found in this provider; default yields nothing.
24-
- **`effective_scope(container)`** *(optional override)* — override this method to report the
25-
scope of whatever the provider ultimately resolves to. A transparent or redirect provider (like
26-
`Alias`) should override it to follow the source chain so that `Container.validate()` checks
27-
callers against the real target scope rather than any nominal scope on the wrapper itself.
24+
- **`redirect_target(container)`** *(optional override)* — return the provider this one
25+
transparently forwards to, or `None` (the default) when resolution terminates here. A transparent
26+
or redirect provider (like `Alias`) overrides it to point at its source, so that `Container.validate()`
27+
follows the redirect to the real target and checks callers against that target's scope rather than any
28+
nominal scope on the wrapper itself.
2829

2930
### `CacheSettings.is_async_finalizer`
3031

0 commit comments

Comments
 (0)