feat: gen-schema entity port + resolver decoupling + delivery-edge unification - #563
Merged
Conversation
sini
force-pushed
the
feat/entity-gen-schema-port
branch
from
May 21, 2026 22:37
6574fac to
a370d30
Compare
sini
force-pushed
the
feat/entity-gen-schema-port
branch
7 times, most recently
from
June 5, 2026 19:36
b2bcfd4 to
1b56211
Compare
sini
force-pushed
the
feat/entity-gen-schema-port
branch
5 times, most recently
from
June 12, 2026 21:17
e3584fe to
b1619a4
Compare
sini
changed the title
sini
marked this pull request as ready for review
June 13, 2026 22:01
Replaces hand-rolled schemaEntryType with gen-schema mkSchemaOption. Sidecars: includes, excludes. Computed: isEntity (structural content only). Extracts resolvedCtxModule (id_hash, resolved, collisionPolicy) to _types.nix for entity type reuse. collisionPolicy flows through deferred module merge to entity instances (not a sidecar) preserving existing ctx.host.collisionPolicy resolution path.
den.hosts now accepts both forms:
- Legacy: den.hosts.x86_64-linux.igloo = { ... }
- Flat: den.hosts.igloo = { system = "x86_64-linux"; ... }
The outer option type uses a permissive submodule with deepMergeAttrs
freeformType (lib.recursiveUpdate-based merge that avoids the infinite
recursion lib.types.anything causes with cross-option references).
The apply function preprocesses flat entries into two-level form and
re-evaluates through the original attrsOf systemType, so all 6
consumers see the canonical { system.name = hostConfig } shape.
Same pattern as den.hosts: deepMergeAttrs + preprocessHosts + apply. Cross-entity host lookup and osConfig injection preserved.
Covers: id_hash, freeform, topology, meta introspection, isEntity computed, schema includes sidecar.
Update flake inputs and references to match the renamed repo at github:sini/gen-schema.
gen-schema flattened _meta into _-prefixed options and renamed sidecars → collections. nix-effects changed bindAttrs so true is a literal param, not an optionality marker — translate __args values to fx.bind.optionalArg before bind.fn.
The resolver binding rule (spec §3) claims transitive descendants fan out
by DAG nesting ({ host, user } at environment -> for-each-host-for-each-
user), but only DIRECT children worked. fanOut enumerated children off
scopeCtx.${scopeKind} (the scope's own record), so a descendant-of-a-
descendant (toy under pet) was looked up as host.toys (absent) -> inert;
and the fanned intermediate (pet) lived only in the aspect's
__scopeHandlers, invisible to the next level's enumeration.
Fix: thread fanned intermediates through a boundEntities accumulator;
enumerate each descendant off its PARENT-kind record (availRecords =
scopeCtx // boundEntities); and fan the shallowest reachable descendant
first (not alphabetical head) so an intermediate binds before its child.
Direct/cartesian behavior unchanged.
Found by the formal-rule coverage audit. New test:
relationship-fanout/test-transitive-descendant-chain.
CI 987/987.
The transitive fan-out fix inlined pure schema-DAG logic in the bind
handler: `schema.${argKind}.parent or scopeKind` (twice) and the
shallowest-reachable descendant filter. That knowledge belongs in
arg-class.nix, the pure schema-DAG module that already owns
isDescendantOf/childrenOf. Add parentKindOf + fanableDescendants there;
bind.nix calls them. Removes the duplicate parent expression and keeps
parent-chain access in one module. No behavior change.
CI 987/987.
pathSetByScope and the old flat pathSet were written from the same node under the same guard with the same key space — flat == union of the per-scope buckets, exactly. Guards already read pathSetByScope (denful#613), so flat was redundant STATE maintained only for the structural hasAspect + capture consumers. Make pathSetByScope the single membership record; add identity.flattenPathSetByScope (union of buckets) and point the five consumers (has-aspect x2, capture, two internal-api tests) at it. Drop the flat field from defaultState and the now-dead test fixtures. One membership mechanism instead of two — also the HOAG node shape. Behavior-preserving (provable key-equality; fx-constraints exact-count guards it). CI 987/987.
Three branch-local duplications the new shared modules should own:
- schema-kind parent walk: schema.nix ownerChain reimplemented the same
den.schema.${k}.parent traversal arg-class already owns. Add
arg-class.ancestorChain (the one schema-DAG ancestry walk); isDescendantOf
and schema.nix's owner lookup both derive from it.
- includes-chain prefix check: compile-conditional re-inlined the
isAncestor prefix match it already imports constraint.nix for. Export
constraint.isAncestorChain; both sites use it.
- deepMergeAttrs option-type was byte-identical in entities/host.nix and
home.nix; _types.nix (their shared-helper module) now owns it.
No behavior change. CI 987/987.
…ges (Task 16)
Expose `unifiedEdges` as a lazy thunk on the fxResolveFull result, beside
`edgeTrace`. It is the union of:
- the top-level mechanism edges (default-fold + provides + routes +
instantiate) the oracle (edge-trace.nix) builds from end-state, MINUS its
`spawnEdges` rewalk arm (which undercounts each spawn as one edge);
- the SURFACED spawn edges, collected from the drain-fold spawn (mkDrained);
- the per-host + B′ instantiate projection edges (mkInstantiateEdges).
mkDrained now returns { classImports; spawnEdges; } so the host-own invocation's
spawn `.edges` feed the union (the B′ invocation discards them — B′ delivery is
covered by the per-host mkInstantiateEdges). The per-host projection is factored
into `perHostProjection`, shared by mkInstantiateArgs (module assembly,
unchanged) and the edge collector. edge-trace.nix gains `extractTopLevelEdges`
(the per-component seam) so oracle and unified set share the exact constructor
calls.
Additive, non-breaking: unifiedEdges is not yet consumed by production
materialization (exposed for a later task). 996/996 CI green.
…ortEdges (Task 17) Build materializeUnified, ONE ordered-dispatch fold that interleaves provides + routes in topoSortEdges order, reusing the EXISTING per-spec materializers (provides.applyOneProvide | route.applySimpleRouteEdge | applyComplexRouteEdge). Design B (order-only): independents keep the provides-before-routes construction order via a STABLE topoSortEdges, so the unified fold is byte-equivalent to phase2∘phase3; synthesize edges land after their producers exactly as applyRoutes' internal toposort already orders them. Additive + non-breaking — no production site switches; the engine is proven equivalent, not yet consumed. - toposort.nix: lock-in comment on the stable ascending-index Kahn emit order. - provides.nix: factor applyOneProvide out of the applyProvidesEdges fold body (identical per-spec body, byte-stable). - route.nix: surface the per-spec materializers + ordering helpers. - materialize-unified.nix: the engine. - resolve.nix: lazy materializeEquiv surface (dispatch order + both fold results). - fx-materialize-unified.nix: equivalence proof over plain / provides / routes / provides+routes / isolated-guest canaries + a doFinalMerge case. Equivalence holds byte-exact on every canary (dispatch order identical, structural fingerprint identical). - fx-toposort-edges.nix: independent-edges-stable unit test.
…sequencing (Task 17)
…fied
The per-host (resolve.nix perHostProjection) and spawn (edges/materialize.nix
assembleSpawnSubtree) sites still ran the legacy phase2 (applyProvidesEdges) -
phase3 (applyRoutes) folds purely to feed their edge collectors' perScope
content source. Delivery already flowed through materializeUnified (Task 17).
Expose the post-fold accumulator from materializeUnified via a new exposeAcc
flag (one fold, both the merged output and {classImports;perScope}), then
repoint both edge collectors to that accumulator and delete the redundant
phase2/phase3 folds. The applyProvides/applyRoutes injection params drop out of
mkSpawnNode/assembleSpawnSubtree accordingly.
applyProvidesEdges + the applyRoutes wrapper + route.nix:applyRoutes are kept:
they remain the materializeEquiv byte-equivalence oracle the fx-materialize-
unified suite compares against.
Byte-stable: full CI 1011/1011, no diffs.
Task 18.2 + 18.3 (one CI-green unit).
18.2 — Make edgeTrace the PRODUCTION delivery-edge object. Its fold-ordered
provides+routes portion is now CAPTURED from the production materializeUnified
folds (materializeUnified{exposeEdges=true}.edges) at all three sites — top-level
(resolve.nix), spawn (.edges in edges/materialize.nix assembleSpawnSubtree), and
per-host (edges/instantiate-edges.nix mkInstantiateEdges) — instead of being
re-derived via providesEdges/routeEdges. The default-fold (merge) + instantiate
(flake-output) edges stay constructor-built (deterministic structural edges, no
drift surface). unifiedEdges is now an alias for edgeTrace (one representation).
The legacy end-state re-derivation (extractEdgeTrace, with its spawn rewalk arm)
is kept as a distinct legacyEdgeTrace field for the differential. The
fx-unified-edges + fx-edge-unification-gate suites repoint their oracle to
legacyEdgeTrace.
18.3 — New fx-oracle-production-differential suite diffs edgeTrace (production)
against legacyEdgeTrace on a spawn + an instantiate topology: production is a
superset of legacy-minus-rewalk, and the production-only delta carries the spawn's
surfaced fold edge / the per-host fold. Re-baseline the delivery-edges fixtures:
the suppressed route twins are gone (production captures kept routes only), the
host-level spawn rewalk edge is gone, and instantiate topologies grow by their
per-host fold edges.
…ins from legacy arm The subset relation is production ⊇ legacy\rewalk\suppressed: production folds orderedKeptRoutes only, so dedup-suppressed route twins are never materialized (their absence is faithful). Today every CI suppressed twin key-aliases its kept sibling, but stripping suppressed from the legacy arm keeps the gate sound for a future distinct-key suppression (rule-2 redundant-root / adapterKey).
edge-trace.nix is now the LEGACY end-state re-derivation (the legacyEdgeTrace differential arm), not the live trace; the live edgeTrace is the captured production object. default.nix: the spawn surfaces real edges via the shared constructors now. materialize-unified.nix: it IS production at every site (Task 17) + exposeEdges feeds the captured trace. materialize.nix: the spawn .edges provides+routes are captured, consumed by edgeTrace. sourceVia docs kept intact.
fmway
previously requested changes
Jun 15, 2026
…alog) check-constraint read the fleet-wide flatConstraintRegistry, so a sibling entity's exclude leaked into a node's check — eval-order dependent: a host walked earlier (iceberg < igloo) suppressed a sibling host that INCLUDED the same aspect (igloo.networking.hostName resolved to the default instead of the included value). This is the denful#613 sibling-leak, but for the exclude APPLICATION path (check-constraint) rather than the conditional-guard hasAspect pathSet that denful#613 fixed. Fix: look excludes/substitutes up in the ENTITY-scoped registry (currentScope + ancestors via scopedConstraintRegistry), mirroring the denful#613 guard fix. ownerChain is preserved so within-scope include nesting still isolates. The scope walk is cycle-guarded (visited set) — scopeParent can carry a cycle in spawn/forward merged sub-pipelines, and check-constraint runs for every node. Regression test mirrors github.com/tschan/den-hasaspect-bug (the bogus+working pair); both directions now pass. Full CI 1021/1021.
fmway
self-requested a review
June 15, 2026 02:27
…elete flat The denful#613-analog follow-up + simplification. All three exclusion readers (check-constraint, dispatch-policies, policy/schema late dispatch) now go through ONE scope+ancestor lookup (scopedConstraintsFor / scopedConstraintsForScope); the fleet-wide flatConstraintRegistry — which leaked excludes across siblings, eval-order dependent — is deleted (write + init gone). Key: the late-policy dispatch scopes to the SIBLING it emits for (sib.scopeId), not the parent scope, so schema-tier excludes (den.schema.KIND.excludes, registered at the resolved kind scope) still reach their descendants — no position-independent broadcast special-case needed. Dedup: extracted foldScopeAncestors (cycle-guarded scope+ancestor fold); the denful#613 guard's collectScopeConstraints AND the guard pathSet (scopedPathSet) now reuse it instead of three near-identical walks — scopedPathSet also gains the full visited-set cycle guard. Both exclusion flavors are now sibling-isolated: aspect-content (aspects.X.excludes) and policy-name. Regression test issue-613-policy-exclude-sibling added. Full CI 1023/1023.
fmway
approved these changes
Jun 15, 2026
xfo-0
added a commit
to xfo-0/nixfos
that referenced
this pull request
Jun 18, 2026
Behavior-preserving on AO05/nl0x/grpht (toplevel byte-identical bar the self-source hash; diff-closures empty, known_hosts/nix.conf unchanged); also clears the installer niri-route->all-hm-users blocker via #563 resolver decoupling. #563 is still an OPEN draft. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this delivers
This branch began as the gen-schema entity port and grew into a five-part stabilization of den's resolver and delivery pipeline. Each part builds on the last and is covered by CI. 1023/1023 tests (up from 825 at branch start).
1. gen-schema entity port (the original scope)
den.schemanow uses gen-schema'smkSchemaOption/mkSchemaEntryType(sidecar extraction, computed fields,__functorwrapping);resolvedCtxModuleextracted to shared_types.nix.id_hashvia gen-schemamkInstanceType(context-free kind+name identity).den.hosts.igloo = { … },den.homes."tux@igloo" = { … }), preprocessed to the canonical shape — all consumers see the unchanged{ system.name = entity }.den.reservedKeys(user-extensible structural keys);settingsreserved as structural.2. Isolation + delivered-child host (microvm guests)
isolatedmarker, recorded per-scope at scope creation.*.configre-runs eval with the base module list);reinstantiatekept as a core route flag.hasAspectre-keyed by entityid_hashrather than scope-string, fixing a false-negative (core.impermanence→ identity paths) for ancestor-nested hosts.guest-osclass — added and removed within this branch, never in a release — left no surface; isolated guests carry honestnixosidentity.)3. Resolver decoupling — binding half
Purifies who binds what, where emission lands, removing the host/user-specific machinery the resolver core carried (zero
host/user/homeManagerliterals remain in the fx resolver core).{ user, … }homeManager aspect no longer leaks to every user.den.lib.perHost/perUser/perHomeare kept as deprecated shims (they shipped in main) — now thin aliases over the new rule.4. Delivery-edge unification — delivery half
Collapses nine ad-hoc delivery mechanisms into one
(S, T, P, M)edge algebra: a single mechanism-free mode switch, one constructor per mechanism, an explicit per-root context projection, and a record-level toposort for ordering — production now materializes delivery as one toposorted edge fold per root, replacing the four hand-threaded phase-fold sites.deliver { from; to; at ? []; mode ? "merge"; }primitive;routeandprovidesretained as permanent thin shims over it (signatures unchanged).findHostScopeId's name-infix heuristic dissolved into an exact scope-creation link.fxResolveFull,fxResolveImports, the per-host re-walk, the spawn re-entry) all fold the singlematerializeUnifiedengine overtopoSortEdges(unifiedEdges); the per-mechanism phase orchestration is deleted. The toposort is stable (construction-order tiebreak), so byte-equivalence to the prior phase order holds.edgeTracerenders the captured edges the production fold dispatched (oracle ≡ production by construction, no drift surface), withlegacyEdgeTraceretained as the differential's "before". A production-vs-legacy differential proves the prior spawnrewalkundercount + dedup-suppressed twins are eliminated.assertEdgeParity+ a parity corpus, so den-hoag (v2) can be validated against the v1 production edge set as a pure diff. Schema/runbook live in the den-architecture papers.attribute missingthrow; witness test added).5. Constraint-registry unification + sibling-exclusion fixes (#613)
The
#613scope-isolation fix (conditional-guardhasAspect) is extended to the exclude/substitute application path, and the constraint registry is unified.aspects.X.excludesand policy-name excludes no longer leak across sibling entities (an eval-order-dependent bug: a host walked earlier suppressed a sibling that included the same aspect/policy). Both flavors are now entity-scoped (scope + ancestors).flatConstraintRegistryis deleted: all three readers (check-constraint, the policy dispatch, and the late-policy dispatch) go through one scope+ancestor lookup. The late-policy dispatch scopes to the sibling it emits for, so schema-tier excludes (den.schema.KIND.excludes) still reach their descendants.foldScopeAncestorsunderlies the constraint walk and the guard pathSet walk (the latter also gains the full cycle guard).Breaking change
One breaking change versus the merge base — a semantic change, no API removals:
provides/policy only. This applies uniformly to plain parametric functions and to the deprecatedperHost/perUser/perHomeshims (whose old self-suppression was the Bug: homeManager content on host-included aspects is silently dropped unless the aspect is user-parametric #609 bug). Documented inexplanation/parametric.mdx(the rule + the silent-inert footgun) andreference/lib-deprecated.mdx.No public API is removed:
perHost/perUser/perHome,route,provides, the legacy two-level entity form — all retained (the first three as warning shims). Release notes will be drafted asheads-upGH discussions at release time permaintainers.mdx.Dependency note
den's root flake is a pure library (
outputs = _: import ./nix). gen-schema is resolved via the CI template lock, pinned togithub:sini/gen-schema— the same personal-repo + CI-lock-fallback pattern den already uses for nix-effects. Staying as-is for now.Verification
nix develop -c just ci).entity-isolationsuite (realevalModulesoutput) + full CI with no diffs; the oracle-vs-production differential gates the edge-set faithfulness. A nix-config host system-closure-hash re-verification across the swap is the only out-of-repo step, taken at the den input bump.deliverAPI suite, the edge-toposort gates (fx-materialize-unified,fx-unified-edges,fx-edge-unification-gate,fx-oracle-production-differential,fx-edge-parity), and the BUG: Unexpected result forhost.hasAspect#613 sibling-exclusion regressions all green.Status
Ready for review. Deliverability audit complete (dependency resolution, breaking-change docs, leftover-scaffolding sweep, templates): gen-schema stays at
sini(matches nix-effects); flake-parts-modules template pin fixed;perHostshim restored; docs corrected. Specs, plans, the per-task deviation ledger, and the frozen parity contract live in the den-architecture papers; a net-behavior-change summary is published as a gist.