Skip to content

perf(graph): emit incremental graph snapshot shards from the resident Program #1055

Description

@samchon

Outcome

Make ttscgraph serve export content-addressed graph shard deltas from its
resident compiler state instead of rebuilding and serializing one complete
project dump after every changed request.

This is the TypeScript producer half of
samchon/compiler-graph#63
and the mission in
samchon/compiler-graph#162.
The consumer-side Graph Snapshot Protocol and atomic store already exist in
@samchon/graph; this issue owns the work that only ttsc can make
incremental.

Problem

ttscgraph serve keeps one driver.Session and correctly reports initial,
reload, unchanged, incremental and rebuild modes. However, every changed
response still follows this path:

  1. graphSession.Snapshot calls changedSnapshot;
  2. changedSnapshot calls buildDump;
  3. buildDump calls graph.Build(program) over all user source files;
  4. it calls graph.SourceTexts(program) over all compiler sources;
  5. graph.NewDump projects and sorts the complete graph; and
  6. the serve response serializes the complete dump as one JSON value.

The downstream @samchon/graph adapter can partition that dump after receipt,
but it cannot recover the producer work, allocation or serialization already
paid. An ordinary private-body edit therefore reuses the compiler Program while
still performing graph work proportional to the whole project.

Required invariant

For one immutable compiler generation, publish graph shards whose identities
bind the exact source/checker text, build universe, configuration and dependency
resolution that produced them. A no-op reuses the committed manifest without
rebuilding graph facts. An ordinary edit recomputes and transmits only the
compiler-invalidated semantic closure. A configuration, root-set, dependency
or public-API movement may invalidate a wider closure, but the producer must
name the changed and deleted shards explicitly.

The consumer must never observe a partial generation. Cancellation, malformed
output, digest disagreement, input movement, compiler failure or a stale base
leaves the previous committed generation usable.

Approach

  • Evolve the versioned ttscgraph serve protocol in coordination with
    @samchon/graph. Preserve an explicit compatibility failure for mismatched
    client and producer versions.
  • Keep graph extraction beside the resident driver.Session; do not start a
    second compiler or reparse source that the Program already owns.
  • Refactor graph construction so file/target-owned facts can be built and
    cached as immutable shards. Cross-file edges belong to a deterministic owner
    and retain exact endpoint identity.
  • Derive invalidation from compiler source replacement and the resolved
    import/type/configuration topology. A body-only edit must not invalidate
    unrelated shards; a signature or resolution movement must invalidate every
    dependent shard whose facts can change.
  • Emit a full initial shard manifest, then base-generation deltas with changed
    shard payloads and explicit deletions. Include source and disk digests,
    diagnostics, producer/compiler identity and generation coordinates.
  • Keep ttscgraph dump as a compatible one-shot complete dump. Do not make the
    incremental serve path reconstruct that complete dump merely to split it
    again.

Consequence surface

  • packages/ttsc/internal/graph: shardable construction, deterministic fact
    ownership and immutable cached results;
  • packages/ttsc/cmd/ttscgraph: resident generation, invalidation, protocol,
    cancellation, retry and shutdown;
  • packages/graph: client protocol types and compatibility errors;
  • samchon/compiler-graph: normalized transaction adapter and atomic store;
  • platform packages that ship ttscgraph; and
  • graph documentation and exact protocol/schema parity tests.

The existing full-dump CLI, diagnostics, project-reference handling, plugin
application, Windows/POSIX path identity and source/disk provenance must remain
correct.

Acceptance and verification

  • Initial load publishes a complete, deterministic shard generation whose
    assembled dump is fact-for-fact equal to the current full builder oracle.
  • A resident no-op emits no shard payload and performs no graph rebuild.
  • A private body edit changes only its source shard and any compiler-proven
    dependent closure; unrelated shard digests and payloads are reused.
  • Public signature, import resolution, project-reference, config, dependency,
    source create/delete/rename and linked-plugin changes invalidate the complete
    required closure without running clean or starting another compiler.
  • Cancellation, crash, malformed frame, stale base, digest mismatch and input
    movement preserve the prior published generation; retry converges.
  • Cross-file calls, accesses, instantiations, type references, inheritance,
    overrides, exports, renders, diagnostics and external leaves remain identical
    to the complete-build oracle after every accepted delta.
  • Deterministic tests cover empty, singleton, unchanged, body edit, public API
    edit, create, delete, rename, configuration movement, analysis error,
    cancellation and recovery.
  • A pinned real-project trace separates compiler refresh, graph extraction,
    serialization and consumer commit. It proves no declaration-proportional RPC
    path and no whole-project graph serialization on the ordinary edit lane.
  • pnpm format, focused Go tests, pnpm test:go, pnpm build and the relevant
    graph feature tests pass.

Do not close this issue with a consumer-only cache, a full-dump partitioning
pass, an mtime-only key or a benchmark-specific shortcut.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions