Close all 7 findings from the end-to-end claims audit (round 2)#11
Merged
Conversation
…SQL success SQL tasks now route through ProviderRegistry built from conduit.yaml connections in run/apply/backfill/serve. The subprocess fallback that printed 'SQL execution completed' with rows_affected=0 is now a hard error naming the unconfigured connection. (Claims-audit finding 1) Also: cmd_run's final failure message now surfaces the actual task error (e.g. the missing-connection message) to stderr instead of a generic "see task output above", and the native SQL provider's completion log line was reworded to "SQL execution finished via provider" so it can't be confused with the deleted fake-stub message. Signed-off-by: Jayveer Singh <[email protected]>
Both failure branches returned Ok(()) so CI gating on conduit apply was impossible. (Claims-audit finding 2, CLI half) Signed-off-by: Jayveer Singh <[email protected]>
DeploymentPlan now records the environment revision it was generated against; apply refuses a plan whose base version no longer matches the live environment, and refuses plans targeting a different environment. Docs updated to the real conflict output. (Claims-audit finding 3) Signed-off-by: Jayveer Singh <[email protected]>
cmd_apply now evaluates each executed task's contracts against its emitted evidence via ContractEvaluator. Error-severity failures abort before the environment is updated, exit non-zero, and print the DeploymentValidation summary. (Claims-audit finding 5, contracts half) Signed-off-by: Jayveer Singh <[email protected]>
…n output Signed-off-by: Jayveer Singh <[email protected]>
…nto run/apply Tasks with incremental config now get a real IncrementalContext (env vars via TaskContext.extra_env, SQL rewritten via rewrite_sql), emitted watermarks advance a WatermarkStore persisted at .conduit/watermarks.json, and --full-refresh actually forces a full refresh. (Claims-audit finding 5, incremental half) Also fixes a prerequisite gap found while wiring this up: the YAML `incremental:` block was parsed into YamlIncrementalConfig but never attached to the compiled Task (ParsedTask had no incremental field and resolver.rs hardcoded `incremental: None`), so declaring `incremental:` on a task was silently a no-op. yaml_parser.rs now resolves it via the existing resolve_incremental_config and threads it through ParsedTask into Task.incremental. Signed-off-by: Jayveer Singh <[email protected]>
…ig and task context trigger_run now inserts environment/triggered_by into the scheduler run config (the scheduler reads them from there); the serve executor and conduit run use the run's environment instead of hardcoding production; run gains --env. (Claims-audit finding 4, threading half) Signed-off-by: Jayveer Singh <[email protected]>
…g blank AppState now opens the durable snapshots_db (shared path with the CLI), loads environments.json (persisting after every env mutation), and rehydrates the run cache from the durable event log. Restarting serve no longer loses the operational view. (Claims-audit finding 4, persistence half) Signed-off-by: Jayveer Singh <[email protected]>
…onment Plans generated via POST /plan are cached by id; apply looks up the reviewed plan, enforces target-environment and base-version (409 on stale), executes tasks through the provider registry, validates contracts, stores snapshots, and records the env update with history. (Claims-audit finding 2, API half) Signed-off-by: Jayveer Singh <[email protected]>
conduit run executes dispatched tasks on a semaphore-bounded pool instead of serially awaiting each; backfill runs partitions through a JoinSet bounded by --max-concurrent. (Claims-audit finding 7, concurrency half) Signed-off-by: Jayveer Singh <[email protected]>
conduit worker now runs the real gRPC worker runtime; cluster status calls the ClusterStatus RPC; cluster drain uses a new DrainWorker RPC whose directive is delivered on the worker's next heartbeat. Worker-side SQL stub now fails honestly instead of reporting success. (Claims-audit finding 6, part A) Signed-off-by: Jayveer Singh <[email protected]>
Signed-off-by: Jayveer Singh <[email protected]>
…d dispatches to workers The banner-only path is gone: --distributed serves the coordinator gRPC endpoint on --bind (durable assignment recovery under .conduit/), maps scheduler dispatches onto the distributed protocol, and feeds worker results back into the scheduler. Exit code reflects the run outcome. (Claims-audit finding 6, part B) Signed-off-by: Jayveer Singh <[email protected]>
kill() without wait() leaves a zombie until the test binary exits; also silences the clippy zombie_processes warning. Signed-off-by: Jayveer Singh <[email protected]>
…mented behavior
API reference: remove unrouted endpoints (run cancel, SSE logs,
snapshots CRUD), fix path mismatches (dags/compile, dags/{id}/runs,
environments/promote, lineage/trace/*, /ws/events), document the real
plan/apply semantics (plan_id cache, 409 stale-plan, 422 apply_failed),
the real auth (--auth-enabled, Bearer keys), per-IP rate limiting, the
real error taxonomy, and real request/response shapes throughout.
CLI reference: rewritten from actual --help output. Removes fictional
commands (schedule, events, audit-log, snapshot *, health,
verify-snapshots, cleanup) and documents the real ones that were
missing (impact, backfill, worker, cluster, query, preview, env
set-policy/diff/history).
Concept docs: replace fictional snapshot/schedule/webhook/audit-log/
verify-snapshots/replay-with-modifications examples with the real
equivalents (env history/rollback/diff, replay --events-only, events
API, /ws/events), fix CONDUIT_ENV -> CONDUIT_ENVIRONMENT, and state
honestly what retention and event triggers do and don't support.
CLI help: run --distributed now says workers must connect.
Also commit the claims-audit fix plan. (Claims-audit finding 7, docs
half + residue from findings 2-6; closes the 2026-07-13 plan's Task 12)
Signed-off-by: Jayveer Singh <[email protected]>
Schema impact: report unavailableThe |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes all 7 findings from the 2026-07-13 end-to-end claims audit. Plan:
docs/superpowers/plans/2026-07-13-claims-audit-fixes.md.Findings → fixes
ProviderRegistrybuilt fromconduit.yamlin run/apply/backfill/serve; the subprocess fallback that printed fake success is now a hard error naming the unconfigured connection.POST /applyexecutes the stored plan (looked up byplan_id), validates contracts, stores snapshots, and bumps the environment version; CLI apply exits non-zero on any task failure or execution error.base_environment_version; both CLI and API reject stale plans (409 over HTTP) and plans targeting a different environment.TaskContext(run --env, APIenvironmentfield); serve reopens persistent snapshots/environments/run history instead of starting blank.ContractEvaluatoragainst emitted evidence and blocks on Error severity; incremental tasks get a realIncrementalContext(SQL rewrite +CONDUIT_*env vars), watermarks persist to.conduit/watermarks.json, and--full-refreshworks on run/apply/backfill.run --distributedstarts a real coordinator and dispatches to connected workers over gRPC;worker,cluster status, andcluster drainare real RPCs that fail loudly when no coordinator is reachable.--max-tasksand--max-concurrentare honored (semaphore-bounded); API reference, CLI reference, and concept docs rewritten to match actual routes, commands, and output.Verification
cargo test --workspace: 64 test binaries, 1,151 tests, 0 failures — including new black-box tests for real DuckDB SQL execution, loud missing-provider failure, stale-plan rejection, contract-blocked apply, watermark persistence, server restart recovery, environment propagation, and a two-process distributed run.cargo fmt --checkandcargo clippy --workspace --all-targetsclean.🤖 Generated with Claude Code