Fix the 10 defects surfaced by the end-to-end claims audit#10
Merged
Conversation
Correctness: - Retries: the scheduler now re-dispatches tasks after the retry delay via an internal timer channel, so `conduit run` no longer hangs on a failing task with retries (serve and backfill inherit the fix). Exponential backoff is wired end to end via `retry_backoff` (SDK kwarg, YAML field, compiler extraction); the dead executor RetryPolicy is removed. - Fingerprints: the compiler hashes Python task source (nested @task def plus module-level same-name def) into Task.source_hash and the planner folds it into the content key, so body edits are detected as Modified and cascade downstream. - Event sourcing: the scheduler persists run/task lifecycle events, the environment manager emits create/promote/rollback, and apply emits SnapshotCreated + PlanApplied; run/serve/backfill/env all attach the store, so `conduit replay` reconstructs real history. - Cron: `conduit serve` ticks the scheduler at the top of every minute, with per-DAG per-minute dedup; scheduler-initiated runs are recorded in the API run cache so they appear in /runs and the UI. - Pools: dispatch acquires slots, completion/failure releases them and wakes waiters across runs, retries re-acquire, and undefined pools run unlimited with a warning; definitions load from conduit.yaml. - SQL params: named :param placeholders are rewritten to native placeholders and bound as real query parameters (type-inferred) in all six sqlx providers; referencing a missing param errors clearly. - `conduit run` exits non-zero when the run fails; /api/v1/runs supports dagId/dag_id filtering; `env diff` prints full snapshot IDs. - Completed runs keep per-task logs (16KB cap), exposed as taskLogs in the run detail API and rendered on the run detail page. Honesty: - serve no longer seeds fabricated demo runs unless --demo is passed; seeded runs are labeled triggered_by="demo". - Docs match reality: no cgroup claim (timeouts are enforced; declared CPU/memory limits are not yet), stdout protocol with env/file XCom (not stdin), 12 implemented + 20 stub providers, qualified single-binary wording, Fingerprint documented as 64-bit SipHash. 1,121 workspace tests pass (24 added, written failing-first). Python SDK: 185 pass.
Schema impact: no changes detectedCompared 4 task outputs across 6 DAGs — every output is identical. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes every defect found in the end-to-end claims audit, each with failing-first tests and live verification:
Correctness
conduit runno longer hangs forever on a failing task with retries; serve and backfill inherit the fix. Exponential backoff wired end to end viaretry_backoff(SDK kwarg → YAML/decorator extraction → scheduler); dead executorRetryPolicyremoved.@taskdef + module-level same-name def) intoTask.source_hash; the planner folds it into the content key, so edits show as Modified and cascade.conduit replaynow reconstructs actual history.conduit serveticks the scheduler every minute with per-DAG per-minute dedup; scheduler-initiated runs appear in/runsand the UI.conduit.yaml; undefined pools run unlimited with a warning.:nameplaceholders rewritten to native placeholders and bound as real query parameters (type-inferred) across all six sqlx providers; injection payloads are literals; missing params error clearly.conduit runexits non-zero on a failed run;/api/v1/runssupportsdagId/dag_idfiltering;env diffprints full snapshot IDs; completed runs keep per-task logs (APItaskLogs+ run detail UI).Honesty
conduit serve --demo) and labeledtriggered_by="demo".Tests: 1,121 workspace tests pass (24 added), Python SDK 185 pass.
🤖 Generated with Claude Code