refactor(query_plan): keep from_source and legacy FROM fields in sync (P3 groundwork)#30
Merged
Merged
Conversation
Groundwork for P3 (correlated subqueries). The executor reads `from_source` first and only falls back to the deprecated `from_table` / `from_subquery`, but several transformers rewrote just one of the two. The parser populates both, so the stale copy is the one that gets executed. - in_operator_lifter, expression_lifter: repointed the FROM clause at the lifted CTE via `from_table` only, leaving `from_source` on the original base table (which has no lifted column). - cte_hoister, into_clause_remover: mutated `from_subquery` while `from_source::DerivedTable` kept a pre-hoist clone. Adds `impl SelectStatement` (there was none) with `set_from_table`, `set_from_subquery` and `map_from_subquery` so the two representations can only move together, and routes the four sites through them. `map_from_subquery` replaces the take-then-restore pattern, which left `from_source` stale in between and dropped the derived-table alias. Both regression tests parse real SQL rather than hand-building an AST: the existing fixtures set `from_source: None`, an input the parser never produces, which is why the desync survived. Verified both tests fail without the fix. Neither desync currently yields wrong results -- the stale copies happen to be valid statements -- so this pins the invariant before P3 starts relying on `from_source` being authoritative. Also deletes column_dependency_lifter.rs: absent from query_plan/mod.rs, referenced nowhere, and no longer compiles against the current AST. cargo test 672 passed (+2); DuckDB parity contract holds unchanged at 64 AGREE / 1 DIFFER / 7 GAP; examples FORMAL tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
test_examples.py hardcoded ./target/release/sql-cli, so the entire examples suite aborted on win32 with "not found" before running a single case. Mirrors the resolution already used by tests/comparison/engines.py. Note the suite also needs PYTHONIOENCODING=utf-8 PYTHONUTF8=1 on a cp1252 console, same as the parity harness, or it dies on the checkmark glyph. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This was referenced Jul 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for P3 — correlated subqueries (
docs/SQL_PARITY.md). No parity behaviour changes here; this makesfrom_sourcetrustworthy before P3 starts depending on it.The problem
The executor (
query_engine.rs:1217) readsfrom_sourcefirst and only falls back to the deprecatedfrom_table/from_subquery. The parser populates both. Several transformers rewrote only one — so the stale copy is the one that actually executes.in_operator_lifter.rs:218from_table→ lifted CTE,from_source→ original base table (no lifted column)expression_lifter.rs:187cte_hoister.rs:61from_subquery;from_source::DerivedTablekeeps a pre-hoist cloneinto_clause_remover.rs:44The fix
Adds
impl SelectStatement(there was none) withset_from_table,set_from_subquery,map_from_subquery, and routes all four sites through them so the representations can only move together.map_from_subqueryreplaces the take-then-restore pattern — that leftfrom_sourcestale in between and dropped the derived-table alias.Also deletes
column_dependency_lifter.rs: absent fromquery_plan/mod.rs, referenced nowhere, and no longer compiles against the current AST. Removes 9 deprecated-field sites and 14SqlExpressionmatch arms from the later P3 audit.Honest scoping
Neither desync currently produces wrong results. The stale copies happen to be valid statements — pre-hoist
from_sourcestill carries its own CTEs, so the executor gets a correct answer by a different route. I tried and failed to build an end-to-end query that exposes either. This pins the invariant rather than fixing live breakage.Both regression tests parse real SQL instead of hand-building an AST — the existing fixtures set
from_source: None, an input the parser never produces, which is exactly why the desync survived. Verified both fail without the fix:Verification
cargo test: 672 passed (+2). One pre-existing failure,history_protection_integration, unrelated — it setsAPPDATAto a tempdir but still loads the real roaming history.--check: contract holds, unchanged at 64 AGREE / 1 DIFFER / 7 GAP.localhostendpoints).cargo fmt --checkclean; no new clippy warnings in touched files.Second commit
tests/integration/test_examples.pyhardcoded./target/release/sql-cli, aborting the whole examples suite on Windows before running a case. Mirrors the resolution already intests/comparison/engines.py. Unrelated to the refactor — split out so it can be dropped independently.Not addressed (deliberately)
The wider
from_sourcemigration. It is blocked on real AST work —TableSourcehas no table-function variant (recursive_parser.rs:1132) andTableSource::Tablecannot carry an alias — which would dragJoinClausein too. Bigger than P3 itself; revisit separately.🤖 Generated with Claude Code