Version 3.0.0 Proposal draft with sqlglot as the parsing library (#617)

* wip - working version with sqlglot to refactor

* wip - extract bracketed names into new dialect

* wip - Rewrite REPLACE INTO → INSERT INTO in _ast.py._parse() before sqlglot parses it, so sqlglot produces a proper exp.Insert AST instead of exp.Command and parses it correctly without falling back to regex

* wip - Rewrite REPLACE INTO → INSERT INTO in _ast.py._parse() before sqlglot parses it, so sqlglot produces a proper exp.Insert AST instead of exp.Command and parses it correctly without falling back to regex

* add docstings, refactor to simplify most complex methods, add few tests from open issues to verify if it's handling the issues better than the old version, remove internal tokens and produce only list of strings if needed, remove compatibility layer to v1

* add tests from open issues that now passes and some small fixes to accommodate additional 3 tests

* accept capitalization and explicit as from sqlglot as opinionated defaults to simplify the bodies extraction

* simplify logic, refactor into classes with related functionalities

* additional simplification and cleanup

* remove unnecessary wrappers

* further simplification - add also architecture overview with charts and main notes

* next portion of cleanup, renaming files, update also agents.md file, switch to ruff for formating and linting

* refactor other functionalities from ast parser into separate classes

* change to ruff also in CI. add mypy and fix typing errors, add mypy to CI

* fix remaining mypy errors in untyped code

* further fixes and duplication cleanup

* fix unused code, bump coverage - add todo to revisit corner cases later for now mark nocover as unreachable from parser and this is the only entrypoint we want for majority of the tests

* add features to handle unnamed queries, extracting properly hive tables from queries with subscripts, some additional issues that were already fixed were documented by tests, some cleanup and refactor to decrease unreachable paths

* fix mypy - add additional test for next already solved issue

* add additional test for next already solved issue

* remove unreachable stars without table node handling - it's either raw star or star with table when prefixed with table name/alias - unreachable code

* raise more meaningful error on invalid queries, raise on cte without name instead of silently skipping, extract mypy and ruff into separate workflows

* reorder methods, refactor complicated conditions into helper methods, add more descriptive docstrings and add sample queries in majority of the code flow branches to easier navigate the code

* handle redshift append clause with custom dialect, clean up table extractor and add more descriptive docstrings

* fix typing to go with 3.10 flow not deprecated typing ones. add support for nested ctes, cleanup nested resolver and simplify code there. remove unnecessary guards

* additional cleanup in dialect_parser.py and query_type_extractor.py

* cleanup in parser and docstring refactor, cleanup of thr remaining todoes

* fixes after the code review

* additional changes and optimizations after initial review

* break internal cyclical imports, change test to use parser instead of internal method

* update docs and readme, additional cleanup when possible -> use native sqlglot features whenever possible

* minor cleanup

---------

Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]>

Author: collerek

.flake8

@@ -23,9 +23,7 @@ jobs:
               if: "${{ steps.metadata.outputs.update-type ==
                   'version-update:semver-minor' ||
                   steps.metadata.outputs.update-type ==
-                  'version-update:semver-patch' ||
-                  steps.metadata.outputs.dependency-names ==
-                  'black' }}"
+                  'version-update:semver-patch' }}"
 
               # https://cli.github.com/manual/gh_pr_merge
               run: gh pr merge --auto --squash "$PR_URL"

.github/workflows/auto-merge-dependabot.yml

@@ -0,0 +1,30 @@
+name: Lint
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version: "3.12"
+
+    - name: Install Poetry
+      uses: snok/[email protected]
+      with:
+        version: latest
+        virtualenvs-create: true
+        virtualenvs-in-project: true
+
+    - name: Install dependencies with poetry
+      run: poetry install --no-root
+
+    - name: Lint with ruff
+      run: make lint

.github/workflows/black.yml

@@ -42,9 +42,6 @@ jobs:
     - name: Install Python wheel support to speed up things
       run: pip install wheel
 
-    - name: Pre-install black
-      run: pip install black
-
     # https://github.com/marketplace/actions/install-poetry-action
     - name: Install Poetry
       uses: snok/[email protected]
@@ -77,8 +74,5 @@ jobs:
         pip install coveralls
         poetry run coveralls --service=github
 
-    - name: Lint with pylint
-      run: make lint
-
     - name: Build a distribution package
       run: poetry build -vvv

.github/workflows/lint.yml

@@ -0,0 +1,30 @@
+name: Type Check
+
+on:
+  push:
+    branches: [ master ]
+  pull_request:
+
+jobs:
+  type-check:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version: "3.12"
+
+    - name: Install Poetry
+      uses: snok/[email protected]
+      with:
+        version: latest
+        virtualenvs-create: true
+        virtualenvs-in-project: true
+
+    - name: Install dependencies with poetry
+      run: poetry install --no-root
+
+    - name: Type check with mypy
+      run: make type_check