|
| 1 | +# Copilot instructions — dbt-sqlserver |
| 2 | + |
| 3 | +Purpose |
| 4 | +- Quickly orient AI coding agents to the structure, conventions, and workflows required to make safe, useful changes to the dbt-sqlserver adapter. |
| 5 | +- Focuses on concrete, discoverable facts: major entry points, test/run commands, files to edit for common changes, and CI/development quirks. |
| 6 | + |
| 7 | +Branch and patching purpose |
| 8 | +- This repository comes from a fork of upstream dbt-sqlserver |
| 9 | +- This file lives in the `patched` branch and documents the current state of the codebase |
| 10 | +- All new feature and bugfix branches should be created off `master` and merged into `patched` for internal production ready code. |
| 11 | + |
| 12 | +Big picture (keep in mind) |
| 13 | +- This repository implements a dbt adapter for Microsoft SQL Server / Azure SQL. Core adapter code is under `dbt/adapters/sqlserver/` and is built on top of `dbt.adapters.fabric` (`FabricAdapter`). |
| 14 | +- Key responsibilities: |
| 15 | + - Connection & auth: `sqlserver_connections.py` (pyodbc + AAD support; extends FabricConnectionManager) |
| 16 | + - Credentials: `sqlserver_credentials.py` (dataclass, inherits FabricCredentials) |
| 17 | + - Adapter surface: `sqlserver_adapter.py` (exports `ConnectionManager`, `Column`, `Relation`, constraint rendering, date function, and incremental strategies) |
| 18 | + - Relation policies and limits: `relation_configs/policies.py` and `sqlserver_relation.py` (identifier length enforcement — max 127 chars) |
| 19 | + - SQL helper macros & materializations: `dbt/include/sqlserver/macros/…` and `dbt/include/sqlserver/materializations/…` (use these as canonical SQL patterns) |
| 20 | + |
| 21 | +Concrete code patterns & examples |
| 22 | +- Adapter subclass pattern: `SQLServerAdapter(FabricAdapter)` sets class attributes to concrete implementations. To add new surface behavior, add/override methods in `sqlserver_adapter.py` and supply new classes for `ConnectionManager`, `Column`, or `Relation`. |
| 23 | +- Connection/auth extension: `sqlserver_connections.py` merges Fabric's `AZURE_AUTH_FUNCTIONS` and adds `"serviceprincipal"` and `"msi"` keys. To add new auth types, provide a function that returns an `azure.core.credentials.AccessToken` and register it in `AZURE_AUTH_FUNCTIONS`. |
| 24 | +- Constraints: `render_model_constraint` produces SQL like `add constraint {name} unique nonclustered(col1,col2)` — note the adapter intentionally uses nonclustered constraints. |
| 25 | +- Limits and pagination: `SQLServerRelation.render_limited()` uses SQL Server `TOP` and special aliasing for `limit == 0`. |
| 26 | +- Versioning: adapter version is declared in `dbt/adapters/sqlserver/__version__.py` — `sqlserver_connections` uses it to set `APP` in the connection string. |
| 27 | + |
| 28 | +Tests & CI (how to run locally and what CI does) |
| 29 | +- Install dev/test deps locally with `make dev` or `uv venv .venv && uv pip install --python .venv/bin/python -r dev_requirements.txt`. |
| 30 | +- Unit tests: `make unit` or `pytest -n auto -ra -v tests/unit`. |
| 31 | +- Functional tests: `make functional` or: |
| 32 | + - Example: `pytest -ra -v tests/functional --profile "ci_sql_server"` |
| 33 | + - Functional tests select dbt profile via `--profile` (see `tests/conftest.py` for profiles: `ci_sql_server`, `ci_azure_*`, `user`, `user_azure`) |
| 34 | + - CI sets environment variables such as `DBT_TEST_USER_1`, `SQLSERVER_TEST_DRIVER` (see `.github/workflows/integration-tests-sqlserver.yml`). |
| 35 | +- Local SQL Server for functional testing: `make server` (uses `docker-compose.yml` and `devops/server.Dockerfile`) or `docker compose up -d`. |
| 36 | +- CI images and behavior: workflows build and use GHCR images named like `ghcr.io/${{ github.repository }}:CI-<py>-msodbc18` and `server-<version>`; updating CI container logic requires editing `devops/*` and `.github/workflows/*`. |
| 37 | + |
| 38 | +Development environment & onboarding |
| 39 | +- Recommended Python version for development: Python 3.10. |
| 40 | +- Preferred bootstrapping flow uses `uv` and a local `.venv`: |
| 41 | + - `uv venv .venv` |
| 42 | + - `uv pip install --python .venv/bin/python -r dev_requirements.txt` |
| 43 | +- Install dev dependencies and enable pre-commit with `make dev`; run `make help` to list Makefile targets (e.g. `make unit`, `make functional`, `make server`). |
| 44 | +- A devcontainer is provided (since v1.7.2) for a consistent development environment — look for `.devcontainer` in the repo when onboarding. |
| 45 | + |
| 46 | +Test environment file |
| 47 | +- Functional tests expect a `test.env` file in the repo root. Create it from the sample: `cp test.env.sample test.env`. |
| 48 | +- Functional tests require three test users for grants-related tests. Ensure these environment variables are set in `test.env` or your environment: |
| 49 | + - `DBT_TEST_USER_1` |
| 50 | + - `DBT_TEST_USER_2` |
| 51 | + - `DBT_TEST_USER_3` |
| 52 | + |
| 53 | +Releasing a new version |
| 54 | +- To cut a release: bump the version in `dbt/adapters/sqlserver/__version__.py`, create a git tag named `v<version>`, and push it to GitHub. The `release-version` workflow will publish the package to PyPI. |
| 55 | + |
| 56 | +Project conventions and gotchas |
| 57 | +- Many classes use Python dataclasses (credentials, relation policies) — follow that style for new small value types. |
| 58 | +- String length limit: identifiers are limited to 127 chars (`MAX_CHARACTERS_IN_IDENTIFIER` in `relation_configs/policies.py`). Tests and runtime enforce this in `sqlserver_relation.py`. |
| 59 | +- SQL is generated with explicit SQL Server semantics (e.g., uses `TOP`, nonclustered constraints, `CAST(... as datetimeoffset)` for time filters). Prefer existing macros in `dbt/include/sqlserver/macros` when creating SQL. |
| 60 | +- Authentication modes: `authentication` in credentials can be `sql`, `serviceprincipal`, `msi` or Fabric-provided options (see `sqlserver_connections.py` and tests). When adding auth modes, add tests under `tests/functional` and `tests/unit/adapters/mssql`. |
| 61 | +- Logging/debug: connection-level logs use logger name `sqlserver` (see `sqlserver_connections.py`). Use these logs when debugging connection/auth issues. |
| 62 | + |
| 63 | +Where to change common features |
| 64 | +- To change connection behavior or add a new driver tweak: edit `dbt/adapters/sqlserver/sqlserver_connections.py` and corresponding unit tests in `tests/unit/adapters/mssql`. |
| 65 | +- To add adapter-level behavior: edit `dbt/adapters/sqlserver/sqlserver_adapter.py` and add tests under `tests/functional/adapter/dbt`. |
| 66 | +- To expose or change macros/materializations: update files under `dbt/include/sqlserver/macro*` and `materializations/*`. |
| 67 | + |
| 68 | +Quick checklist for a typical change |
| 69 | +- Add/modify implementation in `dbt/adapters/sqlserver/*`. |
| 70 | +- Add unit tests in `tests/unit` and functional tests in `tests/functional` (pick proper profile). |
| 71 | +- Run `make dev` to install test deps and enable pre-commit, run `make unit` and `make functional` locally where possible. |
| 72 | +- If the change impacts connection images or CI, update `devops/Dockerfile*`, `docker-compose.yml`, and `.github/workflows/*` accordingly. |
| 73 | + |
| 74 | +If anything here is unclear or you need more detail on specific areas (auth flows, CI images, macros), find files or behavior you want expanded and iterate. |
0 commit comments