Skip to content

Commit a3990fd

Browse files
burmeciaclaude
andauthored
docs: add CLAUDE.md for AI assistant guidance (supabase#569)
Comprehensive documentation for AI assistants including: - Repository structure and workspace configuration - ForeignDataWrapper trait and core interfaces - Development workflows (building, testing, debugging) - Code patterns and conventions for FDW implementations - Available native and Wasm FDWs - CI/CD pipelines and contribution guidelines https://claude.ai/code/session_01EYhGdmQMcA4JJK69tXTUYq Co-authored-by: Claude <[email protected]>
1 parent 1c45063 commit a3990fd

1 file changed

Lines changed: 342 additions & 0 deletions

File tree

CLAUDE.md

Lines changed: 342 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,342 @@
1+
# CLAUDE.md - AI Assistant Guide for Wrappers
2+
3+
## Project Overview
4+
5+
Wrappers is a development framework for PostgreSQL Foreign Data Wrappers (FDWs), written in Rust. It enables querying external data sources (APIs, databases, files) as if they were regular PostgreSQL tables. The project is maintained by [Supabase](https://supabase.com).
6+
7+
**Documentation**: https://fdw.dev/ | **API Docs**: https://docs.rs/supabase-wrappers
8+
9+
## Repository Structure
10+
11+
```
12+
wrappers/
13+
├── supabase-wrappers/ # Core FDW framework library (crates.io: supabase-wrappers)
14+
├── supabase-wrappers-macros/ # Procedural macros (#[wrappers_fdw])
15+
├── wrappers/ # Native FDW implementations (PostgreSQL extension)
16+
│ └── src/fdw/ # Individual FDW implementations
17+
├── wasm-wrappers/ # WebAssembly-based FDW implementations (separate workspace)
18+
│ └── fdw/ # Individual Wasm FDW crates
19+
└── docs/ # MkDocs documentation site
20+
```
21+
22+
## Workspace Configuration
23+
24+
The project uses Cargo workspaces with the following structure:
25+
26+
- **Main workspace** (`Cargo.toml`): Contains `supabase-wrappers`, `supabase-wrappers-macros`, and `wrappers`
27+
- **Wasm workspace** (`wasm-wrappers/fdw/Cargo.toml`): Separate workspace for Wasm FDWs (excluded from main)
28+
29+
**Rust version**: 1.88.0 (specified in `workspace.package`)
30+
**pgrx version**: 0.16.1 (PostgreSQL extension framework)
31+
32+
## Key Components
33+
34+
### supabase-wrappers (Core Framework)
35+
36+
The core library providing the `ForeignDataWrapper` trait (`supabase-wrappers/src/interface.rs`):
37+
38+
```rust
39+
pub trait ForeignDataWrapper<E: Into<ErrorReport>> {
40+
// Required methods
41+
fn new(server: ForeignServer) -> Result<Self, E>;
42+
fn begin_scan(&mut self, quals: &[Qual], columns: &[Column], sorts: &[Sort], limit: &Option<Limit>, options: &HashMap<String, String>) -> Result<(), E>;
43+
fn iter_scan(&mut self, row: &mut Row) -> Result<Option<()>, E>;
44+
fn end_scan(&mut self) -> Result<(), E>;
45+
46+
// Optional methods for modification
47+
fn begin_modify(&mut self, options: &HashMap<String, String>) -> Result<(), E>;
48+
fn insert(&mut self, row: &Row) -> Result<(), E>;
49+
fn update(&mut self, rowid: &Cell, new_row: &Row) -> Result<(), E>;
50+
fn delete(&mut self, rowid: &Cell) -> Result<(), E>;
51+
fn end_modify(&mut self) -> Result<(), E>;
52+
53+
// Optional methods
54+
fn re_scan(&mut self) -> Result<(), E>;
55+
fn get_rel_size(...) -> Result<(i64, i32), E>;
56+
fn import_foreign_schema(...) -> Result<Vec<String>, E>;
57+
fn validator(options: Vec<Option<String>>, catalog: Option<Oid>) -> Result<(), E>;
58+
}
59+
```
60+
61+
### Key Data Types (interface.rs)
62+
63+
- `Cell`: Enum representing data values (Bool, I8-I64, F32-F64, String, Date, Timestamp, Json, Uuid, arrays)
64+
- `Row`: Collection of column names and cells
65+
- `Column`: Column metadata (name, number, type_oid)
66+
- `Qual`: WHERE clause predicate (field, operator, value, use_or)
67+
- `Sort`: ORDER BY specification
68+
- `Limit`: LIMIT/OFFSET values
69+
70+
### supabase-wrappers-macros
71+
72+
Provides the `#[wrappers_fdw]` attribute macro that generates:
73+
- `<name>_fdw_handler()` - FDW handler entry point
74+
- `<name>_fdw_validator()` - Option validation function
75+
- `<name>_fdw_meta()` - Metadata function
76+
77+
Usage:
78+
```rust
79+
#[wrappers_fdw(
80+
version = "0.1.0",
81+
author = "Supabase",
82+
website = "https://example.com",
83+
error_type = "MyFdwError" // Required
84+
)]
85+
pub struct MyFdw { ... }
86+
```
87+
88+
## Available FDWs
89+
90+
### Native FDWs (wrappers/src/fdw/)
91+
92+
| FDW | Feature Flag | Supports Write |
93+
|-----|--------------|----------------|
94+
| BigQuery | `bigquery_fdw` | Yes |
95+
| ClickHouse | `clickhouse_fdw` | Yes |
96+
| Stripe | `stripe_fdw` | Yes |
97+
| S3 Vectors | `s3vectors_fdw` | Yes |
98+
| S3 | `s3_fdw` | No |
99+
| Firebase | `firebase_fdw` | No |
100+
| Airtable | `airtable_fdw` | No |
101+
| Auth0 | `auth0_fdw` | No |
102+
| AWS Cognito | `cognito_fdw` | No |
103+
| DuckDB | `duckdb_fdw` | No |
104+
| Apache Iceberg | `iceberg_fdw` | No |
105+
| Logflare | `logflare_fdw` | No |
106+
| Redis | `redis_fdw` | No |
107+
| SQL Server | `mssql_fdw` | No |
108+
| HelloWorld | `helloworld_fdw` | No (demo) |
109+
110+
### Wasm FDWs (wasm-wrappers/fdw/)
111+
112+
Cal.com, Calendly, Clerk, Cloudflare D1, HubSpot, Infura, Notion, Orb, Paddle, Shopify, Slack, Snowflake
113+
114+
## Development Workflows
115+
116+
### Prerequisites
117+
118+
```bash
119+
# Install Rust toolchain
120+
rustup install 1.88.0
121+
rustup default 1.88.0
122+
123+
# Install pgrx
124+
cargo install --locked cargo-pgrx --version 0.16.1
125+
126+
# Initialize pgrx with PostgreSQL
127+
cargo pgrx init --pg15 /usr/lib/postgresql/15/bin/pg_config
128+
129+
# For Wasm development
130+
cargo install --locked cargo-component --version 0.21.1
131+
rustup target add wasm32-unknown-unknown
132+
```
133+
134+
### Building
135+
136+
```bash
137+
# Build native FDWs
138+
cd wrappers
139+
cargo build --features "native_fdws pg15"
140+
141+
# Build specific FDW
142+
cargo build --features "stripe_fdw pg15"
143+
144+
# Build Wasm FDWs
145+
cd wasm-wrappers/fdw
146+
cargo component build --release --target wasm32-unknown-unknown
147+
```
148+
149+
### Running Interactive Development
150+
151+
```bash
152+
cd wrappers
153+
cargo pgrx run pg15 --features stripe_fdw
154+
```
155+
156+
Then in psql:
157+
```sql
158+
create extension if not exists wrappers;
159+
create foreign data wrapper stripe_wrapper
160+
handler stripe_fdw_handler
161+
validator stripe_fdw_validator;
162+
```
163+
164+
### Testing
165+
166+
```bash
167+
# Start test containers
168+
cd wrappers
169+
docker compose -f .ci/docker-compose-native.yaml up -d
170+
171+
# Run all native FDW tests
172+
cargo pgrx test --features "native_fdws pg15"
173+
174+
# Run specific FDW tests
175+
cargo pgrx test --features "stripe_fdw pg15"
176+
177+
# Run Wasm FDW tests (requires building Wasm packages first)
178+
docker compose -f .ci/docker-compose-wasm.yaml up -d
179+
cargo pgrx test --features "wasm_fdw pg15"
180+
```
181+
182+
### Code Quality
183+
184+
```bash
185+
# Format check
186+
cargo fmt --check
187+
188+
# Clippy (must pass with -D warnings)
189+
RUSTFLAGS="-D warnings" cargo clippy --all --tests --no-deps --features native_fdws,helloworld_fdw
190+
```
191+
192+
### Installing Extension
193+
194+
```bash
195+
cargo pgrx install --pg-config /path/to/pg_config --features stripe_fdw
196+
```
197+
198+
## Code Patterns and Conventions
199+
200+
### Error Handling Pattern
201+
202+
Each FDW defines a custom error type:
203+
204+
```rust
205+
use thiserror::Error;
206+
207+
#[derive(Error, Debug)]
208+
enum MyFdwError {
209+
#[error("API error: {0}")]
210+
ApiError(String),
211+
#[error("Invalid option: {0}")]
212+
InvalidOption(String),
213+
}
214+
215+
impl From<MyFdwError> for ErrorReport {
216+
fn from(e: MyFdwError) -> Self {
217+
ErrorReport::new(PgSqlErrorCode::ERRCODE_FDW_ERROR, e.to_string(), "")
218+
}
219+
}
220+
```
221+
222+
### FDW Implementation Pattern
223+
224+
```rust
225+
use supabase_wrappers::prelude::*;
226+
227+
#[wrappers_fdw(
228+
version = "0.1.0",
229+
author = "Author",
230+
website = "https://example.com",
231+
error_type = "MyFdwError"
232+
)]
233+
pub struct MyFdw {
234+
// State fields
235+
client: Option<Client>,
236+
rows: Vec<Row>,
237+
row_idx: usize,
238+
}
239+
240+
impl ForeignDataWrapper<MyFdwError> for MyFdw {
241+
fn new(server: ForeignServer) -> Result<Self, MyFdwError> {
242+
// Initialize from server options
243+
Ok(Self { client: None, rows: vec![], row_idx: 0 })
244+
}
245+
246+
fn begin_scan(
247+
&mut self,
248+
quals: &[Qual],
249+
columns: &[Column],
250+
sorts: &[Sort],
251+
limit: &Option<Limit>,
252+
options: &HashMap<String, String>,
253+
) -> Result<(), MyFdwError> {
254+
// Fetch data, apply pushdown
255+
self.row_idx = 0;
256+
Ok(())
257+
}
258+
259+
fn iter_scan(&mut self, row: &mut Row) -> Result<Option<()>, MyFdwError> {
260+
if self.row_idx < self.rows.len() {
261+
row.replace_with(self.rows[self.row_idx].clone());
262+
self.row_idx += 1;
263+
Ok(Some(()))
264+
} else {
265+
Ok(None)
266+
}
267+
}
268+
269+
fn end_scan(&mut self) -> Result<(), MyFdwError> {
270+
self.rows.clear();
271+
Ok(())
272+
}
273+
}
274+
```
275+
276+
### File Organization for FDWs
277+
278+
Each native FDW follows this structure:
279+
```
280+
wrappers/src/fdw/<name>_fdw/
281+
├── mod.rs # Error types and module exports
282+
├── <name>_fdw.rs # Main implementation
283+
└── tests.rs # pgrx tests
284+
```
285+
286+
### Query Pushdown
287+
288+
FDWs receive pushdown hints through `begin_scan`:
289+
- `quals`: WHERE predicates to filter at source
290+
- `sorts`: ORDER BY to sort at source
291+
- `limit`: LIMIT/OFFSET to limit at source
292+
293+
Use `Qual::deparse()` to convert to SQL-like strings.
294+
295+
## PostgreSQL Version Support
296+
297+
Supported via feature flags: `pg13`, `pg14`, `pg15` (default), `pg16`, `pg17`, `pg18`
298+
299+
## CI/CD
300+
301+
GitHub Actions workflows in `.github/workflows/`:
302+
- `test_wrappers.yml`: Tests native and Wasm FDWs
303+
- `test_supabase_wrappers.yml`: Tests core framework
304+
- `release.yml`: Releases native FDWs
305+
- `release_wasm_fdw.yml`: Releases Wasm FDWs
306+
- `coverage.yml`: Code coverage
307+
- `docs.yml`: Documentation deployment
308+
309+
## Common Tasks
310+
311+
### Adding a New Native FDW
312+
313+
1. Create directory `wrappers/src/fdw/<name>_fdw/`
314+
2. Add feature flag in `wrappers/Cargo.toml`
315+
3. Add to `native_fdws` feature list
316+
4. Implement `ForeignDataWrapper` trait
317+
5. Add tests in `tests.rs`
318+
6. Add documentation in `docs/catalog/`
319+
320+
### Debugging
321+
322+
Use pgrx `notice!` macro for debug output:
323+
```rust
324+
use pgrx::notice;
325+
notice!("Debug: {:?}", value);
326+
```
327+
328+
### Working with Options
329+
330+
Options come from `CREATE SERVER` and `CREATE FOREIGN TABLE`:
331+
```rust
332+
fn begin_scan(&mut self, ..., options: &HashMap<String, String>) {
333+
let table_name = options.get("table").unwrap_or(&"default".to_string());
334+
}
335+
```
336+
337+
## Important Notes
338+
339+
- Native FDW contributions are not currently accepted until API stabilizes (v1.0)
340+
- Wasm FDWs are preferred for community contributions
341+
- Materialized views with foreign tables may cause backup restoration issues
342+
- Use `rowid_column` table option to enable INSERT/UPDATE/DELETE operations

0 commit comments

Comments
 (0)