-
Notifications
You must be signed in to change notification settings - Fork 484
feat(cli): port supabase seed buckets to native TypeScript
#5651
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Coly010
wants to merge
25
commits into
develop
Choose a base branch
from
claude/refine-local-plan-j6ksdp
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
25 commits
Select commit
Hold shift + click to select a range
4cd9b2c
feat(cli): port `seed buckets` to native TypeScript
claude 2249cdf
test(cli): add seed buckets e2e + --local/--linked mutual-exclusivity
Coly010 b26a497
fix(cli): seed buckets Go-parity fixes from review
Coly010 3862ec8
fix(cli): seed buckets gateway auth, host, and key resolution Go-parity
Coly010 87b42f7
chore(cli): fix SIDE_EFFECTS.md table formatting
Coly010 c044de7
fix(cli): seed buckets stream uploads and resolve object paths under …
Coly010 bf9d7e2
fix(config): accept numeric file_size_limit and default vector bucket…
Coly010 6476f51
feat(cli): seed buckets --linked seeds the remote project (Go parity)
Coly010 6befb3d
feat(cli): trust the local Kong CA when seeding a TLS-enabled local s…
Coly010 5254b3a
fix(cli): seed buckets inherits storage-level file_size_limit and pre…
Coly010 f26e879
fix(cli): seed buckets matches Go LoadConfig validation/merge/env
Coly010 25f2a2b
fix(cli): seed buckets storage-limit validation, linked cache, config…
Coly010 cec81c1
fix(cli): seed buckets fails on malformed Storage list responses (Go …
Coly010 d4532c6
fix(cli): seed buckets requires exactly 200 from the Storage gateway …
Coly010 ea9e000
fix(cli): seed buckets parses create/update responses and hints local…
Coly010 97a2417
fix(cli): seed buckets accepts null vector list and fails on empty li…
Coly010 91274c0
fix(cli): seed buckets brackets IPv6 gateway host and rejects malform…
Coly010 b332a62
fix(cli): seed buckets size-numeral grammar + base-URL-derived local …
Coly010 5725ca8
fix(cli): seed buckets target by flag.Changed, JSON no-op result, rej…
Coly010 7d8ac8a
fix(cli): seed buckets accepts underscored size numerals (Go ParseFlo…
Coly010 e026b9a
fix(cli): seed buckets flag-conflict telemetry + Kong CA for explicit…
Coly010 3cac899
fix(cli): seed buckets validates config sizes before the no-op short-…
Coly010 1ca442e
fix(cli): seed buckets suppresses the local gateway hint on connectio…
Coly010 a1eb6fd
fix(cli): match Go's unconditional supabase-dir join for seed buckets…
Coly010 c8145f5
fix(cli): open symlink targets in seed buckets walk to match Go isUpl…
Coly010 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
174 changes: 147 additions & 27 deletions
174
apps/cli/src/legacy/commands/seed/buckets/SIDE_EFFECTS.md
Large diffs are not rendered by default.
Oops, something went wrong.
27 changes: 27 additions & 0 deletions
27
apps/cli/src/legacy/commands/seed/buckets/buckets.classify.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| /** | ||
| * Vector-bucket error classifiers — ports of `isVectorBucketsFeatureNotEnabled` | ||
| * and `isLocalVectorBucketsUnavailable` (`apps/cli-go/internal/seed/buckets/buckets.go:71-84`). | ||
| * | ||
| * Both inspect the error message string. The Storage gateway client raises | ||
| * status errors whose message reproduces Go's `Error status <d>: <body>`, so the | ||
| * same substring checks apply. | ||
| */ | ||
|
|
||
| /** Remote region has not enabled vector buckets yet (`buckets.go:71-73`). */ | ||
| export function legacyIsVectorBucketsFeatureNotEnabled(message: string): boolean { | ||
| return message.includes("FeatureNotEnabled"); | ||
| } | ||
|
|
||
| /** | ||
| * The local Storage service does not expose the vector routes (`buckets.go:75-84`): | ||
| * either it reports the vector service is not configured, or the `ListVectorBuckets` | ||
| * route returns 404 (older local image without vector support). | ||
| */ | ||
| export function legacyIsLocalVectorBucketsUnavailable(message: string): boolean { | ||
| return ( | ||
| message.includes("Vector service not configured") || | ||
| (message.includes("Error status 404:") && | ||
| message.includes("Route POST:") && | ||
| message.includes("ListVectorBuckets")) | ||
| ); | ||
| } |
46 changes: 46 additions & 0 deletions
46
apps/cli/src/legacy/commands/seed/buckets/buckets.classify.unit.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| import { describe, expect, it } from "@effect/vitest"; | ||
|
|
||
| import { | ||
| legacyIsLocalVectorBucketsUnavailable, | ||
| legacyIsVectorBucketsFeatureNotEnabled, | ||
| } from "./buckets.classify.ts"; | ||
|
|
||
| describe("legacyIsVectorBucketsFeatureNotEnabled", () => { | ||
| it("matches when the message mentions FeatureNotEnabled", () => { | ||
| expect( | ||
| legacyIsVectorBucketsFeatureNotEnabled('Error status 400: {"code":"FeatureNotEnabled"}'), | ||
| ).toBe(true); | ||
| }); | ||
|
|
||
| it("does not match an unrelated error", () => { | ||
| expect(legacyIsVectorBucketsFeatureNotEnabled("Error status 500: boom")).toBe(false); | ||
| }); | ||
| }); | ||
|
|
||
| describe("legacyIsLocalVectorBucketsUnavailable", () => { | ||
| it("matches the 'Vector service not configured' message", () => { | ||
| expect( | ||
| legacyIsLocalVectorBucketsUnavailable( | ||
| "Error status 409: The feature Vector service not configured is not enabled", | ||
| ), | ||
| ).toBe(true); | ||
| }); | ||
|
|
||
| it("matches a 404 on the ListVectorBuckets route", () => { | ||
| expect( | ||
| legacyIsLocalVectorBucketsUnavailable( | ||
| "Error status 404: Route POST:/vector/ListVectorBuckets not found", | ||
| ), | ||
| ).toBe(true); | ||
| }); | ||
|
|
||
| it("does not match a 404 on a different route", () => { | ||
| expect( | ||
| legacyIsLocalVectorBucketsUnavailable("Error status 404: Route POST:/something not found"), | ||
| ).toBe(false); | ||
| }); | ||
|
|
||
| it("does not match an unrelated error", () => { | ||
| expect(legacyIsLocalVectorBucketsUnavailable("Error status 500: boom")).toBe(false); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
53 changes: 53 additions & 0 deletions
53
apps/cli/src/legacy/commands/seed/buckets/buckets.e2e.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; | ||
| import { tmpdir } from "node:os"; | ||
| import { join } from "node:path"; | ||
| import { afterAll, beforeAll, describe, expect, test } from "vitest"; | ||
|
|
||
| import { runSupabase } from "../../../../../tests/helpers/cli.ts"; | ||
|
|
||
| const E2E_TIMEOUT_MS = 30_000; | ||
|
|
||
| /** | ||
| * Golden-path e2e: exercises the real compiled-binary boundary for the two | ||
| * network-free paths of `seed buckets`: | ||
| * - an empty `[storage]` config is a no-op (exit 0, no stdout); | ||
| * - `--local --linked` is rejected by the mutually-exclusive flag check. | ||
| * Bucket/object seeding parity is covered by the integration + unit suites. | ||
| */ | ||
| describe("supabase seed buckets (legacy)", () => { | ||
| let projectDir: string; | ||
|
|
||
| beforeAll(() => { | ||
| projectDir = mkdtempSync(join(tmpdir(), "supabase-seed-buckets-e2e-")); | ||
| mkdirSync(join(projectDir, "supabase"), { recursive: true }); | ||
| writeFileSync(join(projectDir, "supabase", "config.toml"), 'project_id = "test"\n'); | ||
| }); | ||
|
|
||
| afterAll(() => { | ||
| rmSync(projectDir, { recursive: true, force: true }); | ||
| }); | ||
|
|
||
| test( | ||
| "is a no-op with exit 0 when no buckets are configured", | ||
| { timeout: E2E_TIMEOUT_MS }, | ||
| async () => { | ||
| const { exitCode, stdout } = await runSupabase(["seed", "buckets"], { | ||
| entrypoint: "legacy", | ||
| cwd: projectDir, | ||
| }); | ||
| expect(exitCode).toBe(0); | ||
| expect(stdout.trim()).toBe(""); | ||
| }, | ||
| ); | ||
|
|
||
| test("rejects passing both --local and --linked", { timeout: E2E_TIMEOUT_MS }, async () => { | ||
| const { exitCode, stdout, stderr } = await runSupabase( | ||
| ["seed", "buckets", "--local", "--linked"], | ||
| { entrypoint: "legacy", cwd: projectDir }, | ||
| ); | ||
| expect(exitCode).toBe(1); | ||
| expect(`${stdout}${stderr}`).toContain( | ||
| "if any flags in the group [linked local] are set none of the others can be", | ||
| ); | ||
| }); | ||
| }); |
56 changes: 56 additions & 0 deletions
56
apps/cli/src/legacy/commands/seed/buckets/buckets.errors.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| import { Data } from "effect"; | ||
|
|
||
| /** | ||
| * Domain errors for `supabase seed buckets`. | ||
| * | ||
| * The Storage service-gateway calls fail with one of two shapes, mirroring Go's | ||
| * `pkg/fetcher`: | ||
| * - transport failure (`failed to execute http request`) → | ||
| * `LegacySeedStorageNetworkError` | ||
| * - non-2xx response (`Error status <d>: <body>`, `pkg/fetcher/http.go:112`) → | ||
| * `LegacySeedStorageStatusError` | ||
| * | ||
| * `message` reproduces Go's verbatim error text so the vector graceful-skip | ||
| * classifiers in `buckets.classify.ts` match on the same substrings Go inspects. | ||
| */ | ||
| export class LegacySeedStorageNetworkError extends Data.TaggedError( | ||
| "LegacySeedStorageNetworkError", | ||
| )<{ | ||
| readonly message: string; | ||
| }> {} | ||
|
|
||
| export class LegacySeedStorageStatusError extends Data.TaggedError("LegacySeedStorageStatusError")<{ | ||
| readonly status: number; | ||
| readonly body: string; | ||
| readonly message: string; | ||
| }> {} | ||
|
|
||
| /** | ||
| * Raised when `supabase/config.toml` cannot be parsed. Mirrors the `config push` | ||
| * CLI-1489 tradeoff (`config/push/push.handler.ts:96-114`): `loadProjectConfig` | ||
| * raises `ProjectConfigParseError` on `env(...)` refs over numeric/bool fields, | ||
| * which Go resolves transparently. | ||
| */ | ||
| export class LegacySeedConfigLoadError extends Data.TaggedError("LegacySeedConfigLoadError")<{ | ||
| readonly message: string; | ||
| }> {} | ||
|
|
||
| /** | ||
| * Raised when `--local` and `--linked` are both passed, reproducing cobra's | ||
| * `MarkFlagsMutuallyExclusive("local", "linked")` (`apps/cli-go/cmd/seed.go:32`). | ||
| */ | ||
| export class LegacySeedMutuallyExclusiveFlagsError extends Data.TaggedError( | ||
| "LegacySeedMutuallyExclusiveFlagsError", | ||
| )<{ | ||
| readonly message: string; | ||
| }> {} | ||
|
|
||
| /** | ||
| * Raised on `--linked` when the project's api-keys response yields no keys, | ||
| * mirroring Go's `tenant.GetApiKeys` → `errMissingKey` ("Anon key not found.", | ||
| * `apps/cli-go/internal/utils/tenant/client.go:16,80-82`), which aborts before | ||
| * the remote Storage client is built. Message matches Go verbatim. | ||
| */ | ||
| export class LegacySeedMissingApiKeyError extends Data.TaggedError("LegacySeedMissingApiKeyError")<{ | ||
| readonly message: string; | ||
| }> {} |
77 changes: 77 additions & 0 deletions
77
apps/cli/src/legacy/commands/seed/buckets/buckets.flags.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| import { Effect } from "effect"; | ||
|
|
||
| import { | ||
| VALUE_CONSUMING_LONG_FLAGS, | ||
| VALUE_CONSUMING_SHORT_FLAGS, | ||
| } from "../../../shared/legacy-db-target-flags.ts"; | ||
| import { LegacySeedMutuallyExclusiveFlagsError } from "./buckets.errors.ts"; | ||
|
|
||
| /** | ||
| * Detects which of `--local` / `--linked` were explicitly set on the command | ||
| * line, reproducing cobra's `pflag.Changed` for `seed`'s | ||
| * `MarkFlagsMutuallyExclusive("local", "linked")` (`apps/cli-go/cmd/seed.go:32`). | ||
| * | ||
| * Effect CLI's parsed flags carry no `Changed` bit, so we re-derive it from raw | ||
| * argv. Value-consuming flags (`--workdir <path>`, `-o <fmt>`, …) skip their | ||
| * value token to avoid false positives like `--workdir --linked`. | ||
| * | ||
| * Returned in cobra's alphabetically-sorted order `["linked", "local"]` so the | ||
| * rendered conflict string matches Go exactly. | ||
| */ | ||
| export function legacySeedChangedTargetFlags(args: ReadonlyArray<string>): ReadonlyArray<string> { | ||
| let linked = false; | ||
| let local = false; | ||
| let skipNext = false; | ||
|
|
||
| for (const token of args) { | ||
| if (skipNext) { | ||
| skipNext = false; | ||
| continue; | ||
| } | ||
| if (token === "--") break; | ||
|
|
||
| if (token.startsWith("--")) { | ||
| const eqIdx = token.indexOf("="); | ||
| const name = eqIdx === -1 ? token.slice(2) : token.slice(2, eqIdx); | ||
| const isBare = eqIdx === -1; | ||
| if (name === "linked") { | ||
| linked = true; | ||
| continue; | ||
| } | ||
| if (name === "local") { | ||
| local = true; | ||
| continue; | ||
| } | ||
| if (isBare && VALUE_CONSUMING_LONG_FLAGS.has(name)) skipNext = true; | ||
| continue; | ||
| } | ||
|
|
||
| if (token.startsWith("-") && token.length >= 2 && token.charAt(1) !== "-") { | ||
| if (token.length === 2 && VALUE_CONSUMING_SHORT_FLAGS.has(token.charAt(1))) { | ||
| skipNext = true; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| const setFlags: Array<string> = []; | ||
| if (linked) setFlags.push("linked"); | ||
| if (local) setFlags.push("local"); | ||
| return setFlags; | ||
| } | ||
|
|
||
| /** | ||
| * Reproduce cobra's `MarkFlagsMutuallyExclusive("local", "linked")` | ||
| * (`apps/cli-go/cmd/seed.go:32`). Go rejects this at flag validation — before | ||
| * `RunE`/`PersistentPostRun` — so it must NOT emit `cli_command_executed`; the | ||
| * command calls this BEFORE `withLegacyCommandInstrumentation`. | ||
| */ | ||
| export const legacyAssertSeedTargetsExclusive = Effect.fnUntraced(function* ( | ||
| args: ReadonlyArray<string>, | ||
| ) { | ||
| const setFlags = legacySeedChangedTargetFlags(args); | ||
| if (setFlags.length > 1) { | ||
| return yield* new LegacySeedMutuallyExclusiveFlagsError({ | ||
| message: `if any flags in the group [linked local] are set none of the others can be; [${setFlags.join(" ")}] were all set`, | ||
| }); | ||
| } | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go registers
--linkedand--localas persistent flags onseed(apps/cli-go/cmd/seed.go:29-32), so existing invocations likesupabase seed --linked bucketsare valid. Here the TSseedparent has no flag config and these flags are attached only to thebucketsleaf, so the parent-position form has no flag to consume before the handler runs; onlysupabase seed buckets --linkedis supported. Move or share these flags at theseedcommand level to preserve the Go command surface.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Accurate — Go registers
--linked/--localas persistent flags on theseedparent (cmd/seed.go:29-32), sosupabase seed --linked bucketsis valid, whereas the TS attaches them to thebucketsleaf, supporting onlysupabase seed buckets --linked. Escalating rather than fixing inline, because the faithful fix is a command/flag-wiring change with parser-level behavior I can't cheaply verify in the loop: declare the two flags viaCommand.withGlobalFlags([...])on theseedgroup (the Effect equivalent of Go's PersistentFlags — position-independent + scoped toseed), then refactor thebucketshandler to read them as global-flag tokens instead of the leafCommand.make("buckets", config)arg, and rethread them intowithLegacyCommandInstrumentation({ flags })for telemetry. (The mutual-exclusivity check already scans argv vialegacySeedChangedTargetFlags, so it's position-independent today.) Tracking as a CLI-surface follow-up; leaving open.