[miniflare] Support remote Hyperdrive bindings in local dev - #14900
[miniflare] Support remote Hyperdrive bindings in local dev#14900mack-erel wants to merge 5 commits into
Conversation
…idge Hyperdrive bindings could only ever point at a local database. Accept an object entry carrying a `remoteProxyConnectionString` alongside the existing plain connection string, which opts the binding into remote mode. For a remote binding, `HyperdriveProxyController.createRemoteTcpBridge` listens on 127.0.0.1 and relays each inbound connection byte-for-byte to the edge Hyperdrive binding over a WebSocket to the remote proxy's `connect` handler. The `hyperdrive:<name>` `external.tcp` designator points at that bridge, which keeps workerd unmodified — aiming a Hyperdrive designator at a Worker service crashes it. Local-only bindings keep taking the existing code path unchanged. Co-Authored-By: Claude Opus 5 <[email protected]>
🦋 Changeset detectedLatest commit: 429ba95 The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
@cloudflare/autoconfig
@cloudflare/config
create-cloudflare
@cloudflare/deploy-helpers
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-pool-workers
@cloudflare/workers-auth
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Allow `remote: true` on a `hyperdrive` binding to opt local dev into connecting to the deployed Hyperdrive configuration at the edge instead of a local database. Hyperdrive moves from `local-only` to `local-and-remote` in the binding support table, and the config validator accepts and validates the new field. When the flag is set and a remote proxy session is available, `hyperdriveEntry` hands miniflare the object form carrying the `remoteProxyConnectionString` so it can stand up the TCP bridge. `localConnectionString` is no longer required for these bindings, since there is no local database to point at. A session is not always there to be had — logged out, offline, or remote bindings turned off — so without one the binding falls back to its `localConnectionString` with a warning, and errors with instructions when it has none. Otherwise the empty connection string reaches miniflare and fails its URL validation with an opaque error. Co-Authored-By: Claude Opus 5 <[email protected]>
…entials The edge Hyperdrive proxy mints per-session credentials and uses the config id as the database name, so a database client reaching it through the TCP bridge must present those values — the user's local placeholder credentials only get as far as the server greeting. Add an `MF-HD-Seed` endpoint to ProxyServerWorker that returns a binding's edge `connectionString`. It is guarded on the header so it never intercepts ordinary connect/RPC/fetch traffic, and returns 400 without `MF-Binding` and 404 for a binding that exposes no connection string. `seedRemoteHyperdriveBindings` fetches that value for every remote Hyperdrive binding and writes it into the local binding config. `LocalRuntimeController` awaits it once the remote proxy session is ready and before miniflare options are built, since that step is synchronous. It is a no-op without a proxy session or remote Hyperdrive bindings. The seeded value is a live credential, so it is never logged. Co-Authored-By: Claude Opus 5 <[email protected]>
Extend the remote bindings connect suite with the Hyperdrive path: a test that drives a raw TCP client through `createRemoteTcpBridge` into a real ProxyServerWorker and asserts bytes are relayed in both directions, and three covering the `MF-HD-Seed` endpoint — the happy path returning the binding's connection string, a 400 for a missing `MF-Binding` header, and a 404 for an unknown binding. Co-Authored-By: Claude Opus 5 <[email protected]>
Co-Authored-By: Claude Opus 5 <[email protected]>
147ec12 to
429ba95
Compare
|
Thanks — both are fair. Fixed the first; here's my reasoning on the second. Empty connection string / cryptic error ( I also covered the adjacent case Devin didn't flag: when the session is unavailable but a
That said, if you'd prefer a stronger guard — a per-session nonce, or folding the seed into the JSRPC surface rather than a plain header — I'm happy to add it. Also fixed two unrelated CI failures: the changeset listed |
2 participants
Fixes #14842.
What
hyperdrivebindings arelocal-onlytoday:wrangler devconnects straight tolocalConnectionString, so a database that is only reachable through the deployed Hyperdrive configuration — for example one whose origin is a private-network VPC Service — cannot be exercised locally at all.This adds
remote: trueforhyperdrivebindings, matching the existing opt-in onkv_namespace/r2_bucket/d1:{ "hyperdrive": [{ "binding": "HYPERDRIVE", "id": "<your-hyperdrive-id>", "remote": true }] }With it set, local dev talks to the deployed Hyperdrive configuration and standard drivers (
mysql2,pg) work unchanged.How
A local TCP bridge, not a Worker designator. Miniflare's
HyperdriveProxyControllergainscreateRemoteTcpBridge, which listens on127.0.0.1and relays each connection byte-for-byte to the edge through the remote-bindings proxy's existingconnecthandler (#14712). Thehyperdrive:<name>designator stays anexternal.tcpservice pointing at that bridge, so workerd is unmodified.That last part is deliberate rather than incidental: pointing a Hyperdrive designator at a Worker service segfaults workerd (cloudflare/workerd#6901). I built workerd from source to check whether that was a quick fix — even with the client-side lifetime fix applied (attaching the adapter and
WorkerInterfaceto the returned stream, mirroringconnectImpl), the crash goes away but bytes still don't flow through a Worker target due to a second, target-side teardown issue. So the relay runs in the Miniflare Node process instead.Credential seeding. The edge mints per-session credentials, so a header-guarded
MF-HD-Seedendpoint onProxyServerWorkerreturns the edge binding'sconnectionString, andseedRemoteHyperdriveBindingswrites it into the local binding config before the (synchronous)buildMiniflareBindingOptionsruns. That is what lets drivers authenticate through the edge proxy, and whylocalConnectionStringbecomes optional for these bindings.Fully opt-in — without
remote: true, the existing local-only path is unchanged.Testing
Four new tests in
remote-bindings-connect.spec.tsdrive a raw TCP client throughcreateRemoteTcpBridgeinto a real (bundled)ProxyServerWorker, asserting bidirectional byte relay, plus theMF-HD-Seedhappy path, a 400 on a missingMF-Bindingheader, and a 404 on an unknown binding. Suite 19/19;packages/wranglerhyperdrive tests 65/65;turbo build check:type34/34 across miniflare + wrangler + remote-bindings; oxfmt and oxlint clean.Also verified end-to-end against a real Hyperdrive configuration (MySQL behind a Cloudflare Tunnel): a stock
mysql2pool authenticates andSELECT VERSION()returns the origin server's version rather than the proxy banner — i.e. queries reach the real database, as in production.Known constraints
MF-Connect-Addressis sent as a fixedhyperdrive.local:0— Hyperdrive's edgeconnect()ignores the address, but the relay path requires the header to be present.On the opt-in semantics
#14842 raised three possible shapes for the opt-in, and this PR implements the simplest one (
remote: true, consistent with the other bindings). The other two are implemented on the same core and I am happy to swap either in: a fallback variant (fall back tolocalConnectionStringwhen the remote session cannot be established), and an experimental-flag-gated variant. I opened this one as a concrete starting point rather than leaving the question abstract — say the word and I will re-submit in whichever direction you prefer.remotecarries the same meaning here as the already-documented opt-in on KV, R2 and D1 bindings. Happy to open a Cloudflare Docs PR for the Hyperdrive binding page if you'd like it called out there.