fix(server): expand ~ and anchor relative package locations at the config

[mlennie]

Two small fixes so a package can live outside the repo, which is where most people want theirs.

That already worked, but only if you had read the code. Two things stood in the way.

~/ was accepted and then never expanded. isLocalPath matches ~/ explicitly, but a tilde path is not absolute, so it got joined onto the server root and died with Failed to mount local directory "<serverRoot>/~/my-packages/sales". Tilde is the most natural way to write a path outside the repo, and the code advertised support for it.

A relative location anchored at the server root, not at the config it appears in. Those are the same directory whenever the config is found at <serverRoot>/publisher.config.json, which covers every config in this repo and every Docker recipe in docs/deployment.md, so nothing changes for them. They diverge under --config <path>: the form docs/configuration.md describes ("point at a directory that holds your own publisher.config.json + packages") did not work, because launched from elsewhere ./sales resolved next to the server root rather than next to the config. Anchoring at the config's directory lets a config and its packages move together, which is what makes one worth committing instead of carrying absolute machine-specific paths.

This is a behaviour change for anyone whose --config <path> names a file in a directory other than the server root, a subdirectory of it included, and whose packages use a relative location. The old rule was undocumented rather than deliberate, so nothing promised it. The symptom if it bites is quiet: a location that cannot be mounted is not fatal to the process, so the server still reports serving, but it fails the whole environment, so every package in it is gone rather than just the one that moved. RELEASE_NOTES.md carries the migration note.

Two cases keep the server root as the anchor: the bundled default, which ships inside the installed package, so a relative location POSTed at runtime under a zero-arg npx @malloy-publisher/server would otherwise resolve somewhere under node_modules; and a --config naming a directory rather than a file, which cannot be read as a config at all, so its parent is nobody's intended anchor.

The second commit declares winston. packages/server/src/logger.ts imports it and no package.json in the repo declares it: it resolves only because the Snowflake and Databricks drivers pull it in transitively, and it is not in the bundler's external list, so bun build inlines whatever the driver tree happened to hoist. The day a driver drops it, the build stops resolving a module the server imports directly. The pinned version is the one already in the lock, so the resolved package set is unchanged.

Docs

docs/configuration.md gains the rule and a config example that parses, and its SERVER_ROOT row no longer claims to be the config's home when --config says otherwise. The malloy-html-data-apps skill said a relative location is "relative to --server_root", which this change falsifies; that text is served to agents as an MCP prompt via skills_bundle.json, so the bundle is regenerated too.

Testing

Gate green: 1138 unit, 207 integration, 0 fail. Verified against a live server rather than from the test suite alone, because a load failure is quiet: the server reports serving either way, omitting the package if it mounted but is bad, or dropping the whole environment if the location could not be mounted at all.

• a ~/ location, a --config bundle with relative locations, and a relative --config each failed before and serve after
• the documented example serves when taken verbatim out of the markdown
• a --config naming a directory boots with no environments rather than a bogus anchor
• the bundled examples still load from packages/server unchanged, which is the no-op claim above
• a real package (examples/storefront, parquet and all) copied outside the repo and served from a config beside it: it loads, its duckdb.table('data/*.parquet') sources resolve to the same totals as in-repo, and its notebook and public/ page serve

The anchoring has its own spec that puts the config and the server root in different directories, since a spec that lets them coincide cannot tell the two rules apart. Each behaviour is mutation-tested: reverting the tilde expansion, the anchor, the bundled-default guard, or the absolute-anchor resolve each fails its own test.

[sagarswamirao]

Can you add info about PUBLISHER_USE_BUNDLED_DEFAULT env var to the configuration.md?

[jswir]

Peer Review

Well-shaped fix overall: resolution is centralized in one resolveLocalPath/resolvePackageLocation helper (unifying the two hand-synced anchor sites that f52cfc38 explicitly flagged), tests assert resolved-path values against the real os.homedir/PUBLISHER_CONFIG_PATH oracle, and the source-containment model is not weakened. One item worth fixing before merge.

Important

1. os.homedir() is evaluated on every local-path resolution, not just ~/ paths — packages/server/src/service/environment_store.ts:131 (default param, reached from resolveLocalPath ~L1142–1146).

resolvePackageLocation(location, anchorDir, homeDir = os.homedir()) is called with two args, so the default fires for ./sales and /data/pkg too — locations that never contain a ~. Two failure modes, both regressions on the common non-tilde path:

• Throw: os.homedir() throws when HOME/USERPROFILE is unset and there's no passwd entry (distroless / runAsUser:<uid> container with no /etc/passwd row). An env using ordinary relative locations would then throw at load where it previously worked.
• Silent wrong path: if os.homedir() returns "", ~/foo → path.join("", "foo") = "foo", which then anchors under the config dir instead of failing loudly — an invalid value that still drives resolution.

Fix both by resolving home lazily inside the startsWith("~/") branch and guarding empty:

const home = homeDir ?? os.homedir();
if (!home) throw new Error(`Cannot expand "~" in location "${location}": home directory is not set`);

Keep the eager default off the non-tilde path.

Nits

1. environment_store_anchoring.spec.ts — the spec pins that the package resolves under the config dir but not the negative (that it's not found under serverRoot). Add the negative assertion so the test can't pass if resolution silently reverts to server-root anchoring.

2. environment_store.ts:131 — location.startsWith("~/") is POSIX-separator-specific: ~\ (Windows) and bare ~ aren't handled. FilesystemPath now advertises `~/${string}` as supported surface (config.ts:74). Behavior for bare ~/~user/foo is correct (rejected as invalid), just untested — add one assertion pinning the rejection, and either handle ~\ or add a "POSIX home paths only" note by the type.

3. environment_store.ts:1142 — resolveLocalPath re-runs getPublisherConfigDir() (two existsSync + one statSync) per package. Load path, not hot path, so minor — but per "resolve once at setup" the anchor could be computed once per environment load and threaded in.

Security / containment

No new bypass. The local mount source was never confined to the server root — #781 wired safeJoinUnderRoot onto the download/target sinks but deliberately left the source unguarded (comment at environment_store.ts:~1497: source "may legitimately be a relative path that resolves outside the server root; only the target is asserted"). Absolute paths and ../ already escaped before this change, so ~/ reaches nothing they couldn't already. Package location values remain operator/config-supplied input under the same trust model as before.

[kylenesbit]

Reviewed the full diff and verified the behaviour independently rather than from the description alone. Approving — this is exactly the fix the outside-the-repo package flow needed. Two small items below are worth folding in before merge.

What I verified

• Live smoke test with the built branch: booted the server from an unrelated cwd with --config <elsewhere>/my-data/publisher.config.json, one package at ./sales beside the config and one at ~/pkg-tilde under an overridden HOME. Both mounted and served, and status reported both packages — the exact shape the new docs/configuration.md example promises.
• Tests: the three affected spec files pass locally (157 pass, 0 fail); all 14 CI checks are green, including the Windows matrix and the Docker smoke test.
• Shape: resolution now lives in one exported helper (resolvePackageLocation) used by both call sites that previously kept hand-synced comments pointing at each other, and the bundled-default / config-names-a-directory fallbacks are each pinned by their own test. Containment is unchanged — same conclusion as the peer review above on the security angle.

Two items before merge

1. @jswir's os.homedir() point is valid — please take it. The default parameter runs os.homedir() on every resolution, including plain ./sales and absolute paths. I probed macOS: unset or empty HOME still resolves via the passwd entry, so it's harmless here — but in a distroless container running as an arbitrary UID with no passwd row it throws, and Docker is a first-class deployment path in this repo. Moving the call inside the ~/ branch and guarding an empty result is a five-line change; the nits (negative assertion in the anchoring spec, pinning the bare-~ rejection) are cheap too.
2. @sagarswamirao's ask is still open: PUBLISHER_USE_BUNDLED_DEFAULT is absent from the configuration.md env-var table, and this change makes that knob more load-bearing (it's one of the cases that decides the anchor). One table row closes it.

One judgment call I'd accept, noting it for the record: the behaviour change also applies to locations POSTed at runtime (they anchor at the config dir when a --config is active), and when the change bites an existing setup the symptom is the same quiet environment-skip. The release note covers migration, the old rule was undocumented, and the affected population — --config pointing away from the server root plus relative locations — is exactly who this fix exists for.

[kylenesbit]

Pushed the two review items directly onto the branch so this can merge without another round trip:

• cf0f55a takes @jswir's item 1 and the nits: the home directory is now resolved lazily inside the ~/ branch only, with a named error when it comes back empty — os.homedir() can throw where HOME is unset and the uid has no passwd entry (distroless containers), and ./sales or an absolute location must never fail on that. New specs pin the throw on empty home, that non-tilde locations never consult home at all, and that only the POSIX ~/ prefix expands (bare ~ and ~user/… stay non-local; a note by FilesystemPath records the POSIX-only choice). The anchoring spec also gains the missing negative: a decoy sales package now sits at the server root with different contents, so a silent revert to server-root anchoring mounts the decoy and fails the content assertion. Verified by mutation — reverting the anchor to serverRootPath fails that spec.
• c602450 answers @sagarswamirao: PUBLISHER_USE_BUNDLED_DEFAULT now has a row in the configuration.md env-var table, covering who sets it (the server itself on a zero-flag start), what disables it (--config / --server_root), and the anchoring consequence (bundled-default mode keeps the server root as the anchor).

Locally green on top of the branch: full unit suite (1141 pass, 0 fail), lint, prettier. The concurrent_package integration specs fail on my machine identically with and without these commits (they need GCS access to gs://publisher_test_packages), so that's environmental — CI will confirm.

Full review with the live smoke-test verification is in the approval above. @mlennie once checks are green this is good to merge from my side.