This repo is the source of a Claude Code plugin. A session here is for developing the plugin, NOT for using it. You are editing skill source; do not try to run the oryxflow skill against this repo (there is no data pipeline here).
Read docs/design/architecture.md first. It is the map of this repo - the
component table, the control/data flows, the invariants, and a "to change X, edit
Y" playbook. It exists so you can edit the right file without exploring.
For the meaning of oryxflow itself (tasks, flows, the data-project conventions),
read skills/oryxflow/SKILL.md - but treat it as the artifact you maintain, not
as instructions for this session.
oryxflow is NOT based on luigi (it once was; now decoupled - base class
oryxflow.core.Task). To explain oryxflow internals (identity, caching, DAG), inspect the installed class (cls.__mro__), neverluigi.*- a leftoverimport luigiproves nothing. Common stale-prior trap.
Use paths relative to the repo root; don't hardcode absolute machine paths (the repo is cloned to different locations by different people). When a tool needs an absolute path, copy the root from the session's Primary working directory verbatim - don't retype it from memory (that is how a path segment gets dropped and lands outside the repo).
A single-skill Claude Code plugin. It ships the oryxflow skill, which activates
when a user works in a oryxflow data-science project. This repo is also its own
marketplace, so it can be installed directly from git or a local path.
.claude-plugin/
plugin.json # manifest (name, version, description, author)
marketplace.json # makes this repo installable as its own marketplace
commands/
init-project.md # /oryxflow:init-project - scaffold a new project into cwd
init-gitlfs.md # /oryxflow:init-gitlfs - put data/ under Git LFS
update-project.md # /oryxflow:update-project - update old project floor to latest
check-standards.md # /oryxflow:check-standards - check names, style, docstrings
migrate.md # /oryxflow:migrate - restructure a messy project into a pipeline
skills/
oryxflow/
SKILL.md # skill entry point - ESSENTIALS only, always in context
reference.md # full reference - loaded ON DEMAND, not by default
ml-patterns.md # ML pipeline task templates - loaded ON DEMAND
resources/
template-minimal/ # the project scaffold init-project copies into a new project
template-prod/ # graduated run-tier add-ons (run_prod.py, run_eda.py); copied by hand, NOT by init
docs/
CHANGELOG.md # user-facing change history
design/
architecture.md # system map + where-to-change playbook (read this first)
design-notes.md # WHY each non-obvious decision was made
README.md # install + quickstart for plugin users
- ASCII only. No emojis/unicode/smart quotes - the skill itself mandates this for Windows safety, and the skill files practice it. Keep it that way.
- Wrap prose at ~78 columns.
- Two-tier content split is load-bearing: keep
SKILL.mdto the essentials an agent needs every time; push depth, tables, and long examples intoreference.md(general) orml-patterns.md(ML), each pointed to fromSKILL.md. Do not letSKILL.mdbloat - it costs context on every activation. Seedocs/design/design-notes.mdfor the reasoning. - Be token-efficient - you MUST. Every word in always-loaded text (
SKILL.md) is re-paid on every activation. Write like an expert prompt engineer: say it once, in the fewest words that still land. The full "why" goes indesign-notes.md(never loaded at runtime). A rule that gets longer under edit is a smell - tighten it or push the depth down a tier. (E.g. a good rule is usually just imperative + the rationalization it blocks + one reason - but treat that as a sample of the style, not a required template.) - Address the agent directly (second person). Skill files are consumed by the coding agent, not read by a human. Never refer to "the agent" / "the AI agent" in the third person, or frame the reader as a human collaborator ("we"/"our", "you and me") - that makes the file read as being about the agent rather than for it. Keep the rationale (it helps the agent decide edge cases); just point it at "you".
- When you change skill behavior, update
docs/design/design-notes.mdif the rationale changed, anddocs/CHANGELOG.mdalways.
This repo is public and git history is forever. Applies to every committed
file - docs, skill text, code, comments, and commit messages - and especially
docs/plans/ and docs/blog/, where real-world context creeps in naturally (a
plan grounded in real projects wants to name them; a blog post wants a real
anecdote - don't). Sanitize as you WRITE, not later: scrubbing after the fact
needs a history rewrite and a force-push, which breaks every existing clone.
Never commit, with the neutral form to use instead:
| Don't | Do |
|---|---|
Absolute local paths - D:\Users\<name>\dev\thing, /home/<name>/... |
/path/to/<repo>, a repo-relative path, or just the repo name |
| An out-of-repo file's location | "tracked in an external note, <filename>.md" |
Client / customer / employer names, incl. as a directory or project slug (acme-re, Acme-cx) |
"a downstream consumer project", "the consumer project" |
A client's data vendors, systems, or task/class names that embed them (ReturnsAcmeVendorAll) |
A generic stand-in (ReturnsBenchmarkAll, "the upstream vendor API") |
| Other private project codenames - a sibling private repo, an internal tool | "a private sibling project", or omit |
| Anything credential-shaped - tokens, keys, bucket names, internal hostnames/URLs | A placeholder (<PYPI_TOKEN>), or reference where it is stored |
A plan or example must stay executable after sanitizing: keep the shape of the real case (task names, param sets, the failure it exhibits) and drop only the identity. If sanitizing would gut the point, it belongs in an external note, not in this repo.
Known carve-out: README.md uses a real local path in its two
--plugin-dir / marketplace add examples, marked # e.g.. That is
deliberate - leave it, and do not treat it as licence for new ones.
claude --plugin-dir /path/to/oryxflow-claude-plugin # load without installing
/reload-plugins # after each edit
/plugin validate . # check both manifests
git config core.hooksPath .githooks # ONE-TIME: enable repo hooks
/plugin validate . (or claude plugin validate .) checks plugin.json and
marketplace.json. Run it before committing manifest changes.
git config core.hooksPath .githooks (one-time per clone) enables the versioned
pre-commit hook in .githooks/. It enforces three things: (1) the scaffold floor
baseline matches in its two homes (the template stamp and SKILL.md); (2) the
compatibility floor (oryxflow >= X) matches across SKILL.md and
docs/CHANGELOG.md; (3) every leading BREAKING: changelog bullet carries a
Migration: clause - see the Release section and the architecture playbook. Each
agreement check only guards that the two values AGREE; deciding a floor change is
migration-worthy (and bumping the value) is still yours.
Publishing is pull-based and VERSION-GATED. .claude-plugin/plugin.json sets an
explicit version, so a consumer running /plugin marketplace update oryxflow
only receives a change when that string CHANGES; same version + new commits =
they keep the cached copy. The version bump IS the publish - there is no separate
publish step, and commits between bumps are invisible to consumers. (Omitting
version would instead make every commit SHA a new version - we do NOT want that
churn, hence the explicit string.) So work in the open on main, and gate what
consumers see behind the bump.
Add bullets under the ## [Unreleased] heading at the top of docs/CHANGELOG.md
(### Added / ### Changed / ### Removed). Do NOT touch plugin.json - it
stays at the last released version, so nothing ships while you work. Commit as
often as you like; across as many days as you like. Nothing here reaches a
consumer until you cut a release.
When you want consumers to pull the accumulated [Unreleased] changes:
- Version + changelog. Rename
## [Unreleased]->## [YY.M.D[.N]] - YYYY-MM-DDwith today's date (YY.M.D, no zero-padding; append.Nfor a second release the same day). Setplugin.jsonversionto the SAME string. INVARIANT: the top DATED changelog heading andplugin.jsonversionmust be equal. Then add a fresh empty## [Unreleased]back on top for the next cycle. - Floor baseline - decide, then stamp (usually SKIP). The floor baseline is a
SEPARATE number from the plugin version: it is the plugin version AS OF the last
MIGRATION-WORTHY scaffold change (currently
26.7.28, at or below the plugin version). Bump it ONLY if a change in this release alters the scaffold in a way existing projects should adopt (resources/template-minimal/**, the projectCLAUDE.md, wiring an old project lacks). Skill / docs / reference-only changes do NOT touch it. To bump: set BOTH homes to this release's version -resources/template-minimal/CLAUDE.md(<!-- oryxflow-floor: X -->) andskills/oryxflow/SKILL.md(thefloor baseline **X**value). The.githooks/pre-commithook blocks the commit if the two disagree, but it cannot know whether you SHOULD have bumped - that judgment is yours. - Validate + ship. Run
/plugin validate .if manifests changed, commit, push. Consumers get it via/plugin marketplace update oryxflow.
The skill also still exists at ~/.claude/skills/oryxflow (the pre-plugin copy).
This repo is becoming canonical. Avoid editing both - once the plugin is
verified, delete the ~/.claude/skills/oryxflow copy so they cannot drift.
resources/template-minimal/ is the project scaffold that init-project copies
into a new project. Edit it directly here - this repo is canonical for it. It
ships the project wiring (tasks.py, flow.py, run.py, cfg.py,
flow_params.py, visualize.py, viz-template.ipynb), the project CLAUDE.md,
docs/oryxflow-data.md, .creds.yaml.example, an eda/ package root, and the
data/, reports/, and reports/render/ dirs. Those three dirs are kept by a
.gitkeep that must be FORCE-added (git add -f): they match the .gitignore
.* dotfile rule, so a plain git add skips them. tasks.py / flow_params.py
ship the intentional PLACEHOLDER SCAFFOLD markers (leave them). A scaffold change
ships like any other at the next release bump (see "Release"); if it is one
existing projects should adopt, also bump the floor baseline in that same release.
resources/template-prod/ is a SEPARATE, graduated add-on scaffold - run_prod.py
(prod tier) and run_eda.py (comparison tier). It is NOT part of init-project:
a project copies these by hand once a distinct run lifecycle appears (the "Run
tiers by lifecycle" guidance in conventions.md). Canonical here; edit directly.
It is NOT a floor file (nothing auto-reconciles it), so a change ships with just a
changelog bullet - no floor-baseline bump. These files name project-specific tasks
they cannot resolve, so they carry PLACEHOLDER SCAFFOLD markers (leave them).
- Commit messages: imperative summary line; end with the Co-Authored-By trailer.
- Default branch is
main(tracksorigin/mainon GitLab). Do not usemaster. - Commit-message tool gotcha: PowerShell here-string syntax (
@'...'@) is NOT a here-string in the Bash tool - the@chars get passed literally into the message (e.g. a subject like@ Rename ...). For multi-paragraph messages, use repeated-mflags with normal quoted strings in the Bash tool, or use the@'...'@here-string only in the PowerShell tool. Don't mix the two.