Experimental: AI Skill for building plugins#57
Conversation
correct property name
This is the only field that actually supports this property currently, the other fields have no UI to support encrypting values
The skill previously used a placeholder @yourhandle in the metadata.json template without prompting Claude to collect this from the user, meaning plugins could be created with an unresolved placeholder. An explicit AskUserQuestion instruction now ensures the author name is always gathered at the point it is needed. The author type silently defaults to "community" to keep the interaction minimal.
The build-plugin skill mentioned that the documentation link in metadata.json should point to docs/README.md, but never explicitly instructed Claude to create the file. This made it easy to skip. The fix adds docs/README.md to the Phase 3 checklist entry, the folder structure diagram, and a new section explaining what the file should contain (prerequisites, config field descriptions, object types, known limitations). The section also explains why it matters — the file is surfaced directly in-product when users add the plugin, so it needs to work as a self-contained setup guide without relying on external links.
Was using incorrect property name
Otherwise Claude can name each perspective the same i.e. Overview, which show next to each other in the UI
Co-authored-by: Dave Clarke <[email protected]>
Claude skill tweaks
The build-plugin SKILL.md was a monolithic ~850-line file loaded in full on every invocation, including auth patterns, field types, pagination options, and viz configs irrelevant to the current phase. This refactors it into a 113-line orchestrator that references six topic-specific files under references/; Claude reads each file only when it reaches the corresponding build phase. A standalone deploy-plugin skill is extracted so validate/deploy/versioning can be invoked independently of the full build flow. Follows Anthropic's recommended progressive disclosure pattern for large skills, which advises keeping SKILL.md under 500 lines. Co-Authored-By: Claude Sonnet 4.6 (1M context) <[email protected]>
The data-streams reference steered the skill toward writing a postRequestScript for almost every stream, even when the response only needed per-column tweaks. In practice most "scripts" were just `data.items.map(...)` reshapes that `pathToData` plus column-level expressions handle without a separate JS file. The Expressions section now states up front that any string leaf under `config` is run through the expression resolver, not just the fields previously shown in examples. The former "Computed columns" subsection is rewritten as "Column expressions" covering both `valueExpression` (computes the underlying value; with `computed: true` it materialises a brand-new column, without it it overrides an existing one) and `formatExpression` (changes only the displayed string, leaves the raw value intact for sort and aggregation). The post-request scripts section gains bullets pointing per-row value transforms, display-only formatting, and simple map-and-rename scripts at the expression mechanisms instead. Also captures a real gotcha verified in the WebAPI handler: `pathToData` is silently ignored when `postRequestScript` is set (the script receives the raw response body as `data`), so the two should not be combined. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
The build-plugin skill's post-request script guidance was correct
but easy to skim past. In practice, LLM invocations of the skill
keep imitating shipped plugins (UniFi, Spotify, OpenSearch) that
use `data.items.map(...)` reshape scripts where pathToData,
expandInnerObjects, and computed valueExpression would suffice.
The data-streams.md "Post-request scripts" section now leads with
an explicit "don't imitate existing plugins" warning, followed by
a "Default: no script" decision table covering the eight common
patterns that don't justify a script: path navigation, primitive
parsing, single-level flattening, constant columns, derived
columns, null coercion, array counts, and renames. The "Use
scripts for" list is tightened — "flattening" is qualified as
deeply nested or array-into-rows, since one-level nesting is
handled by expandInnerObjects.
index-defs.md gains a worked no-script example for the common
paged list shape ({ items: [{ id, attributes: { ... } }] }),
demonstrating sourceId/sourceType via computed valueExpression
and showing how the index definition references dot-notation
column names. The original script example is reframed as the
exception (nested array into rows) rather than the default.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Claude is seeing this property as being required by default due to the examples, it should be added only as needed
Could lead Claude to hallucinate that $schema is a valid property, which it currently is not and will fail validation
Load build-plugin reference content on demand
🧩 Plugin PR Summaryℹ️ No plugins were modified in this PR. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| root = true | ||
|
|
||
| [*] | ||
| charset = utf-8 | ||
| end_of_line = lf | ||
| insert_final_newline = true | ||
| trim_trailing_whitespace = true | ||
|
|
||
| [*.{md,json,js,jsx,ts,tsx}] | ||
| indent_style = space | ||
| indent_size = 4 | ||
|
|
||
| [*.json] | ||
| insert_final_newline = false | ||
|
|
||
| [*.md] | ||
| trim_trailing_whitespace = false | ||
|
|
||
| [*.{js,jsx,ts,tsx}] | ||
| end_of_line = crlf |
There was a problem hiding this comment.
🔴 This PR adds a new .editorconfig at the repo root, which the repo review guidelines explicitly disallow ("Do not introduce formatting tools or config files unless explicitly requested") — the PR description is about the experimental build-plugin skill and never asks for editor config. Beyond the policy point, the [*.{js,jsx,ts,tsx}] block at lines 19-20 sets end_of_line = crlf, which contradicts the [*] end_of_line = lf block above and all 36 existing JS files in the repo (verified LF). Recommend removing the file; at minimum, drop the CRLF override so editors honoring editorconfig don't churn every JS file to CRLF on next save.
Extended reasoning...
What the bug is
This PR adds a new top-level .editorconfig (20 lines, lines 1-20 of the diff). Two distinct problems:
-
Repo policy violation. The repository's review guidelines explicitly state: "Do not introduce formatting tools or config files unless explicitly requested." The PR description is dedicated to the experimental
build-pluginAI skill and makes no mention of editor configuration. The.editorconfigis not required by, related to, or referenced from the skill itself (which lives under.claude/skills/build-plugin/). Reviewers are expected to flag this kind of stowaway config. -
Internal CRLF/LF contradiction that would cause churn. Lines 3-7 declare a global
[*]section withend_of_line = lf. Lines 19-20 then declare a[*.{js,jsx,ts,tsx}]section withend_of_line = crlf. EditorConfig's precedence rule is "more specific section wins," so editors honoring.editorconfig(VS Code, IntelliJ, etc.) will treat JS/TS files as CRLF, even though the same file just said LF a few lines above.
Why this matters in practice
Verifiers independently confirmed via find ... -name "*.js" | xargs file that all 36 existing JS files in the repository use LF line endings — zero CRLF. There is no .gitattributes forcing CRLF. So the moment anyone opens, edits and saves a .js/.jsx/.ts/.tsx file in an editorconfig-aware editor (which is most modern editors), that editor will silently rewrite every line ending to CRLF. The resulting diff will be 100% churn — no logical change, just \r\n on every line — and will land on whatever real edit the developer was making.
Step-by-step proof
- Take any existing JS file, e.g.
.claude/skills/build-plugin/scripts/gen-uuids.js(currently LF — verified by inspecting the diff hunk, which contains no^Mmarkers, and by verifiers runningfileacross all 36 JS files). - Open it in VS Code with the EditorConfig extension enabled (or IntelliJ, which honors editorconfig natively).
- EditorConfig walks up the tree, finds
/.editorconfig, and resolves rules for the file path. Two sections match:[*](lf) and[*.{js,jsx,ts,tsx}](crlf). The more specific section wins →crlf. - Make any trivial edit (add a space, delete it) and save.
- The editor rewrites every line of the file with
\r\n.git diffnow shows every line changed. - Multiply across all 36 JS files as developers touch them over time.
Why existing code doesn't prevent it
There is no .gitattributes file in the repo that would override editorconfig at the git level (e.g. a * text=auto eol=lf line would normalize on commit). Without it, whatever line ending the editor writes is what gets committed.
How to fix
The simplest fix is to remove .editorconfig from this PR — it is unrelated to the build-plugin skill and the PR description doesn't ask for it. If the author wants to keep it, the [*.{js,jsx,ts,tsx}] override at lines 19-20 must be removed (or changed to end_of_line = lf) so it stops contradicting the global [*] block and the actual files in the repo. The indent rules in the other override blocks are fine; only the CRLF line is the problem.
| ``` | ||
| my-plugin/ | ||
| v1/ | ||
| metadata.json | ||
| ui.json | ||
| icon.svg | ||
| custom_types.json | ||
| configValidation.json # preferred: validates config on setup | ||
| docs/ | ||
| README.md # REQUIRED: shown in-product when users add the plugin | ||
| indexDefinitions/ | ||
| default.json | ||
| dataStreams/ | ||
| myStream.json | ||
| scripts/ | ||
| myScript.js | ||
| defaultContent/ | ||
| manifest.json | ||
| scopes.json | ||
| overviewDashboard.dash.json | ||
| deviceDashboard.dash.json # single perspective — no sub-folder needed | ||
| Installations/ # sub-folder only for multiple dashboards of the same type | ||
| manifest.json | ||
| dashboard1.dash.json |
There was a problem hiding this comment.
🔴 The Phase 3 file-structure scaffold in build-plugin/SKILL.md uses lowercase my-plugin/ as the top-level folder, but the same PR's references/metadata.md explicitly says the folder is PascalCase (e.g. MyPlugin), the documentation-link example a few lines below uses plugins/MyPlugin/v1/, and every one of the 19 existing plugin folders in this repo uses PascalCase (DigiCert, GoogleSheets, MetOffice, …). An LLM following the scaffold literally would create the plugin in a lowercase folder that breaks repo convention — change the example to MyPlugin/v1/ (the kebab-case stays only in the metadata.json name field).
Extended reasoning...
The bug
In .claude/skills/build-plugin/SKILL.md, the Phase 3 "File structure" code block starts with:
my-plugin/
v1/
metadata.json
...
This lowercase kebab-case folder name contradicts three load-bearing sources in this same PR and repo:
references/metadata.mdin this PR — under the field notes it states verbatim: "name: lowercase kebab-case (e.g.my-plugin). Folder uses PascalCase (e.g.MyPlugin) — these are separate things." The skill is contradicting its own reference doc.- The same SKILL.md file, ~20 lines below the scaffold — the documentation-link example is
https://github.com/squaredup/plugins/blob/main/plugins/MyPlugin/v1/docs/README.md. So PascalCase is already used as the intended path inside SKILL.md itself. - Existing repo convention — every plugin folder under
plugins/is PascalCase: DattoRMM, DigiCert, FantasyPremierLeague, GoogleSheets, MetOffice, OpenSearch, Phare, Postcoder, RDAP, RSS, Rootly, SendGrid, Snowflake, Spotify, Steam, SumoLogic, TransportForLondon, UniFi, UptimeRobot. There is no precedent for a lowercase folder.
How it manifests
The build-plugin skill is intended to be invoked by an AI agent that will scaffold files literally from the example. Phase 3 is the scaffolding phase; the agent reads the file-tree block and creates directories/files matching it. Because the example shows my-plugin/v1/..., the agent will run something like mkdir -p my-plugin/v1/dataStreams and write files there. By the time Phase 4 instructs the agent to read metadata.md (which contains the PascalCase rule), the folder has already been created with the wrong case — renaming a directory mid-build is a friction point the agent may simply skip, especially because the lowercase name field in metadata.json does match the folder name and looks superficially consistent.
Step-by-step proof
- User runs the skill: "I want to build a plugin for MyService."
- Agent works through the checklist; reaches Phase 3.
- Agent reads the scaffold tree and creates
my-plugin/v1/metadata.json,my-plugin/v1/ui.json, etc. - Agent moves to Phase 4, opens
references/metadata.md, sees the PascalCase rule — but only as a field note aboutnamevs. folder. The agent has already created the folder; no instruction tells it "go back and rename." - PR opens with a new plugin at
plugins/my-plugin/v1/— inconsistent with all 19 existing plugins. Reviewer asks for a rename, which involves rewriting paths in the documentation link insidemetadata.jsontoo.
Why existing code does not prevent it
There is no validator step that checks folder casing — squaredup validate validates JSON contents, not directory names. The PascalCase rule exists only in prose inside metadata.md, which is read after the folder is created. Without correcting the scaffold itself, the rule is documented but unenforced at the moment it matters.
Impact
Low severity (it is a docs/instructional bug, easily fixed by a reviewer), but real — every plugin generated by this skill is at risk of being committed with the wrong folder case, requiring a manual fix before merging. Given this is the new official AI-skill workflow, it will be the dominant pattern going forward.
Fix
Change the scaffold tree to:
MyPlugin/
v1/
metadata.json
...
Leave the name field in metadata.json as my-plugin (kebab-case) — those are separate things, exactly as metadata.md states. No other lines in the scaffold need to change.
| } | ||
| ``` | ||
|
|
||
| **Field notes:** |
There was a problem hiding this comment.
🔴 The metadata.json template hardcodes "type": "hybrid" and the field notes state type: always "hybrid" for Web API plugins, but actual repo practice is the opposite: 18 existing plugins use "cloud" and only 1 (Snowflake) uses "hybrid". An LLM following this skill will produce metadata.json with the wrong runtime model for ~95% of generated plugins, mis-classifying cloud-only SaaS integrations as supporting on-prem agent execution. Recommend defaulting to "cloud" and noting "hybrid" is the exception for APIs that can also be self-hosted (Snowflake, on-prem ServiceNow, etc.).
Extended reasoning...
What the bug is
.claude/skills/build-plugin/references/metadata.md is the canonical reference the skill tells an LLM agent to read before writing metadata.json. It does two things that point the agent at the wrong answer:
- The template at the top of the file hardcodes
"type": "hybrid"(line ~20). - The field notes section states (line 49):
type: always "hybrid" for Web API plugins. Options: "hybrid" (cloud or on-prem agent), "cloud", "onprem".
The word "always" in the field notes is load-bearing — it tells the model to never reconsider.
Why the guidance is wrong
Empirical check against the 19 plugins currently in the repo:
type |
Count | Plugins |
|---|---|---|
cloud |
18 | DattoRMM, DigiCert, FantasyPremierLeague, GoogleSheets, MetOffice, OpenSearch, Phare, Postcoder, RDAP, RSS, Rootly, SendGrid, Spotify, Steam, SumoLogic, TransportForLondon, UniFi, UptimeRobot |
hybrid |
1 | Snowflake (chosen because Snowflake can be self-hosted) |
onprem |
0 | — |
Semantically, hybrid advertises that the plugin can run via the SquaredUp cloud OR via an on-prem agent — appropriate only for products that can themselves be self-hosted/on-prem (Snowflake, on-prem ServiceNow, etc.). For SaaS-only APIs — which is what the skill will overwhelmingly be used to wrap, given the example list (Spotify, Rootly, Phare, Steam, …) — the correct value is cloud.
How the bug manifests
Because this is a brand new skill whose entire purpose is to guide automated plugin generation, the wrong default propagates to every plugin built through it. Nothing else in the skill, the validator, or squaredup validate will flag type: hybrid on a SaaS plugin — the field is valid syntactically, just semantically wrong. The result is that generated plugins will incorrectly advertise on-prem agent support they dont have, misleading the SquaredUp deployment runtime and end users who try to deploy them via an on-prem agent.
Step-by-step proof
- User invokes the skill: "I want to build a plugin for Spotify."
- Skill progresses through Phase 1–3, then in Phase 4 the agent reads
references/metadata.md. - Agent encounters the template at line 20 (
"type": "hybrid") and the field notes at line 49 ("always hybrid for Web API plugins"). - Agent writes
plugins/Spotify/v1/metadata.jsonwith"type": "hybrid". - Compare against the existing
plugins/Spotify/v1/metadata.jsonin the repo:"type": "cloud". The skill produces a different, wrong value than humans chose for the same plugin. - Repeat for any SaaS API (Rootly, Phare, Steam, SendGrid, …) — every one produced by the skill will be
hybrid, every one produced by humans iscloud.
How to fix
In .claude/skills/build-plugin/references/metadata.md:
- Change the template
"type": "hybrid"→"type": "cloud". - Change the field-notes bullet to something like:
type: "cloud" for SaaS-only APIs (the common case). Use "hybrid" only when the product can also be self-hosted/on-prem (e.g. Snowflake, on-prem ServiceNow). "onprem" for products that only run on-prem.
That single edit aligns the skill with 18/19 of the existing repo and removes the misleading "always" instruction.
2 participants
📋 Summary
This PR introduces an experimental skill for building LCPs. It's currently under active development so expect further PRs to fix issues and enhance functionality.
To get started:
Check out the repo
Open a Claude session in the repo
Trigger the skill, e.g. I want to build a plugin for xxxxxx
Notes:
It will probably call squaredup validate and squaredup deploy for you - so make sure the CLI is logged in to the right organisation
It's focused on Web API based data streams, not PowerShell or otherwise right now
It will reference things locally like existing plugins
🔍 Scope of change
📚 Checklist