Summary
First-time Sume CLI dogfooding exposed a real schema/docs/skills drift: bundled skills and docs instruct agents to add --agent to paid submit commands, but the current submit commands reject --agent before execution.
This breaks the intended agent workflow for Avatar and Avatar Video creation. The agent correctly follows the skill guidance, attempts a paid submit with --agent --json, gets unknown option '--agent', then has to recover by removing --agent. The failed attempts do not submit paid jobs, but the behavior is confusing and makes the skill guidance unreliable.
Why this matters
SUM-619 skill routing expects agents to rely on local skill exports and sume tools schema rather than guessing. If the exported skills recommend a command shape the CLI rejects, a fresh agent loses trust in the routing layer and may start improvising around safety/redaction guidance.
This is especially important for paid generation commands because the guidance currently says agent-readable submit output should use --agent --json to avoid leaking sensitive URLs or private identifiers. If submit commands intentionally do not support --agent, the skills/docs need to say that explicitly and route agents to jobs watch/result --agent --json for redacted readback. If submit commands should support agent mode, the CLI should accept it consistently.
Dogfood evidence
Thread: Sume CLI first-time skills dogfood (019f35bd-1bc2-75a1-b88b-1143770af841)
Flow observed:
- Fresh agent started unauthenticated and ran no-cost discovery.
- User asked for 3 Korean cosmetics-ad avatars.
- Agent correctly stopped because auth was missing.
- User approved auth; agent ran browser login and confirmed configured auth without printing secrets.
- Agent requested explicit paid approval before generation.
- User approved.
- Agent attempted to submit avatar creates using the skill-recommended
--agent --json pattern.
- The CLI rejected the submit attempts before any paid/API call with
unknown option '--agent'.
- Agent recovered by removing
--agent, submitted 3 jobs, and moved to bounded jobs watch polling.
The recovery is acceptable, but the first failure is a real product/DX issue in the skill/CLI contract.
Local no-cost repro
Audited commit:
sumelabs/cli main @ 90f3eb95a803945de919a970fe3d8b60ac954bb5
CLI version 0.1.1
tools schema marks avatars.create as needing agent redaction and being a paid mutating command:
/Users/huhchaewon/sume/cli/dist/sume --json tools schema avatars.create
Observed schema summary:
{
"name": "avatars.create",
"command": "sume avatars create --confirm-submit --json ...",
"safety": {
"mutating": true,
"paid_generation_call": true,
"requires_agent_redaction": true,
"requires_confirmation": true,
"returns_sensitive_url": true
}
}
But avatars create --help does not include --agent:
/Users/huhchaewon/sume/cli/dist/sume avatars create --help
Parser-only repro, no confirm flag and no API submit:
/Users/huhchaewon/sume/cli/dist/sume --json avatars create \
--type prompt \
--avatar-handle qa_agent_flag_drift \
--prompt "Parser-only QA prompt" \
--agent
Observed:
{
"error": {
"code": "invalid_argument",
"message": "unknown option '--agent'\n(Did you mean --age?)",
"details": {
"commander_code": "commander.unknownOption"
}
}
}
Same parser-only issue exists for Avatar Video submit:
/Users/huhchaewon/sume/cli/dist/sume --json avatar-videos create \
--avatar-handle qa_handle \
--script "Parser-only QA script" \
--agent
Observed:
{
"error": {
"code": "invalid_argument",
"message": "unknown option '--agent'",
"details": {
"commander_code": "commander.unknownOption"
}
}
}
No provider calls or paid jobs were run for these repro commands.
Affected surfaces seen in repo text
Examples currently recommending --agent on submit commands include:
agent/sume-avatar/SKILL.md
sume avatars create ... --confirm-paid --agent --json
- photo/props examples also include
--agent
agent/sume-avatar-video/SKILL.md
sume avatar-videos create ... --confirm-paid --agent --json
agent/sume-min-latency/SKILL.md
- fast Avatar Video loop includes
--agent on create
agent/sume-spend-capped-dogfood/SKILL.md
- paid QA example includes
--agent on create
agent/sume/references/eval-scenarios.md
- selected avatar to video says
sume avatar-videos create --confirm-paid --agent --json
docs/agent-workflows.md
- submit examples include
--agent
- later says: “Use
--agent --json for submit, watch, and result outputs read by agents.”
README.md
- says: “Use
--agent --json on submit commands when an agent is reading the response.”
The generated src/lib/bundled-skills-data.ts also contains these examples, so any source skill edits must regenerate bundled skill data.
Expected behavior
Pick one contract and make all surfaces agree.
Recommended contract:
- Submit commands that can return sensitive URLs/private readback should accept
--agent and return redacted agent-safe output consistently.
sume tools schema <submit-tool> should advertise only flags the CLI actually accepts.
- Bundled skills/docs should use exactly the accepted command shape.
- If submit commands intentionally should not support
--agent, then:
- remove
--agent from submit examples;
- update
requires_agent_redaction/tool descriptions to explain that submit output is not agent-redacted and agents should immediately use jobs watch/result --agent --json for readback;
- make the safety story explicit so agents do not infer the wrong flag.
Given current skill intent, adding --agent support to submit commands seems cleaner than weakening the agent-safe submit story.
Acceptance criteria
sume avatars create --help and sume avatar-videos create --help either include --agent, or every skill/docs example stops using it for submit commands.
- Parser-only commands with
--agent no longer fail with unknown option if --agent is the chosen contract.
sume tools schema avatars.create --json and sume tools schema avatar-videos.create --json accurately reflect the supported CLI flags and redaction behavior.
- Bundled skills source and generated
src/lib/bundled-skills-data.ts are consistent.
- Regression tests cover:
avatars.create submit command examples vs help/schema;
avatar-videos.create submit command examples vs help/schema;
- no old
sume.so surfaces are introduced;
- paid submit commands still require explicit confirmation;
- agent-safe mode does not print full media URLs, signed URLs, API keys, provider payloads, or private identifiers.
- First-time dogfood flow can follow exported skills without hitting an unknown option before paid submit.
Validation used for this issue
- Read-only local CLI schema/help inspection.
- Parser-only repro commands that fail before API submission.
- No paid/provider calls for the repro.
- No secrets printed.
Summary
First-time Sume CLI dogfooding exposed a real schema/docs/skills drift: bundled skills and docs instruct agents to add
--agentto paid submit commands, but the current submit commands reject--agentbefore execution.This breaks the intended agent workflow for Avatar and Avatar Video creation. The agent correctly follows the skill guidance, attempts a paid submit with
--agent --json, getsunknown option '--agent', then has to recover by removing--agent. The failed attempts do not submit paid jobs, but the behavior is confusing and makes the skill guidance unreliable.Why this matters
SUM-619 skill routing expects agents to rely on local skill exports and
sume tools schemarather than guessing. If the exported skills recommend a command shape the CLI rejects, a fresh agent loses trust in the routing layer and may start improvising around safety/redaction guidance.This is especially important for paid generation commands because the guidance currently says agent-readable submit output should use
--agent --jsonto avoid leaking sensitive URLs or private identifiers. If submit commands intentionally do not support--agent, the skills/docs need to say that explicitly and route agents tojobs watch/result --agent --jsonfor redacted readback. If submit commands should support agent mode, the CLI should accept it consistently.Dogfood evidence
Thread:
Sume CLI first-time skills dogfood(019f35bd-1bc2-75a1-b88b-1143770af841)Flow observed:
--agent --jsonpattern.unknown option '--agent'.--agent, submitted 3 jobs, and moved to boundedjobs watchpolling.The recovery is acceptable, but the first failure is a real product/DX issue in the skill/CLI contract.
Local no-cost repro
Audited commit:
tools schemamarksavatars.createas needing agent redaction and being a paid mutating command:Observed schema summary:
{ "name": "avatars.create", "command": "sume avatars create --confirm-submit --json ...", "safety": { "mutating": true, "paid_generation_call": true, "requires_agent_redaction": true, "requires_confirmation": true, "returns_sensitive_url": true } }But
avatars create --helpdoes not include--agent:Parser-only repro, no confirm flag and no API submit:
/Users/huhchaewon/sume/cli/dist/sume --json avatars create \ --type prompt \ --avatar-handle qa_agent_flag_drift \ --prompt "Parser-only QA prompt" \ --agentObserved:
{ "error": { "code": "invalid_argument", "message": "unknown option '--agent'\n(Did you mean --age?)", "details": { "commander_code": "commander.unknownOption" } } }Same parser-only issue exists for Avatar Video submit:
/Users/huhchaewon/sume/cli/dist/sume --json avatar-videos create \ --avatar-handle qa_handle \ --script "Parser-only QA script" \ --agentObserved:
{ "error": { "code": "invalid_argument", "message": "unknown option '--agent'", "details": { "commander_code": "commander.unknownOption" } } }No provider calls or paid jobs were run for these repro commands.
Affected surfaces seen in repo text
Examples currently recommending
--agenton submit commands include:agent/sume-avatar/SKILL.mdsume avatars create ... --confirm-paid --agent --json--agentagent/sume-avatar-video/SKILL.mdsume avatar-videos create ... --confirm-paid --agent --jsonagent/sume-min-latency/SKILL.md--agenton createagent/sume-spend-capped-dogfood/SKILL.md--agenton createagent/sume/references/eval-scenarios.mdsume avatar-videos create --confirm-paid --agent --jsondocs/agent-workflows.md--agent--agent --jsonfor submit, watch, and result outputs read by agents.”README.md--agent --jsonon submit commands when an agent is reading the response.”The generated
src/lib/bundled-skills-data.tsalso contains these examples, so any source skill edits must regenerate bundled skill data.Expected behavior
Pick one contract and make all surfaces agree.
Recommended contract:
--agentand return redacted agent-safe output consistently.sume tools schema <submit-tool>should advertise only flags the CLI actually accepts.--agent, then:--agentfrom submit examples;requires_agent_redaction/tool descriptions to explain that submit output is not agent-redacted and agents should immediately usejobs watch/result --agent --jsonfor readback;Given current skill intent, adding
--agentsupport to submit commands seems cleaner than weakening the agent-safe submit story.Acceptance criteria
sume avatars create --helpandsume avatar-videos create --helpeither include--agent, or every skill/docs example stops using it for submit commands.--agentno longer fail withunknown optionif--agentis the chosen contract.sume tools schema avatars.create --jsonandsume tools schema avatar-videos.create --jsonaccurately reflect the supported CLI flags and redaction behavior.src/lib/bundled-skills-data.tsare consistent.avatars.createsubmit command examples vs help/schema;avatar-videos.createsubmit command examples vs help/schema;sume.sosurfaces are introduced;Validation used for this issue