Semantic food normalization before provider retrieval — plan and contract

[maxjustships]

Dogfood gap

The resolver is conservative and correctly rejects unsafe provider matches, but it currently receives lexical food text rather than a structured semantic intent. That makes common natural-language food descriptions fail before provider ranking can help.

Literal failure:

"курица сырокопченая 50 г"

• Russian direct query: no USDA resolution.
• English literal smoked chicken: candidate was mixed chicken/beef/pork smoked sausage at confidence 0.77; correctly rejected.
• Agent semantic reformulation chicken breast roasted safely found USDA chicken breast, roll, oven-roasted as a transparent generic proxy.

This should not require agents to memorize hidden English strings, and must not be solved by growing repo aliases/food data.

Objective

Create an agent-first semantic normalization / substitution-planning boundary upstream of deterministic provider retrieval.

The LLM/agent may infer a structured intent from user language; nomnom remains authoritative for provider lookup, confidence gating, provenance and nutrient computation.

Constraints

• No food aliases, food records, translations, synonym corpus, weights or cache in Git production data.
• No arbitrary first search result.
• Brand/SKU queries remain exact-only / photo-barcode-first.
• Unbranded queries may receive generic proxies only with explicit assumptions.
• CLI must not embed a hosted LLM or calculate nutrition itself.

Proposed contract to investigate

Agent constructs a bounded structured intent/candidate payload, e.g.:

{
  "original": "сырокопчёная курица",
  "brand_intent": false,
  "food_type": "deli_poultry",
  "animal": "chicken",
  "preparation": ["smoked", "cured"],
  "candidates": [
    {"query": "smoked chicken breast sliced", "relation": "closest"},
    {"query": "chicken pastrami", "relation": "closest"},
    {"query": "chicken breast roasted", "relation": "generic_fallback"}
  ]
}

nomnom validates this contract, queries/ranks candidates deterministically and records original phrase, normalized candidate, relation, confidence and proxy assumptions. A fallback that materially changes preparation/species/product type must be visible and may require ask policy rather than auto-log.

Research / plan needed before broad implementation

1. Trace existing parser → FoodRepository → cache/OFF/USDA resolution and identify exactly where semantic intent must enter without bypassing safeguards.
2. Classify known failures from dogfood (Russian morphology, cooking/preservation state, product form, generic vs branded query) and generate a small test-only benchmark fixture.
3. Propose CLI contract, storage/provenance fields, user-facing output and policy levels (strict | ask | generic_proxy).
4. Define a staged delivery: observability/resolve-dry-run → candidate contract → ranking integration → evaluation/regressions.
5. State which changes are implementation vs agent workflow guidance; do not prematurely write a giant ontology.

Acceptance examples

• сырокопчёная курица leads to a safe transparent generic proxy or a structured ask; never literal provider mismatch / random mixed-meat sausage.
• куриная пастрома uses the same product-form route.
• Existing direct exact barcode/brand flows unchanged.
• All provenance distinguishes original user phrase from normalized retrieval query.

[maxjustships]

Decision-grade plan (investigation complete)

Root cause

parser.py passes literal food phrase into FoodRepository.resolve; foods.py performs only case/whitespace/limited morphology normalization, then OFF/USDA rank lexical tokens. курица сырокопченая therefore cannot express the intended unbranded deli-poultry/smoked semantic shape. Current safety gates correctly reject the mixed-meat sausage candidate; they must not be weakened.

Recommended delivery

1. Phase A — dry-run contract (first narrow PR): add read-only nomnom resolve --food ... --intent-json ... --json, backed by plan_resolution(..., persist=False). It must touch neither cache, logs, nor aliases. Add a small mocked test-only benchmark.
2. Phase B — controlled log application: log --parse --resolution-intents JSON with exact item mapping and per-log resolution_provenance stored inside items_json (no schema migration). Default semantic policy is ask; generic_proxy allows explicit opt-in auto-application.
3. Phase C — evaluation/release: CI benchmark for Russian/preservation/form examples, exact brand/barcode rejection, no-write dry run, and legacy-log compatibility.

v1 semantic contract

Agent supplies original phrase plus at most 3 ordered candidates (lexical_equivalent | same_form | generic_fallback) and an assumption for fallback. The CLI validates raw query first, validates candidates with existing provider/identity checks, and can only produce generic_proxy (never exact_product) from semantic candidates.

Policy

• brand/SKU/barcode: exact capture only;
• raw safe generic: existing policy;
• same form semantic proxy: ask by default;
• material form/preservation loss (smoked deli poultry -> roasted chicken): always visible fallback, ask by default;
• unsafe/no candidate: photo/barcode or concise ask.

Anti-goals

No static translations/synonyms/food/weight data in Git; no embedded LLM; no first-search-result fallback; no original phrases stored as cache identity.

Recommended next action: implement Phase A only before allowing semantic candidates to affect logs.