Skip to content

Detected labels contain raw GroundingDINO tokenizer artifacts #7

Description

@adharshvenkat

Problem

SemanticDetectorNode._detect (src/natural_nav/natural_nav/semantic_detector.py:256) takes GroundingDINO's predict() phrase output via phrase.strip().lower() with no validation against the known prompt vocabulary.

Evidence

Confirmed on GPU (full 2Hz run): garbage labels appeared in /natural_nav/semantic_map alongside real ones:

  • ##let door (BERT wordpiece continuation: pal + ##let = "pallet")
  • box cardboard box, pallet cardboard box table, pallet chair (multi-category merges across the " . "-separated prompt list)

Impact

  • Pollutes the semantic map with labels the LLM planner can never usefully target.
  • Confusing in any visualization or debug view of the map.

Proposed fix

DEFAULT_PROMPTS is a small, fixed, known vocabulary. Validate each returned phrase against it (exact match, or best-substring match) and drop/snap anything that doesn't correspond to a real prompt, instead of trusting GroundingDINO's raw phrase extraction.

Acceptance criteria

  • No label outside DEFAULT_PROMPTS (or a configured prompt list) ever reaches SemanticMap.update().
  • Unit test covering wordpiece-fragment and multi-category-merge inputs against _detect's output filtering.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions