Skip to content

Anatomic location search returns irrelevant results for multi-word queries #38

Description

@talkasab

Summary

The anatomic location hybrid search returns badly wrong results for straightforward multi-word queries:

  • "cardiac chamber" → returns eye chambers instead of heart chambers
  • "cardiac muscle" → returns shoulder muscles instead of myocardium

The semantic (vector) component appears to be matching on generic terms ("chamber", "muscle") while ignoring the qualifying anatomic context ("cardiac").

Investigation areas

  1. Embedding quality — are the stored embeddings capturing anatomic context, or just individual word similarity? Check what vectors are stored for cardiac structures vs. eye/shoulder structures.
  2. FTS vs. semantic weighting — the hybrid search blends FTS (BM25) and vector scores. If FTS correctly ranks cardiac results higher but the semantic score overwhelms it, the weights need adjustment.
  3. Query preprocessing — is the full query string being embedded, or is it being tokenized/split in a way that loses the multi-word context?
  4. Index coverage — verify the anatomic location database actually contains the expected cardiac entries (heart chambers, myocardium, etc.) with appropriate synonyms.
  5. Eval suiteevals/anatomic_search.py has 10 test cases. Check whether these cases cover multi-word anatomic queries and whether scores are acceptable. Add failing cases for the examples above.

Reproduction

from anatomic_locations import AnatomicLocationIndex

async with AnatomicLocationIndex() as idx:
    results = await idx.search("cardiac chamber")
    # Expected: heart chambers (left ventricle, right atrium, etc.)
    # Actual: eye chambers (anterior chamber, posterior chamber, etc.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions