Skip to content

Refactor hybrid search pattern into oidm-common for reuse across packages #33

Description

@talkasab

Problem

Both findingmodel and anatomic-locations packages implement similar hybridsearch patterns with significant code duplication:

  1. Exact match lookup (by ID, name, slug, synonym)
  2. FTS search using DuckDB's BM25
  3. Semantic search using vector embeddings
  4. RRF fusion to combine FTS + semantic scores
  5. Result ranking and deduplication

The search() and search_batch() methods in findingmodel/index.py duplicate this pattern, and anatomic-locations has its own similar implementation. This leads to:

  • Duplicated logic that's hard to maintain
  • Inconsistent behavior between packages
  • Higher complexity scores (C901) from inlined fusion logic

Proposed Solution

Extract a reusable HybridSearcher or hybrid_search() utility into oidm-common that encapsulates:

# oidm-common/src/oidm_common/search.py
class HybridSearcher(Generic[T]):
    """Reusable hybrid search with RRF fusion."""
    def __init__(
        self,
        exact_fn: Callable[[str], list[T]],
        fts_fn: Callable[[str, int], list[tuple[T, float]]],
        semantic_fn: Callable[[list[float], int], list[tuple[T, float]]],
        id_extractor: Callable[[T], str],
    ): ...
    def search(
        self,
        query: str,
        embedding: list[float] | None,
        limit: int,
    ) -> list[T]: ...

Packages to Update

  1. oidm-common: Add HybridSearcher or equivalent
  2. findingmodel: Refactor DuckDBIndex.search() and search_batch() to use it
  3. anatomic-locations: Refactor AnatomicIndex search methods to use it

Acceptance Criteria

  • oidm-common exports reusable hybrid search utility
  • findingmodel search methods use the shared utility
  • anatomic-locations search methods use the shared utility
  • No C901 complexity suppressions needed
  • All existing tests pass
  • Search behavior unchanged (verify with existing test coverage)

References

  • packages/findingmodel/src/findingmodel/index.py - search(), search_batch()
  • packages/anatomic-locations/src/anatomic_locations/index.py - search implementation
  • packages/oidm-common/src/oidm_common/duckdb.py - existing rrf_fusion(),normalize_scores()

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions