oidm-common 0.2.7 / findingmodel 1.0.4 / oidm-maintenance 0.2.5

oidm-common 0.2.7

Changed

• Embedding cache now uses a SQLite-backed cache engine instead of DuckDB for safer concurrent writes.
• Default cache location moved to a shared oidm-common cache directory used across OIDM apps.

Added

• One-time migration from legacy findingmodel embedding caches into the new shared cache.
• Cache import operations now report merge counts (written/new/updated/skipped).

Fixed

• Automatic legacy diskcache migration now runs only for the default oidm-common runtime cache directory,
  preventing accidental import into custom cache locations.

findingmodel 1.0.4

Added

• Added findingmodel validate PATH [PATH ...] to validate FindingModel JSON files.
• validate accepts files and directories (directories are scanned recursively for *.fm.json).
• Added --reformat to rewrite valid files in-place in canonical JSON format.

oidm-maintenance 0.2.5

Added

• Added embedding cache maintenance commands:
  • oidm-maintain embeddings migrate
  • oidm-maintain embeddings stats
  • oidm-maintain embeddings import-duckdb <source>
  • oidm-maintain embeddings import-cache <source_cache_dir>

Changed

• Embedding migration/import commands now fail if the target cache cannot be opened for writes.
• Embedding import output now shows merge counts (written/new/updated/skipped/total).

oidm-common 0.2.6 / anatomic-locations 0.2.5 / oidm-maintenance 0.2.4

oidm-common 0.2.6

Added

• create_fts_index() now accepts an optional ignore regex for DuckDB FTS tokenization.

anatomic-locations 0.2.5

Changed

• Lowered semantic search minimum similarity threshold from 0.75 to 0.60.

oidm-maintenance 0.2.4

Changed

• Anatomic DB build now preserves alphanumeric FTS tokens (for terms like T12 and C7/T1).
• Updated checked-in anatomic source JSON: notebooks/data/anatomic_locations_noembed.json.

Added

• oidm-maintain anatomic validate CLI command for Pydantic-based source JSON validation with cross-record relationship checks.

oidm-common 0.2.5 / anatomic-locations 0.2.4 / findingmodel 1.0.3 / oidm-maintenance 0.2.3

oidm-common 0.2.5

Added

• _execute_one and _execute_all helpers on ReadOnlyDuckDBIndex — return named dicts via cursor.description, eliminating positional row[N] brittleness across all subclasses

anatomic-locations 0.2.4

Changed

• Rewrote hydration with DuckDB correlated subqueries: codes, synonyms, and references fetched inline in a single query per operation (was 1+3N queries); model_validate replaces 95 lines of manual field extraction; index.py reduced from 1186 to 915 lines
• FTS search now matches synonyms (requires rebuilt database from oidm-maintenance 0.2.3)

Fixed

• _ref() now requires both id and display non-null — previously a NULL display field caused ValidationError during model validation
• Malformed STRUCT entries in containment_children/partof_children are now filtered before model validation, preserving prior null-tolerant behavior

findingmodel 1.0.3

Fixed

• _fetch_index_entry now uses named-dict access via _execute_one — eliminates positional row[N] brittleness

oidm-maintenance 0.2.3

Added

• synonyms_text column added to anatomic_locations schema and FTS index — synonyms are now keyword-searchable (requires database rebuild)
• Roundtrip build→read integration tests using AnatomicLocationIndex

findingmodel 1.0.2 / anatomic-locations 0.2.2 / oidm-common 0.2.3

findingmodel 1.0.2

Added

• Added findingmodel search command for hybrid index search

Changed

• Renamed internal class DuckDBIndex to FindingModelIndex; public API (Index) is unchanged
• Environment variables renamed to FINDINGMODEL_ prefix (FINDINGMODEL_DB_PATH, FINDINGMODEL_REMOTE_DB_URL, etc.); legacy names (DUCKDB_INDEX_PATH, REMOTE_INDEX_DB_*) still accepted

Removed

• Removed findingmodel index stats subcommand; stats is now top-level

anatomic-locations 0.2.2

Added

• get() now accepts descriptions and synonyms in addition to RID identifiers (case-insensitive)
• search_batch() method for searching multiple queries with a single embedding API call

Fixed

• Made sure OPENAI_API_KEY is used for embedding-based search, both from environment and .env file.

oidm-common 0.2.3

Added

• ReadOnlyDuckDBIndex base class providing shared connection lifecycle, auto-open, and context managers for all DuckDB-backed indexes

anatomic-locations 0.2.3 / oidm-common 0.2.4 / oidm-maintenance 0.2.2

oidm-common 0.2.4

Fixed

• generate_embeddings_batch now auto-chunks large batches (2048 per API call) to avoid OpenAI 400 errors on batch sizes >2048

anatomic-locations 0.2.3

Fixed

• FTS search now searches both description and definition columns (was restricted to description only)
• FTS index is now case-insensitive (lower=1), matching findingmodel's configuration
• Search quality: added minimum similarity threshold (0.75) for semantic results and minimum BM25 score (0.5) for FTS-only results, filtering garbage matches

Added

• search() and search_batch() exact-match path now checks synonyms in addition to descriptions
• build_anatomic_test_fixture.py script for rebuilding the test fixture database

Testing

• Added search quality threshold tests and synonym exact-match tests

oidm-maintenance 0.2.2

Fixed

• Laterality determination was inverted: leftRef only → now correctly returns "right" (was "left"); rightRef only → now correctly returns "left" (was "right")
• FTS index creation uses lower=1 (case-insensitive) to match production search behavior

Testing

• Added 6 unit tests for determine_laterality() covering all ref combinations

v0.6.0

Added

• Model Context Protocol (MCP) Server - Expose Finding Model Index search to AI agents:
  • Three tools: hybrid search, model retrieval by ID/name/synonym, index statistics
  • Console script findingmodel-mcp for easy integration with Claude Desktop
  • Complete documentation in docs/mcp_server.md with Claude Desktop config examples
• Tavily API support for finding detail generation with domain filtering
• Anthropic Model Support - Multi-provider AI architecture:
  • Use OpenAI or Anthropic models via MODEL_PROVIDER configuration
  • Tier-based selection ("base", "small", "full") abstracts provider-specific names
  • Optional provider parameter on tool functions for runtime override
• Setup a render_agent_prompt method in tools.prompt_template for using a MD prompt with Pydantic AI agents.

Removed

• No more dependency on instructor library
• Perplexity API integration replaced by Tavily

Changed

• add_details_to_info() now uses Pydantic AI agent with custom Tavily search tool
• Tool functions migrated to tier-based model selection (model_tier parameter)
• get_openai_model() deprecated in favor of get_model(model_tier, provider=None)

Fixed

• Refactored markdown_in tool to use Pydantic AI agent in line with other tools
• Make sure DuckDB extensions are loaded when opening the DuckDB connection

v0.5.0

[0.5.0] - 2025-11-03

Added

• Enhanced Index API Methods - Complete pagination and search capabilities without breaking abstraction:
  • list(limit, offset, order_by, order_dir) - Paginated browsing of all finding models
  • search_by_slug(pattern, match_type, limit, offset) - Pattern-based search with relevance ranking
  • count() and count_search() - Efficient counting for pagination UI
  • get_full(oifm_id) and get_full_batch(oifm_ids) - Retrieve complete FindingModelFull objects
• Manifest-Based Database Downloads - Runtime database version discovery:
  • Databases auto-update from remote manifest.json (no library release needed)
  • Graceful fallback to direct URL/hash for offline scenarios
  • CLI command: db-info to check database versions and status
• Self-Contained Databases - Full JSON storage in DuckDB:
  • Single .duckdb file contains metadata + embeddings + full JSON models
  • Separate finding_model_json table with automatic compression
  • No calls to external sites for full models
• Local ID Generation - Database-backed OIFM/OIFMA ID generation:
  • generate_model_id(source) - Random generation with collision checking
  • generate_attribute_id(model_oifm_id, source) - Attribute ID generation with source inference
  • No network dependency, thread-safe via DuckDB
• Database Path Configuration - Specify database file paths for production/Docker deployments:
  • Set DUCKDB_INDEX_PATH or DUCKDB_ANATOMIC_PATH to use pre-mounted database files
  • Default behavior unchanged (automatic manifest-based downloads)
• Comprehensive Agent Evaluation Suites (2025-10-18 to 2025-11-02) - Five new eval suites for AI agent quality assessment:
  • evals/similar_models.py - Similarity search and duplicate detection (8 cases)
  • evals/ontology_match.py - Multi-backend ontology concept matching (12 cases)
  • evals/anatomic_search.py - Two-agent anatomic location search (10 cases)
  • evals/markdown_in.py - Markdown to finding model parsing (8 cases)
  • evals/finding_description.py - Clinical description quality with LLMJudge (15 cases)
  • All use Pydantic Evals Dataset.evaluate() pattern with focused evaluators
  • Logfire observability via lazy instrumentation pattern
  • Taskfile commands: task evals or task evals:agent_name
  • LLMJudge Evaluator Support (2025-11-02) - Built-in LLM-based quality assessment:
    • Configured for clinical description quality scoring
    • Uses cost-effective gpt-5-nano model
    • Workaround for Pydantic Evals API key bug documented
  • PerformanceEvaluator (2025-10-29) - Reusable evaluator in src/findingmodel/tools/evaluators.py:
    • Configurable time limits for agent performance testing
    • Comprehensive unit tests in test/tools/test_evaluators.py
    • Used across all 5+ eval suites (eliminates ~190 lines duplication)

Changed

• GPT-5 Model Adoption (2025-11-02) - Updated default OpenAI models:
  • openai_default_model: gpt-4o-mini → gpt-5-mini (better capability)
  • openai_default_model_small: gpt-4.1-nano → gpt-5-nano (cost-effective)
  • openai_default_model_full: gpt-5 (unchanged)
• Test Performance Optimization (2025-11-02) - Integration tests 54% faster:
  • Tests explicitly use gpt-4o-mini for speed (278s → 127s total test suite)
  • Production code uses GPT-5 models for capability
  • Clear separation: fast models for CI/CD, capable models for production
• Evaluator Architecture (2025-10-29) - Clean separation of concerns:
  • Inline evaluators in eval scripts (35+ agent-specific evaluators)
  • Reusable evaluators in src/ only when genuinely shared (PerformanceEvaluator)
  • Lazy instrumentation pattern prevents Logfire noise in unit tests
• Anatomic Location Search Enhancement (2025-11-02) - Added model parameter:
  • find_anatomic_locations() accepts optional model parameter
  • Allows per-call model override for performance tuning
• Index schema: Added separate finding_model_json table for blob storage
• Index schema: Added index on slug_name for efficient LIKE queries
• Code organization: Shared helper methods eliminate duplication in list/search/count operations
• Database configuration defaults changed to enable custom paths (previously auto-downloaded only)

Deprecated

• Direct access to Index._ensure_connection() - use new API methods instead (see migration guide)

Removed

• IdManager class - Use Index.add_ids_to_model() and Index.finalize_placeholder_attribute_ids() instead, or the convenience function findingmodel.tools.add_ids_to_model()
• id_manager singleton from findingmodel.tools - Use convenience functions or create Index instances
• IdManager.load_used_ids_from_github() - Index queries database directly
• src/findingmodel/tools/add_ids.py module - All ID generation now handled by Index class
• MongoDB Index backend - DuckDB is now the only Index implementation
• MongoDBIndex class - Use Index (aliased to DuckDBIndex) instead

Fixed

• LLMJudge API Key Configuration (2025-11-02) - Workaround for Pydantic Evals bug:
  • LLMJudge now reads OpenAI API key from environment variable
  • Documented workaround until upstream fix available

v0.4.0

[0.4.0] - 2025-10-20

Added

• DuckDB Index Backend: High-performance local search with HNSW vector indexing, now the default
  • Automatic download of pre-built database files on first use (via Pooch with SHA256 verification)
  • Platform-native file storage using platformdirs (OS-appropriate cache/data directories)
  • Base contributors (4 people, 7 organizations) automatically loaded in new databases
  • Optional remote database URLs configurable via environment variables
  • Utilities for building/updating databases from directories of FM definitions
• Anatomic Locations: Comprehensive support for anatomic location finding and management
  • New anatomic_locations field in FindingModelFull for specifying anatomic index codes
  • Two-agent AI search tool across anatomic_locations, RadLex, and SNOMED CT ontologies
  • CLI commands for building, validating, and viewing statistics on anatomic location databases
  • Demo: notebooks/demo_anatomic_location_search.py
• Ontology Concept Search Tool: High-performance medical concept search
  • Multi-backend support: DuckDB vector search and BioOntology.org REST API (requires BIOONTOLOGY_API_KEY)
  • Protocol-based architecture for pluggable search backends
  • Demo: notebooks/demo_ontology_concept_match.py
• Finding Model Editor Tool: AI-assisted interactive editor with markdown and natural language workflows
• Index API enhancements:
  • get_people() method to retrieve all people from Index (sorted by name)
  • get_organizations() method to retrieve all organizations from Index (sorted by name)
  • Available in both DuckDBIndex and MongoDBIndex implementations
• Documentation: New docs/database-management.md guide for maintainers covering database operations, CLI commands, and write-mode Python API

Changed

• DuckDB is now the default index backend (MongoDB deprecated but still available via [mongodb] extra)
• Database file locations: Files now stored in platform-native directories
  • Auto-download mitigates migration: databases automatically download to new locations on first use
• Test suite reorganization: Reduced from 17 to 12 test files, grouped by feature area
• Documentation restructure: README.md now focuses on user perspective (read-only Index operations), with maintainer/write operations moved to docs/database-management.md

Deprecated

• MongoDB backend: Still functional via pip install findingmodel[mongodb] but no longer the default
  • DuckDB provides better performance and simpler deployment
  • MongoDB configuration commented out in config but available if needed

Removed

• LanceDB dependency: Replaced by DuckDB for better performance and consistency

Migration Notes

Upgrading from v0.3.x:

• For most users: No action required. Databases will auto-download to platform-native locations on first use.
• If using MongoDB: Your MongoDB index will continue to work, but consider migrating to DuckDB for better performance. Use index build CLI to create DuckDB index from your finding model definitions.
• If using custom database paths: Note that default paths have changed to platform-native directories. You can continue using custom paths via CLI --index / --db-path options.

v0.3.2

[v0.3.2] - 2025-08-20

Added

• Added comprehensive test coverage for previously untested functionality:
  • 11 tests for Index.search() functionality including limit, case sensitivity, and multiple terms
  • 7 integration tests for AI tools with @pytest.mark.callout decorator
  • 3 tests for find_similar_models() function including edge cases
  • 15 error handling tests for network failures, MongoDB issues, and invalid data
• Updated test documentation to reflect actual MongoDB implementation of Index
• Fixed all linting issues for clean CI/CD pipeline
• Performance Optimization: Optimized similar_finding_models tool for significantly faster execution:
  • Reduced runtime from 15-20 seconds to 4-9 seconds (up to 75% improvement)
  • Added smart model selection with fallback mechanisms
  • Implemented batch MongoDB queries to reduce round trips
  • Added lightweight term generation with intelligent fallback to full models
• Release Automation: Created comprehensive release automation system:
  • Added scripts/release.py with full release pipeline automation
  • Integrated loguru logging with version-specific log files
  • Added Taskfile commands: task release, task release:check, task release:dry
  • Supports dry-run mode, pre-flight checks, and automatic PyPI publishing

Changed

• Corrected Index documentation in README.md and CLAUDE.md from JSONL to MongoDB implementation
• Fixed find_similar_models() documentation to accurately describe its purpose and signature
• Updated type annotations in test functions to satisfy ruff linting requirements
• Replaced generic Exception catches with specific exception types (JSONDecodeError, ValidationError)

Fixed

• Fixed test validation errors by using proper 3-character source codes (TST, TES, TEX)
• Added missing pytest import in test_tools.py
• Corrected find_similar_models test signature to match actual implementation

v0.3.1

[0.3.1] — 2025–07–06

Fixed

• Dealt with a bug in the setup of OpenAI models by find_similar_models in findingmodel.tools