Skip to content

feat: ModelManager accessor for bidirectional engines (2/N) #9174

@GuanLuo

Description

@GuanLuo

Replace the process-scoped OnceLock<EchoBidirectionalEngine> registry in lib/llm/src/http/service/asr.rs with a ModelManager accessor keyed on model_name, parallel to chat / completions / embeddings.

When this lands, the install_engine / install_echo_engine install-time API retires and handle_socket calls state.manager().get_asr_engine(model_name) instead.

Prerequisite: a bidirectional-engine accessor must exist on ModelManager.

Metadata

Metadata

Assignees

Labels

dynamo-llmRelates to dynamo-llm componentenhancementNew feature or requestlanguage::rustIssues/PRs that reference Rust code

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions