Package
lexical-graph
Problem statement
The opensearch:// vector store (added in #397) connects to OpenSearch endpoints that don't use AWS authentication — HTTP basic auth from OPENSEARCH_USERNAME/OPENSEARCH_PASSWORD, or no auth. It's been validated against a localhost Docker/Finch container. Two deployment shapes aren't covered yet:
- A managed Amazon OpenSearch Service domain (not Serverless) that authenticates with IAM/SigV4.
- Any auth beyond basic/none (e.g. mutual TLS).
Raised in review of #397 (@acarbonetto): OpenSearch can run well beyond localhost (EC2, on-prem, managed Service), and we should support more of those once the localhost path is in.
Proposed solution
Decouple the two concerns the current single flag ties together:
- Auth mechanism —
is_sigv4_auth selects AWS SigV4 vs basic/no auth in create_os_client / create_os_async_client.
- AOSS document handling — the same flag currently also sets the llama-index client's
is_aoss (custom id field, no post-ingest refresh()), via DummyAuth(service='aoss').
A managed Amazon OpenSearch Service domain needs SigV4 auth with non-AOSS document handling (_id + refresh()), so these can't stay coupled. Scope:
- Separate the SigV4-auth signal from the AOSS-behavior signal so a managed OpenSearch Service domain can use SigV4 without AOSS document semantics.
- Add a connection path / config for managed Amazon OpenSearch Service (IAM/SigV4), distinct from both
aoss:// and the basic-auth opensearch://.
- Decide how the endpoint scheme is expressed (opensearch-py accepts an endpoint without an explicit
http:///https://); document the default.
- Tests for the managed-Service auth path and document handling.
Alternatives considered
Leave opensearch:// as basic-auth/no-auth only and document managed Amazon OpenSearch Service as unsupported (current state in #397). Works for self-hosted OpenSearch but excludes managed non-serverless domains.
Package
lexical-graph
Problem statement
The
opensearch://vector store (added in #397) connects to OpenSearch endpoints that don't use AWS authentication — HTTP basic auth fromOPENSEARCH_USERNAME/OPENSEARCH_PASSWORD, or no auth. It's been validated against a localhost Docker/Finch container. Two deployment shapes aren't covered yet:Raised in review of #397 (@acarbonetto): OpenSearch can run well beyond localhost (EC2, on-prem, managed Service), and we should support more of those once the localhost path is in.
Proposed solution
Decouple the two concerns the current single flag ties together:
is_sigv4_authselects AWS SigV4 vs basic/no auth increate_os_client/create_os_async_client.is_aoss(customidfield, no post-ingestrefresh()), viaDummyAuth(service='aoss').A managed Amazon OpenSearch Service domain needs SigV4 auth with non-AOSS document handling (
_id+refresh()), so these can't stay coupled. Scope:aoss://and the basic-authopensearch://.http:///https://); document the default.Alternatives considered
Leave
opensearch://as basic-auth/no-auth only and document managed Amazon OpenSearch Service as unsupported (current state in #397). Works for self-hosted OpenSearch but excludes managed non-serverless domains.