Skip to content

[FEATURE] Support Amazon OpenSearch Service (managed, non-serverless) with SigV4/IAM auth #407

Description

@noel-improv

Package

lexical-graph

Problem statement

The opensearch:// vector store (added in #397) connects to OpenSearch endpoints that don't use AWS authentication — HTTP basic auth from OPENSEARCH_USERNAME/OPENSEARCH_PASSWORD, or no auth. It's been validated against a localhost Docker/Finch container. Two deployment shapes aren't covered yet:

  • A managed Amazon OpenSearch Service domain (not Serverless) that authenticates with IAM/SigV4.
  • Any auth beyond basic/none (e.g. mutual TLS).

Raised in review of #397 (@acarbonetto): OpenSearch can run well beyond localhost (EC2, on-prem, managed Service), and we should support more of those once the localhost path is in.

Proposed solution

Decouple the two concerns the current single flag ties together:

  • Auth mechanismis_sigv4_auth selects AWS SigV4 vs basic/no auth in create_os_client / create_os_async_client.
  • AOSS document handling — the same flag currently also sets the llama-index client's is_aoss (custom id field, no post-ingest refresh()), via DummyAuth(service='aoss').

A managed Amazon OpenSearch Service domain needs SigV4 auth with non-AOSS document handling (_id + refresh()), so these can't stay coupled. Scope:

  • Separate the SigV4-auth signal from the AOSS-behavior signal so a managed OpenSearch Service domain can use SigV4 without AOSS document semantics.
  • Add a connection path / config for managed Amazon OpenSearch Service (IAM/SigV4), distinct from both aoss:// and the basic-auth opensearch://.
  • Decide how the endpoint scheme is expressed (opensearch-py accepts an endpoint without an explicit http:///https://); document the default.
  • Tests for the managed-Service auth path and document handling.

Alternatives considered

Leave opensearch:// as basic-auth/no-auth only and document managed Amazon OpenSearch Service as unsupported (current state in #397). Works for self-hosted OpenSearch but excludes managed non-serverless domains.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions