[FEATURE] Detect AOSS generation via BatchGetCollectionGroup instead of error-string retry

[noel-improv]

Package

lexical-graph

Problem statement

index_exists() currently decides between the Classic and NextGen knn_vector mappings reactively: it sends the Classic mapping first and, if the collection rejects it with a NextGen-incompatible field error, retries once with the NextGen mapping. The trigger is a match on the OpenSearch error text (_is_nextgen_incompatible_field_error).

This works today with any boto3 version and no extra IAM, but it couples detection to an AWS error message that could change. Raised in review of #391.

Proposed solution

Read the collection generation deterministically from the AOSS control plane, and keep the reactive retry as a fallback.

1. Deterministic detection via BatchGetCollectionGroup

As of boto3>=1.43.17, the OpenSearch Serverless BatchGetCollectionGroup API exposes a generation field (CLASSIC | NEXTGEN):

aoss = boto3.client("opensearchserverless")
coll = aoss.batch_get_collection(names=[collection_name])["collectionDetails"][0]
group_name = coll.get("collectionGroupName")
is_nextgen = (
    group_name
    and aoss.batch_get_collection_group(names=[group_name])["collectionGroupDetails"][0].get("generation") == "NEXTGEN"
)

The OpenSearchServerlessGeneration enum added in #391 uses values (CLASSIC/NEXTGEN) that match this field verbatim, so the API result maps straight onto GraphRAGConfig.opensearch_serverless_generation with no interface change.

2. New IAM permissions and the AccessDenied fallback

Detection runs on the default path for every deployment, Classic included, so the API needs two control-plane permissions the toolkit does not currently request:

• aoss:BatchGetCollection
• aoss:BatchGetCollectionGroup

The example templates grant AOSS data-plane access via aoss:APIAccessAll, which does not cover these control-plane calls. Every client role in every shipped CFN template would need the two new actions, and a deployment missing them would fail closed with AccessDenied on a collection that works today. So the API path wants an AccessDenied fallback to the current reactive retry rather than replacing it.

3. Changes required

• Bump boto3>=1.40.61 to >=1.43.17 in lexical-graph/src/graphrag_toolkit/lexical_graph/requirements.txt.
• Add a generation resolver that calls the API and maps the result to OpenSearchServerlessGeneration, falling back to the reactive path on AccessDenied/ClientError or when boto3 is too old.
• Wire the resolver into index_exists() ahead of the reactive retry.
• Add aoss:BatchGetCollection and aoss:BatchGetCollectionGroup to the client-role policy in every AOSS example template (Classic and NextGen).

4. Tests

• API returns NEXTGEN and CLASSIC are each honoured.
• No collection group means Classic.
• AccessDenied falls back to the reactive retry.
• A boto3 older than the new floor falls back.

5. Out of scope

• Removing the reactive retry or the error-string matcher — both stay as the fallback.
• Any change to the opensearch_serverless_generation public interface.

Alternatives considered

Keep the reactive error-string detection as the only mechanism (current state in #391). It works with no new IAM or boto3 floor, but stays coupled to AWS error text.