[New Integration] OpenAI Compliance API

[jamiehynds]

Description

Elastic's existing openai integration covers the OpenAI API Platform plane (Usage, Rate Limits, org Audit Logs on api.openai.com, via an Admin key). It does not cover the ChatGPT Enterprise workspace plane — end-user, app, agent, and admin activity inside a ChatGPT Enterprise/Edu workspace — which is where the security value for SOC teams sits.

This epic delivers that as a new, standalone package (see "Package decision" below), collecting the ChatGPT Enterprise Compliance Logs Platform feed, ECS-aligned, with detection rules and a security dashboard.

SOC use cases

• AI account takeover — login anomalies, impossible travel, anomalous geo/JA3-JA4 on accounts holding corporate data.
• Control tampering — SCIM disabled, IP allowlist removed, audit logging off, role escalation, rogue keys/service accounts.
• Agentic & Codex risk — custom agent runtime (connector-OAuth grants, skills, triggers), Codex security findings, risky GitHub/Codex connections, MCP usage.
• Shadow AI & insider risk — app/connector sprawl, over-permissive sharing, abnormal conversation/file activity.

Package

Ship as a new package, not as data streams on the existing openai package. The two are different OpenAI products: different host (api.chatgpt.com vs api.openai.com), different credential (Compliance-scoped key vs Admin key), different tier (Enterprise/Edu only), and different audience. This mirrors the existing azure + azure_openai split.

• name: openai_chatgpt_enterprise
• title: OpenAI ChatGPT Enterprise
• The existing openai package is unchanged (no rename — package names are immutable post-publish). Optionally sharpen its description to "OpenAI API Platform" for catalog clarity (title/description edits are safe; a name change is not).

Scope — data streams (Compliance Logs Platform)

Input type cel. One data stream per event_type:

Data stream	event_type	Notes
auth	AUTH_LOG	Login/logout success & failure; IP, geo, user agent, JA3/JA4
app_auth	APP_AUTH_LOG	App/connector credential link/unlink (OAuth)
audit	AUDIT_LOG	Admin/RBAC actions (~150): role/SCIM/IP-allowlist/key/service-account/org config
codex_security	CODEX_SECURITY_LOG	Whitespace — scan config, scan findings, proposed-patch PRs
agents	CUSTOM_AGENTS_LOG	Whitespace — agent runtime: connector-OAuth resolved, connector calls, skills, triggers, memory R/W
app	APP_LOG	App/connector tool invocations incl. app_type: MCP
codex	CODEX_LOG	Codex client activity, APP_MCP_CALL, plugin lifecycle
conversation	CONVERSATION_MESSAGE	Per-message content + citations/annotations. Highest volume, PII/secrets, ~24h lag

User control: each data stream must be independently enable/disable-able in Fleet, so customers can collect only the categories they want (e.g. conversation only, or the security-focused streams only). This is why the design is one data stream per event_type and must not be collapsed into a single multi-type stream. The Compliance key + WorkspaceID are configured once at the package level and shared across all enabled streams.

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[github-actions]

tl;dr: this is not a net-new integration in the current repo state — openai already ships an audit data stream that polls OpenAI audit logs, so the right path is to scope this as a gap/extension in packages/openai (or close as duplicate if it fully matches your request).

Recommendation
Treat #19399 as an enhancement/clarification against the existing OpenAI integration, not a new package.

1. Confirm whether the ask is already covered by current OpenAI audit ingestion (/v1/organization/audit_logs).
2. If partially covered, open implementation work under packages/openai/data_stream/audit (fields/pipeline/docs/tests).
3. If fully covered, close as duplicate and point to existing OpenAI audit support and docs.

Findings

I ran a “pick three, keep one” investigation (3 parallel codebase investigations from different angles). The strongest candidate and direct repo evidence:

• packages/openai/manifest.yml:5 — integration description already says it collects OpenAI usage metrics and audit logs.
• packages/openai/data_stream/audit/manifest.yml:1-8 — dedicated audit logs data stream exists.
• packages/openai/data_stream/audit/agent/stream/cel.yml.hbs:36-39 — collector calls OpenAI audit endpoint .../v1/organization/audit_logs with pagination/time filters.
• packages/openai/docs/README.md:13 — docs explicitly reference OpenAI Audit Logs API.
• packages/openai/changelog.yml:7-11 — 2.0.0 entry: “Add audit data stream to fetch audit logs.”

Relevant precedent for compliance-style audit integrations:

• packages/anthropic/data_stream/audit/agent/stream/cel.yml.hbs:69 calls .../v1/compliance/activities.
• packages/anthropic/changelog.yml:7-10 shows initial release scope for compliance activities.

I could not read issue body/comments for #19399 (and several similar issues found by search) because of integrity filtering in this run, so I cannot confirm request nuances beyond the title.

Verification

I validated key paths/endpoints directly in the workspace:

$ git --no-pager ls-files packages/openai/data_stream/audit packages/openai/docs/README.md packages/openai/changelog.yml packages/anthropic/data_stream/audit/agent/stream/cel.yml.hbs
packages/openai/data_stream/audit/agent/stream/cel.yml.hbs
packages/openai/data_stream/audit/manifest.yml
packages/openai/docs/README.md
packages/openai/changelog.yml
packages/anthropic/data_stream/audit/agent/stream/cel.yml.hbs
...

$ grep -nE '/v1/organization/audit_logs|Audit Logs API|Add audit data stream|/v1/compliance/activities' \
  packages/openai/data_stream/audit/agent/stream/cel.yml.hbs \
  packages/openai/docs/README.md \
  packages/openai/changelog.yml \
  packages/anthropic/data_stream/audit/agent/stream/cel.yml.hbs
packages/openai/data_stream/audit/agent/stream/cel.yml.hbs:36: ... "/v1/organization/audit_logs?..."
packages/openai/docs/README.md:13: ... OpenAI Audit Logs API ...
packages/openai/changelog.yml:9: ... Add audit data stream to fetch audit logs.
packages/anthropic/data_stream/audit/agent/stream/cel.yml.hbs:69: ... "/v1/compliance/activities?..."

Detailed Action Plan

1. Clarify scope against current OpenAI audit support

   • Compare issue requirements to existing OpenAI audit implementation at:
     • packages/openai/data_stream/audit/agent/stream/cel.yml.hbs:36-39
     • packages/openai/docs/README.md:113-277

2. If new OpenAI Compliance API behavior is required, extend existing stream (not new package)

   • Update collector request/cursor/auth logic in packages/openai/data_stream/audit/agent/stream/cel.yml.hbs.
   • Update mappings in packages/openai/data_stream/audit/fields/fields.yml (+ base/ECS files if needed).
   • Update ingest transforms in packages/openai/data_stream/audit/elasticsearch/ingest_pipeline/default.yml.

3. Keep docs/examples/tests in sync

   • packages/openai/docs/README.md (setup prerequisites, endpoint/scope expectations, event examples).
   • packages/openai/data_stream/audit/sample_event.json.
   • Pipeline/system tests under:
     • packages/openai/data_stream/audit/_dev/test/pipeline/*
     • packages/openai/data_stream/audit/_dev/test/system/test-default-config.yml

4. Versioning/release metadata

   • Add a new top changelog entry in packages/openai/changelog.yml describing Compliance API enhancements.

5. Triage outcome

   • If no functional gap is found, close as duplicate of existing OpenAI audit capability.
   • If gap exists, convert this issue to an enhancement scoped to packages/openai.

Related Items

Type	Link	Relevance
Issue	#19399	Current triage target ([New Integration] OpenAI Compliance API)
PR	#19174	Recent compliance-style precedent ([New Integration] Anthropic audit logs)
File	packages/openai/data_stream/audit/agent/stream/cel.yml.hbs:36-39	Existing OpenAI audit API collector logic
File	packages/openai/docs/README.md:13	Documents OpenAI Audit Logs API support
File	packages/openai/changelog.yml:7-11	Historical addition of audit data stream
File	packages/anthropic/data_stream/audit/agent/stream/cel.yml.hbs:69	Compliance endpoint pattern (/v1/compliance/activities)

Note

🔒 Integrity filter blocked 13 items

The following items were blocked because they don't meet the GitHub integrity level.

• #19399 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #19399 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #14312 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #17059 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #10118 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #17560 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #15587 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #11203 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #12900 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #10118 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #17059 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [New Integration] OpenAI Compliance API #19399 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #14312 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.