Conversation
…ent OOM The previous OOM fix loaded automations one at a time but still accumulated all runs for a single automation in memory. For orgs with large cloud security check histories, a single automation's runs could exceed the 6GB heap limit. Now uses async generators to stream run batches (50 at a time) through PDF and JSON generation. Peak memory is bounded by one batch of runs + the jsPDF document, regardless of total automation size. - evidence-data-loader: add streamAutomationRuns async generator - evidence-pdf-generator: extract renderRunToPDF, add generateAutomationPDFFromStream - evidence-json-builder: add buildAutomationJsonStream using Readable.from() - evidence-export.service: wire streaming into ZIP export path Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…task The auditor bulk evidence export previously ran in the API process, peaking at ~20% memory per request. Multiple concurrent exports could OOM the container. Now the heavy work (DB queries, PDF generation, ZIP creation) runs in a Trigger.dev background task with its own memory. The API endpoint triggers the task and returns a runId for progress tracking. - Add export-organization-evidence Trigger.dev task (S3 upload + presigned URL) - Change POST /v1/evidence-export/all to trigger background task - Frontend uses useRealtimeRun for progress + auto-download on completion - API process memory stays flat regardless of export size Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…OOM + retry storms Follow-up to the Trigger.dev offload, addressing a 2026-06-04 prod downtime (3 concurrent in-process exports for one org starved the event loop → health checks failed → ECS restarted the tasks; ~11 min degraded). - Stream the ZIP straight to S3 via @aws-sdk/lib-storage Upload + a PassThrough instead of Buffer.concat'ing the whole archive. Peak worker memory is now bounded by one automation's PDF plus the uploader's part buffer (~40MB), never the full ZIP. Populate + upload run concurrently with explicit error forwarding and symmetric upload.abort() on either failure (no orphaned multipart parts). - Run the task on machine large-1x (8GB/4vCPU) — isolated from the API. - Serialize exports per org: task queue concurrencyLimit 1 + per-org concurrencyKey, plus an org+includeJson idempotencyKey (30m TTL) so a burst of retries collapses to a single run instead of N concurrent exports. - Record partial failures in the manifest (hasFailures + failedTasks[]) and in run metadata (tasksFailed) so a partial export is self-evident from the ZIP. - Use ctx.run.id for a unique, traceable S3 key. Tests: streamArchiveToS3 happy-path + both failure paths; controller asserts the per-org concurrency/idempotency trigger options. 43 passing. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Resolve conflicts from the divergent bulk-evidence-export work:
- evidence-export.controller.spec.ts: keep the background-task assertions
(AuditorEvidenceExportController triggers a Trigger.dev job and returns
{ runId, publicAccessToken }); drop main's obsolete direct-stream
(res.setHeader/flushHeaders/archive.pipe) assertions, which referenced a
`res` no longer in scope for that controller.
- evidence-export.controller.ts: drop the orphaned `res.flushHeaders()` that
the auto-merge spliced into exportAllEvidence — that method no longer takes
an @res() param, so the call referenced an undefined `res` and broke the
build. The single-task GET export keeps main's streaming + client-disconnect
hardening.
- bun.lock: regenerated via `bun install` to reconcile the PR's new
@aws-sdk/lib-storage + @aws-sdk/s3-request-presigner deps with main.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…lent failures, invalid hook option)
Three issues in the new bulk-evidence-export progress UI:
1. ExportEvidenceButton: the running export sheet could not be dismissed —
onOpenChange swallowed every close while a run was active, directly
contradicting the in-sheet copy ("You can close this dialog — the export
will continue in the background"). Lifted the useRealtimeRun subscription
from the unmount-on-close ExportProgress child up to the parent so the run
keeps streaming and still auto-downloads on completion after the sheet is
closed, then made the sheet freely dismissable (onOpenChange={setIsOpen}).
(reported by cubic)
2. TasksPageClient: onComplete handled the happy path but, when the finished
run had no downloadUrl, it silently closed the export UI with no download
and no feedback — a completed-but-invalid export looked like a no-op. Added
the missing error toast, matching ExportEvidenceButton. (reported by cubic)
3. Both files passed an `onError` option to useRealtimeRun, which does not
exist on UseRealtimeSingleRunOptions — a type error that was failing the
app build. Removed it and folded failure handling into onComplete(run, err),
treating any non-COMPLETED terminal state (or an err) as a failure, the same
way PolicyHeaderActions does.
Tests: new ExportEvidenceButton.test.tsx (dismiss-while-running, download +
success, metadata fallback, missing-link error, run-failure error, onComplete
err) and added export-flow cases to TasksPageClient.test.tsx; also fixed that
file's stale `downloadAllEvidenceZip` mock (now triggerBulkEvidenceExport).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…om-streaming [dev] [Marfuen] mariano/fix-evidence-export-oom-streaming
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
🎉 This PR is included in version 3.73.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an automated pull request to release the candidate branch into production, which will trigger a deployment.
It was created by the [Production PR] action.
Summary by cubic
Offloaded the auditor bulk evidence export to a Trigger.dev background task and streamed the ZIP directly to S3 to prevent API OOMs. The app now shows progress and auto-downloads when ready.
New Features
POST /v1/evidence-export/all?includeJson=starts an export and returns{ runId, publicAccessToken }. Per‑org serialization and 30m idempotency via concurrency and idempotency keys.export-organization-evidence: streams the ZIP to S3 using@aws-sdk/lib-storage, then returns a presigned URL. Includes a manifest and records partial failures.ExportEvidenceButtonand Tasks page trigger the export, show live progress with@trigger.dev/react-hooks, allow closing the dialog, and auto‑download on completion. Clear toasts on success/failure.@aws-sdk/lib-storage.Migration
POST /v1/evidence-export/alland track with the returnedrunId/publicAccessTokeninstead of downloading directly.APP_AWS_BUCKET_NAME,APP_AWS_REGION,APP_AWS_ACCESS_KEY_ID,APP_AWS_SECRET_ACCESS_KEY(optionalAPP_AWS_ENDPOINT).Written for commit dbfa0a7. Summary will update on new commits.