상용화 준비: 공개 공유 링크 LLM 비용 방어#415
Open
seonghobae wants to merge 88 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR hardens the public share-link API surface against unauthenticated LLM-provider cost abuse by default-disabling mode=llm-draft on /api/share/* endpoints, while keeping authenticated snapshot LLM draft behavior unchanged. It also adds supporting documentation for commercial readiness and API security settings.
Changes:
- Add
SHARE_LINK_LLM_DRAFT_ENABLED(defaultfalse) and enforce it by returning403for share-linkmode=llm-draft. - Add/extend tests to ensure public share links do not call the LLM provider by default and still mask LLM configuration errors when explicitly enabled.
- Document the new default and broader commercial-readiness/security gates in README and
docs/.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents that share-link mode=llm-draft is blocked unless explicitly enabled. |
| docs/commercial-readiness.md | Adds a commercial readiness rubric and records the “public endpoint cost abuse” gate. |
| docs/api-security-checklist.md | Adds the new setting and references the enforcement/test coverage. |
| backend/tests/test_reversing_llm.py | Adds tests asserting share-link LLM draft is disabled by default and remains sanitized when enabled. |
| backend/app/settings.py | Introduces share_link_llm_draft_enabled with default False (env: SHARE_LINK_LLM_DRAFT_ENABLED). |
| backend/app/api/share.py | Enforces the setting by blocking mode=llm-draft on share-link export endpoints. |
| .env.example | Documents and exposes SHARE_LINK_LLM_DRAFT_ENABLED=false default. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.env.example, README.md, backend/app/api/share.py, backend/app/api/snapshots.py, backend/app/main.py, backend/app/schemas.py, backend/app/settings.py, backend/app/spec/llm.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
aee1e6303d01e6ffcf4e98b1047e58ec5e345782 - Workflow run: 28559426960
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (3 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (3 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (8 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (8 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (3 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (3 files)"]
R3 --> V3["docs review"]
Contributor
OpenCode Review Overview
Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (11 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (11 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Workflow: ci.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: ci.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Backend (38 files)"]
S3 --> I3["API and service runtime"]
I3 --> R3["Review risk: Backend (38 files)"]
R3 --> V3["backend tests"]
Evidence --> S4["Docs (39 files)"]
S4 --> I4["operator or user guidance"]
I4 --> R4["Review risk: Docs (39 files)"]
R4 --> V4["docs review"]
Evidence --> S5["Frontend (17 files)"]
S5 --> I5["browser runtime and bundle"]
I5 --> R5["Review risk: Frontend (17 files)"]
R5 --> V5["frontend tests"]
Evidence --> S6["CI script (15 files)"]
S6 --> I6["review and security gate shell path"]
I6 --> R6["Review risk: CI script (15 files)"]
R6 --> V6["bash -n plus Strix self-test"]
Evidence --> S7["Test (3 files)"]
S7 --> I7["regression suite"]
I7 --> R7["Review risk: Test (3 files)"]
R7 --> V7["targeted test run"]
|
…L schemes) Python의 `urllib.parse.urlsplit` 함수가 언더스코어(`_`)가 포함된 URL 스킴(예: `snowflake_invalid://`)을 올바르게 파싱하지 못하고 빈(empty) 스킴으로 평가하는 문제가 있었습니다. 이로 인해 DSN 에러 메시지에 포함된 비밀번호를 추출하여 마스킹(redaction)하는 로직이 동작하지 않아, 로그 및 클라이언트 에러 응답에 데이터베이스 자격 증명이 노출되는 취약점이 존재했습니다. 파싱 전 정규표현식을 통해 스킴 부분을 임시로 알려진 올바른 형태(예: `http://`)로 변경한 뒤 `urlsplit`을 수행하도록 우회 코드를 추가하여, 알 수 없거나 잘못된 스킴을 가진 DSN 문자열에서도 올바르게 비밀번호 후보군을 추출하고 마스킹할 수 있도록 수정했습니다. 또한 해당 이슈에 대한 자동화된 회귀 테스트(regression test)를 추가하고, 관련 내용을 `.jules/sentinel.md` 에 기록하였습니다.
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.env.example, .jules/sentinel.md, README.md, backend/app/api/share.py, backend/app/api/snapshots.py, backend/app/dsn_redaction.py, backend/app/main.py, backend/app/observability.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
7301bc486ff7ac626800a0c0900d5175e67ccc1d - Workflow run: 28560702529
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (4 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (4 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (12 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (12 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (5 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (5 files)"]
R3 --> V3["docs review"]
seonghobae
force-pushed
the
product/commercial-readiness
branch
from
cef4870 to
4c366bf
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.env.example, .jules/sentinel.md, README.md, backend/app/api/share.py, backend/app/api/snapshots.py, backend/app/dsn_redaction.py, backend/app/license_gate.py, backend/app/main.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
f68f1469444d879dbb0d53024e2fd1ab26b10253 - Workflow run: 28561748416
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Changed file (4 files)"]
S1 --> I1["repository behavior"]
I1 --> R1["Review risk: Changed file (4 files)"]
R1 --> V1["required checks"]
Evidence --> S2["Backend (14 files)"]
S2 --> I2["API and service runtime"]
I2 --> R2["Review risk: Backend (14 files)"]
R2 --> V2["backend tests"]
Evidence --> S3["Docs (9 files)"]
S3 --> I3["operator or user guidance"]
I3 --> R3["Review risk: Docs (9 files)"]
R3 --> V3["docs review"]
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| report = generate_report(_explicit_evidence_paths_from_args(args)) | ||
| payload = json.dumps(report, ensure_ascii=False, indent=2, sort_keys=True) | ||
| if args.output == "-": | ||
| print(payload) |
| elif args.output: | ||
| output_path = pathlib.Path(args.output) | ||
| output_path.parent.mkdir(parents=True, exist_ok=True) | ||
| output_path.write_text(payload + "\n", encoding="utf-8") |
| bundle = generate_bundle(args) | ||
| payload = json.dumps(bundle, ensure_ascii=False, indent=2, sort_keys=True) | ||
| if args.output == "-": | ||
| print(payload) |
| else: | ||
| output_path = pathlib.Path(args.output) | ||
| output_path.parent.mkdir(parents=True, exist_ok=True) | ||
| output_path.write_text(payload + "\n", encoding="utf-8") |
| contract_index = 8 if self.contract_event is not None else None | ||
| share_index = 9 if self.contract_event is not None else 8 | ||
| events_index = share_index + 1 | ||
| llm_index = events_index + 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
요약
docs/commercial-readiness.md를 갱신했습니다.mode=llm-draft호출을 기본값에서 차단해, 인증 없는 URL이 외부 LLM provider 비용을 만들지 못하게 했습니다.SHARE_LINK_DEFAULT_TTL_HOURS=168을 추가하고, 프로젝트 오너용 목록/삭제 API로 공개 링크를 폐기할 수 있게 했습니다.APP_ENV=productionstartup guard로 OIDC, 공개 HTTPS CORS origin, 32자 이상 secret, 대상 DB allowlist, 공유 링크 기본 만료, 라이선스 검증값을 강제합니다.LLM_MAX_PROMPT_CHARS와LLM_MAX_OUTPUT_TOKENS를 적용해 provider 호출 전 비용 상한을 둡니다.LICENSE_KEY호환은 유지하면서LICENSE_PUBLIC_KEY기반 Ed25519 서명 온프레미스 토큰 검증을 추가했습니다.python -m app.license_tokensCLI를 추가해 Ed25519 keypair 생성과 계약별 signed license token 발급/재발급을 운영자가 수행할 수 있게 했습니다.LICENSE_REVOKED_TOKEN_IDS,LICENSE_REVOKED_SUBJECTS를 추가해 특정 서명 토큰jti또는 고객sub를 만료 전 회수할 수 있게 했습니다.GET /api/billing/usage를 추가해 현재 사용자 소유 프로젝트 범위의 프로젝트/시트/연결/스냅샷/공유 링크/활성 공유 링크 사용량, 적용 한도, 라이선스 검증 방식을 집계합니다.BILLING_MAX_*한도를 추가해 프로젝트/연결/스냅샷/공유 링크 생성 시 유료 플랜 사용량 제한을 적용할 수 있게 했습니다.BILLING_PORTAL_URL,BILLING_SUPPORT_URL,ACCOUNT_REACTIVATION_URL을 billing usage 응답과 account-deactivated 응답 헤더로 노출해 미납/계약 중단 고객의 다음 조치 경로를 제공합니다.ACCOUNT_DEACTIVATED_SUBJECTS를 추가해 계약 중단/미납/abuse 대상 OIDC subject를 DB 접근 전 403으로 차단하고, production에서는 재활성화 또는 지원 URL 없이 비활성 목록을 켤 수 없게 했습니다.deploy/prometheus/pg-erd-cloud-alerts.yml)를 추가해 HTTP 5xx, p95 latency, authz failure, share audit failure, job failure/wait alerts를 배포 가능하게 했습니다.npm run test:a11y)와 CIAccessibility smoke단계를 추가했습니다.npm run test:e2e)와 CIBrowser E2E smoke단계를 추가해 demo workspace load, editor toolbar interaction, screenshot rendering을 검증합니다.npm run test:visual)와 CIVisual regression단계를 추가해 demo editor desktop 1280x720, mobile review 390x844, share/export modal 1280x720 baseline을 검증합니다.docs/ui-ux/visual-regression-baseline.md를 추가해 baseline 갱신 사유, Figma/참조 증거, 브라우저 증거, 승인자, No-Go 조건, 갱신 명령 순서를 고정했습니다.docs/legal/commercial-release-approval.md를 추가해 이용약관/개인정보/SLA/보안 신고/운영 runbook의 판매 전 승인 기록 gate를 명시했습니다.상용화 판단
현재 제품은 실행 가능한 MVP 기반이지만 그대로 일반 판매 가능한 패키지는 아닙니다. 이번 PR은 P0/P1/P2 리스크 중 공개 엔드포인트 비용/노출 남용, 운영 설정 누락, LLM 비용 폭주, 복구 절차 부재, 온프레미스 라이선스 검증, 라이선스 발급/재발급 CLI, 라이선스 회수 목록, 사용량 집계 API, 기본 사용량 한도 enforcement, billing/reactivation URL 노출, subject 기반 계정 비활성화와 재활성화 경로, 운영 alert artifact, 접근성 smoke gate, 브라우저 E2E smoke gate, desktop/mobile/share-export 픽셀 baseline 시각 회귀 gate, baseline 갱신 승인 절차, 법무/지원 승인 gate 문서화를 우선 닫습니다. 리뷰 봇 또는 장시간 분석 봇 대기는 blocker로 보지 않습니다.
남은 주요 No-Go는 실제 과금 포털/요금제 변경 실행 자동화, 릴리즈별 법무 승인 기록 첨부, Firefox/WebKit 등 추가 브라우저 baseline과 first-run setup 같은 다른 핵심 workflow별 visual baseline입니다.
검증
cd backend && PYTHONPATH=. uv run pytest -q tests/test_billing_usage.py tests/test_auth_security.py::test_get_current_user_rejects_deactivated_subject_before_db tests/test_production_config.py-> expected RED before implementation: missing settings/schema/header/guardcd backend && PYTHONPATH=. uv run pytest -q tests/test_billing_usage.py tests/test_auth_security.py::test_get_current_user_rejects_deactivated_subject_before_db tests/test_production_config.py-> 12 passed, 1 warningcd backend && PYTHONPATH=. uv run pytest -q tests/test_billing_usage.py tests/test_auth_security.py tests/test_production_config.py tests/test_observability.py-> 59 passed, 1 warningcd backend && PYTHONPATH=. uv run mypy app-> successcd backend && PYTHONPATH=. uv run pytest -q-> 272 passed, 1 warningnpm run test:e2e -- e2e/visual-regression.spec.ts-> expected RED: missing snapshot baselinecd frontend && npm run test:visualafter adding mobile viewport -> expected RED: missingdemo-editor-mobile.pngcd frontend && npm run test:visualafter adding share/export modal -> expected RED: missingshare-export-modal.pngcd frontend && npm run test:visual -- --update-snapshots-> 3 passed, desktop/mobile/share-export baselines generatedcd frontend && npm run test:visual-> 3 passedcd frontend && npm run test:visual-> 3 passedcd frontend && npm run test:e2e-> 1 passedcd frontend && npm run test:a11y-> 4 passedcd frontend && npm run typecheck-> successcd frontend && npm test -- --run-> 105 passedcd frontend && npm run build-> successhttp://127.0.0.1:5173/demo mode loaded,편집기interaction reached toolbar/search/share controls, console warnings/errors empty, viewport screenshot captured.git diff --check-> successcd backend && uv run pytest -q tests/test_usage_quotas.py tests/test_billing_usage.py tests/test_api_connections.py tests/test_api_snapshots.py tests/test_reversing_llm.py-> 37 passed, 1 warningcd backend && uv run pytest -q tests/test_license_tokens.py tests/test_license_gate.py-> 15 passedcd backend && uv run pytest -q tests/test_license_tokens.py tests/test_license_gate.py tests/test_observability.py tests/test_production_config.py-> 40 passed, 1 warningcd backend && uv run python -m app.license_tokens generate-keypair --format json-> successruby -e 'require "yaml"; YAML.load_file(ARGV.fetch(0)); puts "ok"' deploy/prometheus/pg-erd-cloud-alerts.yml-> okb85b154: checks are running from the new push.Strix/OpenCode/review scheduler delays are treated as automation delay unless they produce a concrete source finding.추가 반복(80ec906): first-run empty dashboard visual baseline을 추가해 빈 계정 온보딩 화면을 release visual gate에 포함했습니다.
?demo-workspace=empty데모 진입점은 production API에 영향을 주지 않고 deterministic empty workspace를 렌더링합니다.cd frontend && npm run test:visual-> 4 passedcd frontend && npm run test:e2e-> 1 passedcd frontend && npm run test:a11y-> 4 passedcd frontend && npm run typecheck-> successcd frontend && npm test -- --run-> 105 passedcd frontend && npm run build-> successgit diff --check-> success추가 반복(501b34d): 판매 릴리즈별 법무/지원 승인 기록을
docs/legal/release-approvals/*.jsonmanifest로 표준화하고, CI에서scripts/ci/validate_commercial_release_approval.py로 승인 필드와 필수 문서 경로를 검증하도록 했습니다. example manifest는 형식 검증용이며 실제 판매 승인 기록으로 간주하지 않습니다.python scripts/ci/validate_commercial_release_approval.py-> successpython scripts/ci/validate_codeql_backfill.py-> successgit diff --check-> success추가 반복(a55ea3a):
/api/me실패 시 account-deactivated/billing support 헤더를 프런트 API 오류에 보존하고, 인증 gate에서 rawgetMe failed: 403대신 인증 필요/권한 없음/계정 비활성화 메시지와 재활성화·지원 링크를 표시하도록 했습니다.cd frontend && npm run test:a11y-> 5 passedcd frontend && npm run typecheck-> successcd frontend && npm test -- --run-> 106 passedcd frontend && npm run test:e2e-> 1 passedcd frontend && npm run test:visual-> 4 passedcd frontend && npm run build-> successgit diff --check-> success추가 반복(0ef2af4):
docs/operations/alert-thresholds.md를 추가해 Prometheus alert별 severity, 고객 영향, 1차 대응, owner, escalation rule을 상용 운영 gate로 고정하고, release approval manifest의 필수 문서로 검증되게 했습니다.python scripts/ci/validate_commercial_release_approval.py-> successruby -e 'require "yaml"; YAML.load_file(ARGV.fetch(0)); puts "ok"' deploy/prometheus/pg-erd-cloud-alerts.yml-> okgit diff --check-> success추가 반복(e8bb71c):
POST /api/billing/plan-change를 추가해 target plan을 검증하고,BILLING_PORTAL_URL이 있으면 target_plan query가 포함된 portal redirect action을 반환하며, portal이 없고 support URL만 있으면 contact-support action으로 fallback합니다. 둘 다 없으면 503으로 과금 경로 누락을 명확히 실패시킵니다.cd backend && PYTHONPATH=. uv run pytest -q tests/test_billing_usage.py-> 5 passed, 1 warningcd backend && PYTHONPATH=. uv run mypy app-> successcd backend && PYTHONPATH=. uv run pytest -q tests/test_billing_usage.py tests/test_production_config.py tests/test_auth_security.py::test_get_current_user_rejects_deactivated_subject_before_db-> 15 passed, 1 warningcd backend && PYTHONPATH=. uv run pytest -q-> 275 passed, 1 warningpython scripts/ci/validate_commercial_release_approval.py-> successgit diff --check-> success