Skip to content

test(e2e): cover member organization budget enforcement - #1071

Draft
saurya wants to merge 1 commit into
mainfrom
e2e/member-org-budget-enforcement
Draft

test(e2e): cover member organization budget enforcement#1071
saurya wants to merge 1 commit into
mainfrom
e2e/member-org-budget-enforcement

Conversation

@saurya

@saurya saurya commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Migration

Ports https://github.com/OpenHands/saas-deploy/pull/580 from OpenHands/saas-deploy so the test ships with the OpenHands-Cloud chart release it validates.

This pull request is based directly on main, which includes the foundational Playwright harness from OpenHands-Cloud #1048.

Original scenario contract

Scenario

Adds one Playwright spec with one test proving a regular member cannot start a conversation after the organization cap is already exceeded. The admin sets a near-zero org cap; the secondary member performs the rejection probe.

No-spend proof

  • Requires existing month-to-date organization spend above 0.000001; the test never spends to reach the cap.
  • Requires the member start task to end as ERROR with no app_conversation_id.
  • Asserts organization current_spend is exactly unchanged before and after the probe.

Validation

  • npm ci completed once in the isolated clone
  • npm run typecheck
  • focused ESLint and Prettier checks ✅
  • npx playwright test tests/member-org-budget-enforcement.spec.ts --list
  • repository-wide npm run lint ⚠️ blocked by 15 pre-existing errors in ConversationPage.ts, HomePage.ts, and smoke.spec.ts; this new spec has no lint errors
  • Live execution not attempted because required authentication fixtures are unavailable locally.

Environment contract

  • e2e_tests/fixtures/auth.json: organization admin or owner in a dedicated non-personal org
  • SECONDARY_AUTH_STATE: valid storage state for an existing regular member with that same org selected
  • TEST_MEMBER_EMAIL: exact email for the secondary member
  • Working LiteLLM budget sync and org current_spend > 0.000001
  • No concurrent budget-mutating tests

Missing or unsuitable fixtures raise explicit errors; the test never silently skips.

Cleanup

Closes the secondary context, defensively deletes any unexpectedly created conversation, and restores the complete original organization budget snapshot in finally.

Risk

Medium: temporarily blocks all org conversation starts and uses a secondary identity. Run only against a dedicated test organization. The test is designed not to incur LLM spend.

This pull request was created by an AI agent (OpenHands) on behalf of the user.

Port validation

  • Exactly one changed file: e2e_tests/tests/member-org-budget-enforcement.spec.ts.
  • Exactly one Playwright test in the spec.
  • Focused ESLint and Prettier checks passed.
  • tsc --noEmit passed against the Cloud harness.
  • Chromium Playwright --list discovery passed.
  • Live execution still requires the explicit authentication and TEST_* contracts documented above.

AI disclosure: This pull request was migrated by an AI agent (OpenHands) on behalf of the user.

@github-actions github-actions Bot added the type: test Adding or fixing tests label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: test Adding or fixing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant