Skip to content

test(e2e): switch organization-provided LLM profiles - #1082

Draft
saurya wants to merge 1 commit into
mainfrom
test/ohe-member-switch-org-llm-profiles
Draft

test(e2e): switch organization-provided LLM profiles#1082
saurya wants to merge 1 commit into
mainfrom
test/ohe-member-switch-org-llm-profiles

Conversation

@saurya

@saurya saurya commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Migration

Ports https://github.com/OpenHands/saas-deploy/pull/592 from OpenHands/saas-deploy so the test ships with the OpenHands-Cloud chart release it validates.

This pull request is based directly on main, which includes the foundational Playwright harness from OpenHands-Cloud #1048.

Original scenario contract

Scenario

Covers a member switching a live Canvas conversation between two LLM profiles supplied through the organization-scoped profile endpoint.

Environment contract

  • BASE_URL: required non-production OHE deployment; production is rejected.
  • e2e_tests/fixtures/auth.json: authenticated member storage state.
  • TEST_ORG_LLM_PROFILE_ONE: first org-admin-configured profile visible to the member.
  • TEST_ORG_LLM_PROFILE_TWO: second, different org-admin-configured profile.
  • TEST_LLM_SWITCH_PROMPT: harmless conversation launch prompt.

The test proves both names came from /api/organizations/{org}/profiles and throws actionable errors rather than skipping.

Cleanup and isolation

  • Switches the conversation to both configured profiles and verifies backend active_profile after each switch.
  • Proves conversation switching did not change the member's global active profile.
  • Deletes the created conversation in finally.
  • Adds one spec with one test; shared harness files are unchanged.

Validation

  • npm ci once: passed.
  • npm run lint: blocked by 15 pre-existing errors in existing page objects and smoke.spec.ts; the new file is clean.
  • Focused ESLint, Prettier, and tsc --noEmit: passed.
  • Playwright --list: passed.
  • Live execution unavailable locally because the auth fixture and configured organization profiles were absent.

AI disclosure: This pull request was created by an AI agent (OpenHands) on behalf of the user.

Port validation

  • Exactly one changed file: e2e_tests/tests/member-switch-org-llm-profiles.spec.ts.
  • Exactly one Playwright test in the spec.
  • Focused ESLint and Prettier checks passed.
  • tsc --noEmit passed against the Cloud harness.
  • Chromium Playwright --list discovery passed.
  • Live execution still requires the explicit authentication and TEST_* contracts documented above.

AI disclosure: This pull request was migrated by an AI agent (OpenHands) on behalf of the user.

@github-actions github-actions Bot added the type: test Adding or fixing tests label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: test Adding or fixing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant