v2.4.4

.github/ISSUE_TEMPLATE/post_release_checklist.md

@@ -0,0 +1,58 @@
+---
+name: Post-release checklist
+about: Verify deployment and downstream updates after a reVISit release
+title: 'Post-release Checklist vX.Y.Z'
+labels: QA
+assignees: ''
+---
+# Post-release Checklist vX.Y.Z
+
+## [Study Repository](https://github.com/revisit-studies/study)
+
+### Release Automation
+
+- [ ] Confirm the `Release new version` workflow completed successfully
+- [ ] Confirm the release commit and `vX.Y.Z` tag were created and version-pinned URLs reference the new tag
+- [ ] Confirm the GitHub release was created and review the generated release notes
+- [ ] Confirm the production deployment completed successfully
+- [ ] Smoke test the production website
+
+### Storage
+
+- [ ] Run the Firebase cleanup in dry-run mode and review the planned deletions (`./scripts/clear-out-firebase.sh --include-dev --jobs 8`)
+- [ ] Clear Firebase test data (`./scripts/clear-out-firebase.sh --include-dev --jobs 8 --execute`)
+
+## [Documentation Repository](https://github.com/revisit-studies/reVISit-studies.github.io)
+
+- [ ] Confirm the documentation deployment completed successfully
+- [ ] Spot-check the live documentation and library list
+- [ ] Remove old bug fix version
+
+## [Template Repository](https://github.com/revisit-studies/template)
+
+- [ ] Run the `Sync from Upstream` workflow with `upstream_ref` set to `vX.Y.Z`
+- [ ] Confirm the deployment completed successfully
+- [ ] Smoke test the deployed website
+
+## [Replication Studies Repository](https://github.com/revisit-studies/replication-studies)
+
+- [ ] Run the `Sync from Upstream` workflow with `upstream_ref` set to `vX.Y.Z`
+- [ ] Confirm the deployment completed successfully
+- [ ] Smoke test the deployed website
+
+## [ReVISitPy Repository](https://github.com/revisit-studies/revisitpy)
+
+- [ ] Confirm the `Create a new minor version and push to PyPi and GitHub` workflow completed successfully
+- [ ] Confirm the new [`revisitpy`](https://pypi.org/project/revisitpy/) version is available on PyPI
+
+## [ReVISitPy Server Repository](https://github.com/revisit-studies/revisitpy-server)
+
+- [ ] Confirm the `Create a new version and push to PyPi and GitHub` workflow completed successfully
+- [ ] Confirm the new [`revisitpy-server`](https://pypi.org/project/revisitpy-server/) version is available on PyPI
+
+## TODOs
+
+Create or link issues for any bugs found during post-release verification.
+Add any other required follow-up tasks.
+
+- [ ] ...

.github/ISSUE_TEMPLATE/pre_release_checklist.md

@@ -0,0 +1,183 @@
+---
+name: Pre-release checklist
+about: Prepare and review an upcoming reVISit release
+title: 'Pre-release Checklist vX.Y.Z'
+labels: QA
+assignees: ''
+---
+# Pre-release Checklist vX.Y.Z
+
+## Release Pull Request
+
+- [ ] Confirm the release pull request targets `main`
+- [ ] Confirm the pull request title exactly matches `vX.Y.Z`
+- [ ] Confirm all required checks pass
+- [ ] Confirm the pull request has the required approvals
+- [ ] Confirm the release pull request preview deployed successfully
+- [ ] Create the post-release checklist issue and link it here
+
+## [Study Repository](https://github.com/revisit-studies/study)
+
+### Run Studies
+
+Run each study in the release PR preview. Confirm it completes without unexpected errors and its participant data appears correctly in the analysis pages.
+
+#### Demo Studies
+
+- [ ] `demo-html`
+- [ ] `demo-html-input`
+- [ ] `demo-form-elements`
+- [ ] `demo-training`
+- [ ] `demo-html-trrack`
+- [ ] `demo-svelte-trrack`
+- [ ] `demo-react-trrack`
+- [ ] `demo-screen-recording`
+- [ ] `demo-image`
+- [ ] `demo-vega`
+- [ ] `demo-video`
+- [ ] `demo-dynamic`
+- [ ] `demo-condition`
+- [ ] `demo-style`
+- [ ] `demo-yaml`
+- [ ] `demo-ranking-widget`
+- [ ] `demo-upset`
+- [ ] `demo-choropleth-map`
+
+#### Example Studies
+
+- [ ] `example-VLAT-adaptive`
+- [ ] `example-brush-interactions`
+- [ ] `example-cleveland`
+- [ ] `example-llm-chatbot`
+- [ ] `example-mvnv`
+
+#### Tutorials
+
+- [ ] `tutorial`
+- [ ] `tutorial-replication`
+
+#### Tests
+
+- [ ] `test-audio`
+- [ ] `test-component-timeout`
+- [ ] `test-device-restriction`
+- [ ] `test-library`
+- [ ] `test-likert-matrix`
+- [ ] `test-parser-errors`
+- [ ] `test-randomization`
+- [ ] `test-skip-logic`
+- [ ] `test-step-logic`
+
+#### Libraries
+
+- [ ] `library-adaptive-vlat`
+- [ ] `library-beauvis`
+- [ ] `library-berlin-num`
+- [ ] `library-calvi`
+- [ ] `library-color-blindness`
+- [ ] `library-demographics`
+- [ ] `library-graph-literacy-scale`
+- [ ] `library-mic-check`
+- [ ] `library-mini-vlat`
+- [ ] `library-nasa-tlx`
+- [ ] `library-previs`
+- [ ] `library-quis`
+- [ ] `library-sam`
+- [ ] `library-screen-recording`
+- [ ] `library-smeq`
+- [ ] `library-sus`
+- [ ] `library-ueq`
+- [ ] `library-ueq-s`
+- [ ] `library-umux`
+- [ ] `library-umux-lite`
+- [ ] `library-virtual-chinrest`
+- [ ] `library-vlat`
+
+### Analysis Pages
+
+- [ ] Study Summary
+  - [ ] Filter works correctly
+  - [ ] Statistics look correct
+- [ ] Participant View
+  - [ ] Filter works correctly
+  - [ ] Reject / Undo reject participant works correctly
+  - [ ] Replay timeline looks correct
+- [ ] Replay
+  - [ ] Replay works correctly at different play speeds
+  - [ ] Check provenance data (audio, screen, etc.)
+  - [ ] Navigate to the next and previous participant IDs
+  - [ ] Add participant tags
+  - [ ] Navigate to the next and previous participant tasks
+  - [ ] Add task tags
+  - [ ] Load transcripts (e.g. `demo-screen-recording`)
+  - [ ] Download audio/screen recordings (e.g. `demo-screen-recording`)
+  - [ ] Reject / Undo reject participant works correctly
+- [ ] Trial Stats
+  - [ ] Filter works correctly
+  - [ ] Overview stats look correct
+  - [ ] Vega stimulus for answers looks correct
+- [ ] Coding
+  - [ ] Filter works correctly
+  - [ ] Add text tags
+  - [ ] Add annotations
+  - [ ] Replay works correctly at different play speeds
+  - [ ] Navigate to the next and previous participant IDs
+  - [ ] Add participant tags
+  - [ ] Navigate to the next and previous participant tasks
+  - [ ] Add task tags
+  - [ ] Load transcripts (e.g. `demo-screen-recording`)
+  - [ ] Download audio/screen recordings (e.g. `demo-screen-recording`)
+  - [ ] Reject / Undo reject participant works correctly
+- [ ] Live Monitor
+  - [ ] Filter works correctly
+  - [ ] Check participant count header
+  - [ ] Check live monitor tracker
+- [ ] Config
+  - [ ] Filter works correctly
+  - [ ] Download configs (by selecting from the table) / Download config (single download from actions column)
+  - [ ] Compare configs
+  - [ ] View config
+- [ ] Manage
+  - [ ] Changing ReVISit Modes updates the study card and study browser
+  - [ ] Add a new stage and edit stage color
+  - [ ] Create a data snapshot
+  - [ ] Confirm Manage is only accessible to administrators
+- [ ] File download
+  - [ ] JSON export
+  - [ ] Tidy download export
+  - [ ] Download audio recordings
+  - [ ] Download screen recordings
+  - [ ] Download provenance data
+- [ ] Change study from the study dropdown
+- [ ] Click Go to study and confirm it opens the study
+
+### Generated Files
+
+- [ ] If parser types changed, regenerate the study config schemas using `yarn generate-schemas`
+- [ ] If libraries changed, regenerate library documentation and example studies using `yarn generate-library-docs` and `yarn generate-library-examples`
+
+### Documentation
+
+- [ ] Update comments in `store/types.ts`, `parser/types.ts`, `storage/types.ts`, `storage/engines/types.ts`
+- [ ] Update `typedocReadMe.md`
+- [ ] Verify documentation links in the [Study Repository](https://github.com/revisit-studies/study) are up-to-date and point to the current reVISit documentation pages
+
+## [Documentation Repository](https://github.com/revisit-studies/reVISit-studies.github.io)
+
+- [ ] Update documentation for release changes
+- [ ] Update screenshots
+- [ ] Review documentation for accuracy, typos, outdated content, and expired links
+- [ ] Review [library list](https://revisit.dev/docs/designing-studies/plugin-libraries/)
+- [ ] Check the adoption Google Form for newly published papers
+- [ ] Validate example code
+
+## [ReVISitPy Repository](https://github.com/revisit-studies/revisitpy)
+
+- [ ] If parser types or schemas changed, confirm they are compatible with ReVISitPy code generation
+
+## TODOs
+
+Create or link any release-blocking issues or bugs found during release QA.
+Add any other tasks that must be completed before the release.
+
+- [ ] ...

.github/workflows/dependency-lock-drift.yml

@@ -0,0 +1,52 @@
+name: Dependency lockfile drift
+
+on:
+  pull_request:
+    paths:
+      - '.github/workflows/dependency-lock-drift.yml'
+      - '.npmrc'
+      - 'package.json'
+      - 'package-lock.json'
+      - 'scripts/check-lockfile-drift.mjs'
+      - 'yarn.lock'
+  push:
+    branches:
+      - dev
+      - main
+    paths:
+      - '.github/workflows/dependency-lock-drift.yml'
+      - '.npmrc'
+      - 'package.json'
+      - 'package-lock.json'
+      - 'scripts/check-lockfile-drift.mjs'
+      - 'yarn.lock'
+
+jobs:
+  check:
+    name: Check npm lock matches Yarn lock
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '24'
+
+      - name: Use the lockfile generator version
+        run: npm install --global [email protected]
+
+      - name: Check npm lockfile metadata
+        run: |
+          cp yarn.lock "$RUNNER_TEMP/yarn.lock"
+          npm install --package-lock-only --ignore-scripts
+          cp "$RUNNER_TEMP/yarn.lock" yarn.lock
+          git diff --exit-code -- package-lock.json
+
+      - name: Check for dependency drift
+        run: node scripts/check-lockfile-drift.mjs
+
+      - name: Verify npm install and build
+        run: |
+          npm ci
+          npm run build

.github/workflows/lint_build.yaml

@@ -11,4 +11,20 @@ on:
 
 jobs:
   lint:
-    uses: revisit-studies/.github/.github/workflows/lint.yaml@main
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: lts/*
+          cache: 'yarn'
+
+      - name: Install yarn packages (from cache if available)
+        run: yarn install --frozen-lockfile
+
+      - name: Run linter
+        run: yarn lint
+
+      - name: Build application
+        run: yarn build

.github/workflows/playwright.yml

@@ -10,7 +10,7 @@ jobs:
   test:
     name: 🧪 ${{ matrix.project }} E2E Tests
     runs-on: ${{ matrix.os }}
-    timeout-minutes: 20
+    timeout-minutes: 40
     env:
       PLAYWRIGHT_BROWSERS_PATH: ${{ matrix.cache_dir }}
       PW_CI_REPORTER: dot
@@ -55,9 +55,13 @@ jobs:
           path: ${{ matrix.cache_dir }}
           key: ${{ runner.os }}-${{ matrix.project }}-pw-${{ hashFiles('**/.playwright-version') }}
 
-      - name: Install Playwright Browsers
-        if: steps.playwright-cache.outputs.cache-hit != 'true'
-        run: yarn playwright install --with-deps ${{ matrix.project }}
+      - name: Install Playwright Chromium
+        if: steps.playwright-cache.outputs.cache-hit != 'true' && matrix.project == 'chromium'
+        run: yarn playwright install --with-deps --only-shell chromium
+
+      - name: Install Playwright WebKit
+        if: steps.playwright-cache.outputs.cache-hit != 'true' && matrix.project == 'webkit'
+        run: yarn playwright install webkit
 
       - name: Run Playwright tests
         run: yarn test --project=${{ matrix.project }}

.npmrc

@@ -0,0 +1 @@
+legacy-peer-deps=true

AGENTS.md

@@ -27,14 +27,17 @@ The study configurations are defined in JSON files with schemas. These configs s
 Imported Libraries
 We provide several libraries to facilitate the development of user studies, such as vlat, mini-vlat, nasa-tlx, color-blindness, and more. These libraries are components and sequences that are defined in public/libraries/. Each library has its own folder containing the necessary code and assets. Use only libraries that exist under public/libraries/ (no ad-hoc local component imports in study configs). You can import these libraries into your study configs to enhance the functionality and user experience of your studies by adding the libraries to the top level `importedLibraries` field in the study config, and then referencing the components in the `baseComponent` field or in sequences and the sequences in the `sequences` field. When referencing the components and sequences use the following syntax `"$libraryName.components.componentName"` and `"$libraryName.sequences.sequenceName"` respectively.
 
+Large Media Assets
+Large bundled-library media should not be committed directly to this repository. Put heavyweight shared library assets in the public `revisit-studies/library-assets` repository, tag the asset release, and reference files from study/library configs with tag-pinned raw GitHub URLs such as `https://raw.githubusercontent.com/revisit-studies/library-assets/v1/...`.
+
 Storage Engines
 ReVISit supports multiple storage backends for storing study data, including Firebase and Supabase. This allows researchers to choose the backend that best fits their needs in terms of scalability, ease of use, and integration with other tools. The storage engines are abstracted in the codebase, allowing for easy addition of new storage solutions in the future. The storage engine implementations can be found in src/storage/ directory.
 
 How you should interact with the codebase
-When working with the ReVISit codebase, work only with the source code files available to you. If you need an external library, please ask for approval first (and include how well used the library is). Make sure to follow best practices for React and TypeScript development, including proper state management, component structuring, and code documentation. Pay extra attention to lifecycle methods and hooks to ensure optimal performance and avoid memory leaks, including any updates to existing code. If you encounter any issues or have suggestions for improvements, feel free to bring them up for discussion. You can run git commands but don't run them unless asked to. Don't interact with GitHub directly. Always check package.json for the scripts available to you for building, testing, and running the project.
+When working with the ReVISit codebase, work only with the source code files available to you. If you need an external library, please ask for approval first (and include how well used the library is). Make sure to follow best practices for React and TypeScript development, including proper state management, component structuring, and code documentation. Pay extra attention to lifecycle methods and hooks to ensure optimal performance and avoid memory leaks, including any updates to existing code. If you encounter any issues or have suggestions for improvements, feel free to bring them up for discussion. You can run git commands but don't run them unless asked to. Always check package.json for the scripts available to you for building, testing, and running the project.
 
 Testing
-When adding a new feature or modifying code try to maximize unit test coverage. Unit tests should be colocated with the files they are testing, have the same names as the file with .spec., and use the vitest framework. Apply this to both UI/react code as well as non-UI code. For UI code, we use playwright for end-to-end testing. Try to add e2e tests for any new features that involve user interaction. E2E tests are located in the tests/ directory at the root of the project. Don't run `yarn test` directly; instead, describe the tests you want to run, and I'll handle executing them. You can run unittests locally using `yarn unittest`. Preferred commands are those listed in package.json (e.g., `yarn unittest`, `yarn lint`, `yarn typecheck`, `yarn serve`, `yarn build`).
+When adding a new feature or modifying code try to maximize unit test coverage. Unit tests should live in a sibling `tests/` folder near the code they are testing, keep the same base name as the tested file with `.spec.`, and use the vitest framework. For example, `src/store/hooks/useReplay.ts` should be tested in `src/store/hooks/tests/useReplay.spec.tsx`. Root-level app specs should live in `src/tests/`. Apply this to both UI/react code as well as non-UI code. For UI code, we use playwright for end-to-end testing. Try to add e2e tests for any new features that involve user interaction. E2E tests are located in the `tests/` directory at the repo root. Don't run `yarn test` directly; instead, describe the tests you want to run, and I'll handle executing them. You can run unittests locally using `yarn unittest`. Preferred commands are those listed in package.json (e.g., `yarn unittest`, `yarn lint`, `yarn typecheck`, `yarn serve`, `yarn build`).
 
 Parser
 When adding new features, sometimes it's required to update the parser and the associated types. The parser is located in src/parser/. The parser is responsible for validating and transforming the study config JSON files into a format that the application can use. When updating the parser, ensure that you also update the corresponding types in src/parser/types.ts to reflect any changes made to the study config schema. Make sure to add unit tests for any new parser functionality to ensure its correctness. Additionally, changes to the parser types will require updates to the generated JSON schema files located in src/schemas/. You can regenerate these schema files by running `yarn generate-schemas`.