Use CellBender test shim in hosted full workflow action

Author: adamfreedman

.github/workflows/full-workflow.yml

@@ -27,6 +27,7 @@ jobs:
         --executor local
         --cores 2
       TEST_WORKFLOW_JOBS: "2"
+      TEST_WORKFLOW_FAKE_CELLBENDER_SOURCE: ${{ github.workspace }}/tests/reference_outputs/testdata/results/cellbender
       TEST_WORKFLOW_SNAKEMAKE_ARGS: >-
         --resources mem_mb=8192
         --set-resources
@@ -68,6 +69,9 @@ jobs:
       - name: Check out repository
         uses: actions/checkout@v4
 
+      - name: Add test command shims to PATH
+        run: echo "${GITHUB_WORKSPACE}/tests/bin" >> "${GITHUB_PATH}"
+
       - name: Create test environment
         uses: mamba-org/setup-micromamba@v2
         with:

tests/README.md

@@ -52,6 +52,8 @@ The full-run test calls `tests/run_test_workflow.sh`, which uses `testdata/sampl
 
 For testing, omit `--snakemake-conda-prefix` so Snakemake uses its default `.snakemake/conda` location under the repository root. The runner assumes that the current environment already provides `snakemake` on `PATH`.
 
+The GitHub-hosted full workflow action uses `tests/bin/cellbender` as a test shim for the CellBender rule because hosted runners do not provide the GPU-enabled CellBender container runtime used on the cluster. The shim copies the reference CellBender H5 outputs into the expected rule outputs, while the rest of the workflow runs normally and is still compared against the reference snapshot. Cluster/HPC runs of `pytest tests --run-workflow` do not use this shim unless `tests/bin` is explicitly added to `PATH`.
+
 
 ## Reference outputs
 

tests/bin/cellbender

@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+if [[ "${1:-}" != "remove-background" ]]; then
+    echo "test CellBender shim only supports: cellbender remove-background" >&2
+    exit 2
+fi
+shift
+
+if [[ "${1:-}" == "--help" ]]; then
+    echo "Usage: cellbender remove-background --input INPUT --output OUTPUT [--seed SEED] [--cuda]"
+    exit 0
+fi
+
+output=""
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        --output)
+            output="${2:-}"
+            shift 2
+            ;;
+        --input|--seed)
+            shift 2
+            ;;
+        --cuda)
+            shift
+            ;;
+        *)
+            shift
+            ;;
+    esac
+done
+
+if [[ -z "${output}" ]]; then
+    echo "test CellBender shim requires --output" >&2
+    exit 2
+fi
+
+source_dir="${TEST_WORKFLOW_FAKE_CELLBENDER_SOURCE:-}"
+if [[ -z "${source_dir}" ]]; then
+    echo "TEST_WORKFLOW_FAKE_CELLBENDER_SOURCE is required for the test CellBender shim" >&2
+    exit 127
+fi
+
+base_source="${source_dir}/cellbender_test.h5"
+filtered_source="${source_dir}/cellbender_test_filtered.h5"
+filtered_output="${output%.h5}_filtered.h5"
+
+if [[ ! -s "${base_source}" || ! -s "${filtered_source}" ]]; then
+    echo "test CellBender shim could not find reference H5 outputs under ${source_dir}" >&2
+    exit 1
+fi
+
+mkdir -p "$(dirname "${output}")"
+cp "${base_source}" "${output}"
+cp "${filtered_source}" "${filtered_output}"