Skip to content

[Code health] Update integration test config to reduce flakiness #3807

Description

@gino-m

From GitHub job analysis in failing test https://github.com/google/ground-android/actions/runs/28511787637/job/84533015752?pr=3806

Problem Analysis

The job is failing during Android instrumented tests with a critical error:

Failed to install split APK(s): [/home/runner/work/ground-android/ground-android/app/build/outputs/apk/local/debug/app-local-debug.apk]
Failed to install-write all apks
com.android.ddmlib.InstallException: Failed to install-write all apks

The root cause is an Android emulator connectivity/stability issue. The emulator becomes unresponsive during APK installation, as evidenced by:

  1. Shell command timeout: ShellCommandUnresponsiveException when trying to install the APK
  2. Emulator device going offline: adb: device offline appears in the logs
  3. Console connection failures: [EmulatorConsole]: Failed to start Emulator console for 5554

Solution

The issue stems from insufficient resource allocation or emulator instability in the CI environment. Here are the recommended fixes:

1. Increase Emulator Stability Timeout

The emulator needs more time to stabilize before APK installation. Modify .github/workflows/test-e2e.yml to add retry logic and extended timeouts:

- name: Wait for emulator to be ready
  run: |
    MAX_TRIES=120  # Increased from current value
    ATTEMPT=0
    while [ $ATTEMPT -lt $MAX_TRIES ]; do
      if adb -s emulator-5554 shell getprop sys.boot_completed | grep -q '1'; then
        echo "Emulator fully booted"
        sleep 10  # Additional stabilization time
        break
      fi
      echo "Waiting for emulator... ($((ATTEMPT+1))/$MAX_TRIES)"
      sleep 2
      ((ATTEMPT++))
    done
    
    if [ $ATTEMPT -eq $MAX_TRIES ]; then
      echo "Emulator failed to boot"
      exit 1
    fi

2. Add APK Installation Retry Logic

Wrap the test execution with retry logic in your Gradle configuration:

// In your build.gradle or build.gradle.kts
android {
    testOptions {
        execution 'ANDROIDX_TEST_ORCHESTRATOR'
        unitTests {
            includeAndroidResources = true
        }
    }
}

3. Reduce Test APK Size

The split APK installation is failing. Consider:

  • Disabling multi-APK splits for CI builds
  • Reducing dex method count using R8/ProGuard optimization
android {
    buildTypes {
        debug {
            // Optimize for CI environment
            minifyEnabled true
            useProguard false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
}

4. Increase CI Runner Resources (if available)

Update .github/workflows/test-e2e.yml to use more robust hardware:

runs-on: ubuntu-latest-xl  # Use larger runner if available

5. Add Emulator Health Checks Before Testing

Insert a verification step before running tests:

- name: Verify emulator health before testing
  run: |
    adb -s emulator-5554 shell getprop ro.bootloader
    adb -s emulator-5554 shell getprop ro.hardware
    adb -s emulator-5554 shell dumpsys meminfo | head -20

Immediate Action

The most effective short-term solution is implementing APK installation retry logic with exponential backoff, combined with extended emulator boot validation to ensure full stability before attempting installation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: code healthImprovements to readability or robustness of codebase

    Type

    Fields

    No fields configured for Task.

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions