Skip to content

Deadlock in firebase-common heartbeat DataStore path can wedge Firebase Installations getToken()/getId() forever #8016

@Nek-12

Description

@Nek-12

[READ] Step 1: Are you in the right place?

Yes. This appears to be a deadlock bug in the Firebase Android SDK heartbeat storage path.

[REQUIRED] Step 2: Describe your environment

  • Android Studio version: not relevant for the deadlock; reproducible in production app runtime
  • Firebase Component: firebase-common / heartbeat path, with impact surfacing through firebase-installations, firebase-config, firebase-messaging, and firebase-sessions
  • Component version:
    • Firebase BoM: 34.11.0
    • firebase-common: 22.0.1
    • firebase-installations: 19.1.0
    • firebase-config: 23.0.1
    • firebase-messaging: 25.0.1
    • firebase-crashlytics: 20.0.4
    • firebase-perf: 22.0.4

[REQUIRED] Step 3: Describe the problem

We are seeing a deterministic deadlock inside Firebase heartbeat storage. This causes FirebaseInstallations.getToken(false) / getId() to never complete, which in turn makes Firebase Remote Config / Sessions / Messaging startup hang forever.

The deadlock appears to be in the firebase-common heartbeat/DataStore implementation:

  • DefaultHeartBeatController.registerHeartBeat()
  • HeartBeatInfoStorage.storeHeartBeat()
  • JavaDataStorage.editSync()
  • DataStore transform thread re-enters HeartBeatInfoStorage.getStoredUserAgentString() on the same monitor

Steps to reproduce:

In our app this reproduces on fresh install when Firebase is manually initialized via FirebaseApp.initializeApp(...) and then used immediately after init. With provider-based init (FirebaseInitProvider) it does not reproduce in the same deterministic way, which suggests the bug is interleaving-sensitive rather than absent.

The deadlock cycle is:

  1. HeartBeatInfoStorage.storeHeartBeat(...) is synchronized
  2. inside it, SDK calls JavaDataStorage.editSync(...)
  3. JavaDataStorage.editSync(...) uses runBlocking { dataStore.edit { ... } }
  4. the DataStore edit transform invokes HeartBeatInfoStorage.getStoredUserAgentString(...)
  5. getStoredUserAgentString(...) is also synchronized on the same HeartBeatInfoStorage instance
  6. caller thread waits for DataStore transform; DataStore transform waits for the same monitor held by caller thread

Concrete thread dump evidence from a hung process:

Thread A:

  • Firebase Background Thread #3
  • com.google.firebase.datastorage.JavaDataStorage.editSync(JavaDataStorage.kt:207)
  • com.google.firebase.heartbeatinfo.HeartBeatInfoStorage.storeHeartBeat(HeartBeatInfoStorage.java:195)
  • holds com.google.firebase.heartbeatinfo.HeartBeatInfoStorage

Thread B:

  • DefaultDispatcher-worker-2
  • com.google.firebase.heartbeatinfo.HeartBeatInfoStorage.getStoredUserAgentString(HeartBeatInfoStorage.java:133)
  • called from HeartBeatInfoStorage.lambda$storeHeartBeat$2(...:200)
  • called from JavaDataStorage$editSync$1$1.invokeSuspend(JavaDataStorage.kt:220)
  • blocked waiting for the same HeartBeatInfoStorage monitor held by thread A

Upstream impact:

  • FirebaseInstallationServiceClient.openHttpURLConnection() calls Tasks.await(heartBeatController.getHeartBeatsHeader())
  • so getToken(false) / getId() can hang before network I/O even starts
  • higher-level APIs depending on FIS then hang indefinitely

Relevant Code:

Relevant SDK structure from the published sources:

// HeartBeatInfoStorage.java
synchronized void storeHeartBeat(long millis, String userAgentString) {
  firebaseDataStore.editSync(
      (pref) -> {
        Preferences.Key<Set<String>> storedUserAgent =
            getStoredUserAgentString(pref, dateString);
        ...
      });
}

private synchronized Preferences.Key<Set<String>> getStoredUserAgentString(
    MutablePreferences preferences, String dateString) {
  ...
}
// JavaDataStorage.kt
fun editSync(transform: (MutablePreferences) -> Unit): Preferences = runBlocking {
  dataStore.edit { transform(it) }
}

This looks like a self-deadlock in the SDK internals.

Can you confirm and advise on a fix/workaround?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions