fix(health): paginate reads + downsample/bucket query_health_data (Fixes #536) - #538
Conversation
#536) High-volume record types (e.g. BloodGlucose from a Dexcom CGM, ~288 records/day) crashed query_health_data with a generic tool error. The read built ReadRecordsRequest with no pageSize, so Health Connect returned a full ~1000-record page in a single Binder transaction that exceeded the ~1MB limit and threw before serialization. This lands three related changes on a shared paginated read: 1. Paginated read (crash fix). HealthConnectClientRecordReader now sets a bounded pageSize (READ_PAGE_SIZE=1000) and loops over response.pageToken until exhausted, with a MAX_RECORDS=50_000 ceiling to bound memory. read's signature is unchanged; a PageFetcher seam makes the loop unit-testable. 2. Downsampled raw mode (default). Repository.queryRecords now returns a QueryResult(records, totalCount, downsampled). When mapped records exceed the cap (limit if given, else DEFAULT_MAX_RECORDS=500) it evenly downsamples across the whole range (first and last retained) rather than trimming one end, and the tool response gains total_count and downsampled. Real record objects are preserved, so the wire contract is unchanged. 3. Bucketed mode (new bucket param). query_health_data takes an optional bucket like 1m/5m/1h/1d. Bucketing is guarded BEFORE reading: if the expected bucket count exceeds MAX_BUCKETS=1000 it errors with the count and limit. It applies only to instantaneous scalar types via a per-type bucketValues extractor colocated with the existing mappers in VitalsQueries and BodyQueries (BloodGlucose, HeartRate, HRV, RestingHeartRate, RespiratoryRate, OxygenSaturation, Body/BasalBody temperature, Weight, Height, BodyFat, BoneMass, LeanBodyMass, Vo2Max); session/multi-value/ cumulative types (SleepSession, ExerciseSession, BloodPressure, SkinTemperature, Steps, etc.) return a clear error. Per non-empty window it emits {start, count, min, max, avg}. Tool description updated to document all three behaviors. This is an MCP tool-behavior change, not an app-UI change, so no docs/ux-decisions.md entry. Co-Authored-By: Claude Opus 4.8 <[email protected]>
monkopedia-reviewer
left a comment
There was a problem hiding this comment.
Fixes #536 with the agreed design. Verified: paginated HC read (pageSize+pageToken, MAX_RECORDS ceiling) removes the Binder-overflow crash; default query downsamples via downsampleEvenly (spread across the range, first+last retained) with total_count/downsampled; getSummary stays un-downsampled (reads category.query directly); bucket mode parses {s,m,h,d}, guards expected-bucket-count > MAX_BUCKETS with an error BEFORE reading, aggregates scalar types to {start,count,min,max,avg} skipping empty windows, and errors clearly on non-scalar/session/cumulative types + bad specs. Per-type scalar extractor placed on CategoryQueries (defaulted null) next to the mappers — clean. Comprehensive tests (paginate, downsample cap+spread, bucket happy/guard/non-bucketable/bad-format). integrations+app tests, builds, ktlint/detekt green. MCP tool-behavior change, no ux-decisions entry needed. Approving; merge on green.
Carries #538 (health reads paginate + bucket/downsample, fixes #536) and #543 (HTML Fastlane description) into a release, so the F-Droid recipe can point `commit:` at a real release tag instead of a side commit off v1.0.7. Adds the versionCode 9 changelog and syncs fdroid/com.rousecontext.yml to 1.0.8/vc9. Also records that the submitted fdroiddata copy carries no MaintainerNotes field, at the maintainer's request on fdroiddata!42096. Claude-Session: https://claude.ai/code/session_01YYSPuh9bZEJkrzvnEpY1QK Co-authored-by: Monkopedia <[email protected]> Co-authored-by: Claude Opus 5 <[email protected]>
Problem
query_health_datacrashed with a generic "Error occurred during tool execution" on high-volume record types (reported forBloodGlucoseon a device with a Dexcom G7 CGM, ~288 records/day). The read builtReadRecordsRequestwith nopageSize, so Health Connect returned a full ~1000-record page in a single Binder transaction that exceeded the ~1MB limit and threw before serialization. The app-sidelimitwas applied post-read, so it never helped.Fixes #536.
Changes (three parts on a shared paginated read)
1. Paginated read (the crash fix)
HealthConnectClientRecordReader.readnow sets a boundedpageSize(READ_PAGE_SIZE = 1000) and loops overresponse.pageTokenuntil exhausted, accumulating records with aMAX_RECORDS = 50_000ceiling to bound memory.read(type, from, to)'s signature is unchanged. APageFetcherseam makes the loop unit-testable without a real Health Connect client.2. Downsampled raw mode (default)
HealthConnectRepository.queryRecordsnow returnsQueryResult(records, totalCount, downsampled). When mapped records exceed the cap (limitif provided, elseDEFAULT_MAX_RECORDS = 500) it evenly downsamples across the whole range (first and last retained) rather than trimming one end. Real record objects are preserved, so the per-record wire shape is unchanged; the response object gainstotal_count(pre-downsample) anddownsampled(bool).New response shape (raw):
{ "record_type": "BloodGlucose", "count": 500, "total_count": 8640, "downsampled": true, "records": [ ... ] }3. Bucketed mode (new
bucketparam)query_health_datagains an optionalbucketstring (1m,5m,1h,1d;<int><unit>, unit ∈ s/m/h/d — bad formats error clearly).ceil((to-from)/bucket)exceedsMAX_BUCKETS = 1000, returns aToolResult.Errorwith the count + max, before reading.CategoryQueries.bucketValues(...)override, colocated with the existing per-type mappers inVitalsQueriesandBodyQueries(BloodGlucose, HeartRate, HRV, RestingHeartRate, RespiratoryRate, OxygenSaturation, Body/BasalBody temperature, Weight, Height, BodyFat, BoneMass, LeanBodyMass, Vo2Max). Non-scalar/session/cumulative types (SleepSession, ExerciseSession, BloodPressure, SkinTemperature, Steps, …) return a clear "bucketing not supported" error.{start, count, min, max, avg}.New response shape (bucketed):
{ "record_type": "BloodGlucose", "bucket": "1h", "buckets": [ { "start": "...", "count": 12, "min": 4.8, "max": 7.2, "avg": 6.1 }, ... ] }The
QueryHealthDataTool.descriptiondocuments all three behaviors. This is an MCP tool-behavior change, not an app-UI/onboarding change, so nodocs/ux-decisions.mdentry.Design notes
bucketValuesmethod onCategoryQueries(default returnsnull= not bucketable), overridden inVitalsQueries/BodyQueriesnext to the existing JSON mappers — keeps per-type knowledge in one place while the generic bucket math (bucketize) and guard live in the repository.queryRecords, not the categoryquery(which is also used bygetSummaryand must stay un-downsampled).Tests
New/extended unit tests (all green):
PaginatedReaderTest— multi-page accumulation, boundedpageSize,MAX_RECORDSceiling.BucketAggregationTest—bucketizecount/min/max/avg, empty-window skipping;downsampleEvenlycap/first+last/spread.RealHealthConnectRepositoryTest— downsample under/over cap + default cap; bucket aggregation; too-fine-bucket guard (asserts nothing read); non-bucketable type error.VitalsQueriesTest—bucketValuesextraction and null for non-bucketable vitals.HealthConnectMcpServerTest—total_count/downsampledfields, downsample flag, bucket happy path (+ duration threaded through), repo-error passthrough, bad-format rejection without reading.:integrations:testDebugUnitTest,:integrations:assembleDebug,:app:assembleDebug,:app:testDebugUnitTest --tests "*Health*",ktlintCheck, anddetektall pass.🤖 Generated with Claude Code