Skip to content

[MeshSync] Fix data race on informer stores map#592

Open
singhharsh1708 wants to merge 2 commits into
meshery:masterfrom
singhharsh1708:fix/meshsync-stores-data-race
Open

[MeshSync] Fix data race on informer stores map#592
singhharsh1708 wants to merge 2 commits into
meshery:masterfrom
singhharsh1708:fix/meshsync-stores-data-race

Conversation

@singhharsh1708

@singhharsh1708 singhharsh1708 commented Jul 16, 2026

Copy link
Copy Markdown

Motivation

Handler.stores (the per-GVR informer store set) is replaced wholesale by startDiscovery on the discovery goroutine on every (re)discovery, and read by handleInformerStoreRequestlistStoreObjects on the request-listener goroutine when Meshery Server issues an informer-store request. These run concurrently with no synchronization, so the map field is read and written without a happens-before relationship — a data race (go test -race flags it). This is the same class of bug already fixed for the exec/log-stream sessions map.

Change

  • Add storesMu sync.RWMutex guarding the stores field.
  • replaceStores performs the guarded wholesale swap; snapshotStores copies the current store values under the read lock and returns them, so the lock is never held across cache.Store.List().
  • listStoreObjects iterates snapshotStores() instead of the raw map.

No exported API, wire format, CRD, or config changes. Access is read-mostly, hence RWMutex; the set of objects returned is unchanged.

Testing

  • New meshsync/stores_test.go: TestStoresConcurrentAccess (16 goroutines racing replace/snapshot) passes under -race and reports a data race when the guard is removed; plus completeness and nil-map cases.
  • go test --short ./... -race is green across all packages; go vet and gofmt clean.

Roadmap fit

Prerequisite for the periodic-reconciliation blueprint (docs/design/fd5-periodic-reconciliation.md), which adds a timer-driven reconcile loop that becomes the first concurrent reader of h.stores alongside the existing informer-store path. Submitted as a standalone fix so the reconcile feature builds on a race-free store, keeping the bug fix separate from the feature.

Summary by CodeRabbit

  • Bug Fixes

    • Improved reliability when discovery updates resource stores while requests are being handled.
    • Prevented potential concurrency issues and failures when retrieving stored resources, including when no stores are available.
  • Tests

    • Added coverage for concurrent store updates, complete store snapshots, and empty-store scenarios.

startDiscovery replaces h.stores wholesale on the discovery goroutine while handleInformerStoreRequest ranges it on the request-listener goroutine, an unsynchronized read/write of the map field. Guard it with storesMu (RWMutex) and route both accesses through replaceStores and snapshotStores, mirroring the sessions-map fix. snapshotStores copies the store slice under the read lock and releases it before any store List(), so a store read never blocks the next discovery swap.

Signed-off-by: Harsh Singh <[email protected]>
@github-actions github-actions Bot added the language/go Golang related label Jul 16, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces thread-safe access to the 'stores' map in the 'Handler' struct by adding a 'sync.RWMutex' ('storesMu') and implementing 'replaceStores' and 'snapshotStores' helper methods. These changes prevent race conditions between the discovery goroutine, which updates the stores, and the request listener goroutine, which reads them. Additionally, a new test file 'meshsync/stores_test.go' has been added to verify concurrent access and correct snapshot behavior. I have no further feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5e6d680b-e762-42ff-84b7-b87cc968539e

📥 Commits

Reviewing files that changed from the base of the PR and between 310ccd8 and 3b33aae.

📒 Files selected for processing (4)
  • meshsync/discovery.go
  • meshsync/handlers.go
  • meshsync/meshsync.go
  • meshsync/stores_test.go

📝 Walkthrough

Walkthrough

The handler now protects discovery store replacement with an RWMutex, snapshots stores before listing objects, and adds tests for concurrent access, complete snapshots, and nil store maps.

Changes

Store concurrency

Layer / File(s) Summary
Store swap and snapshot path
meshsync/meshsync.go, meshsync/discovery.go, meshsync/handlers.go
Handler adds storesMu; discovery replaces stores under a write lock, and list handling reads a copied store snapshot under a read lock.
Concurrency and snapshot validation
meshsync/stores_test.go
Tests cover concurrent replacement and reads, complete snapshots, and nil-map snapshots.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing a data race on the informer stores map.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

language/go Golang related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant