Skip to content

fix(search): guard Searcher index against concurrent rebuild double-close#8

Open
walcz-de wants to merge 1 commit into
mudler:mainfrom
walcz-de:fix/searcher-concurrent-rebuild-double-close
Open

fix(search): guard Searcher index against concurrent rebuild double-close#8
walcz-de wants to merge 1 commit into
mudler:mainfrom
walcz-de:fix/searcher-concurrent-rebuild-double-close

Conversation

@walcz-de

Copy link
Copy Markdown

Problem

Searcher.IndexSkills closes and recreates the bleve index without any synchronization:

func (s *Searcher) IndexSkills(skills []Skill) error {
    s.index.Close()            // <- no lock
    os.RemoveAll(s.indexPath)
    index, err := bleve.New(s.indexPath, mapping)
    s.index = index
    ...
}

GitSyncer triggers a re-index via the onUpdate callback (RebuildIndexIndexSkills), and re-index is driven per configured repo. When two rebuilds overlap (e.g. multiple git repos syncing, or a periodic sync racing a manual one), the second s.index.Close() closes an already-closed index and the process panics:

panic: close of closed channel
    .../pkg/git/syncer.go:177   (onUpdate callback)

This is not just an internal error — it crashes the host process. We hit it downstream in LocalAI, where the periodic skill git-sync took down the whole LLM server.

Search has the mirror race: it reads/uses s.index while IndexSkills swaps and closes it → search failed: index is closed.

Fix

Add a sync.RWMutex to Searcher:

  • IndexSkills and Close take the write lock.
  • Search takes the read lock, held across index.Search so a concurrent rebuild cannot Close the index mid-search.

Minimal and surgical — no behavioural change on the single-threaded path.

Test

Adds a -race regression test (pkg/domain/search_test.go) that runs 8 concurrent IndexSkills calls plus interleaved searches. It panics with "close of closed channel" without the fix and passes with it (verified under go test -race).

…lose

IndexSkills closes and recreates the bleve index without synchronization.
The GitSyncer starts one goroutine per configured repo, each calling
RebuildIndex -> IndexSkills. When two rebuilds overlap, the second
s.index.Close() closes an already-closed index and panics with
"close of closed channel", crashing the host process (observed in LocalAI's
periodic skill git-sync taking down the whole LLM server).

Add a sync.RWMutex to Searcher: IndexSkills and Close take the write lock,
Search takes the read lock (held across index.Search so a concurrent rebuild
cannot Close the index mid-search). Includes a -race regression test that
panics without the lock and passes with it.

Signed-off-by: Stefan Walcz <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TudBpPAvgnGgWpYwoe7ZZs
walcz-de added a commit to walcz-de/LocalAI that referenced this pull request Jul 11, 2026
…nt-rebuild double-close fix)

Pin github.com/mudler/skillserver to our fork carrying the Searcher RWMutex
fix (walcz-de/skillserver@62ca218). Without it, concurrent RebuildIndex ->
IndexSkills calls (one GitSyncer goroutine per configured skill git-repo)
double-close the bleve index and panic "close of closed channel", crashing
the whole LocalAI server — this took the stack down on 2026-07-11 05:20.

Same replace-directive pattern as the LocalAGI override. Upstream PR:
mudler/skillserver#8. Once merged upstream and LocalAI bumps its skillserver
pin, this replace can be dropped.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TudBpPAvgnGgWpYwoe7ZZs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant