Skip to content

fix: prevent removed external sources from being refreshed - #59

Open
jihtsan wants to merge 1 commit into
mudler:mainfrom
jihtsan:codex/32-stop-removed-source-refresh
Open

jihtsan wants to merge 1 commit into
mudler:mainfrom
jihtsan:codex/32-stop-removed-source-refresh

Conversation

@jihtsan

@jihtsan jihtsan commented Nov 11, 2026

Copy link
Copy Markdown

A source refresh could finish fetching after DELETE returned and put the removed file and vectors back. Re-registering the same collection could also schedule duplicate refreshes.

Make collection registration idempotent and check that a refresh still belongs to the same source before storing. A collection lock orders refresh writes and removals without blocking source operations in other collections.

Closes #32.

Validation:

  • Three deterministic regressions cover a late fetch after deletion, duplicate scheduling, and deletion in one collection while another collection's write is blocked. Ran them 10 times with the race detector. The late-fetch test uses a real local Chromem store and a stub embeddings endpoint.
  • go test -race -count=1 ./rag ./rag/engine -run 'TestSource|TestMemoryChromem' and go vet ./rag ./rag/engine . passed.
  • go test -count=1 ./... reports the same failures on the clean base and this change: unavailable LocalAI/PostgreSQL services and pre-existing function-signature errors in test/raw. No real model was used, and this change does not coordinate source deletion across server replicas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DELETE /api/collections/name/sources residuals

1 participant