Skip to content

Parse documents in the worker thread after queued edits are applied - #4261

Open
datanoise wants to merge 1 commit into
Shopify:mainfrom
datanoise:fix-stale-parse-before-queued-edits
Open

datanoise wants to merge 1 commit into
Shopify:mainfrom
datanoise:fix-stale-parse-before-queued-edits

Conversation

@datanoise

Copy link
Copy Markdown

Motivation

Since #3637, text synchronization notifications (didOpen/didChange/didClose) go through the worker queue so they are processed in order with requests. But the document is still parsed in the reader thread in BaseServer#start, as soon as a request is read. If that request arrives while a didChange ahead of it is still waiting in the queue, this happens:

  1. The reader thread runs parse! on the document's old source and sets @needs_parsing = false.
  2. The worker applies the queued edit. This updates the source, sets @needs_parsing = true and clears the cache.
  3. The worker processes the request. Nothing parses again, so the request uses the outdated parse_result. For textDocument/diagnostic, the stale response is also stored with cache_fetch.

Because of the cache, pulling diagnostics again returns the same stale result until the next edit. In practice, a syntax error from Prism stays visible after the code is fixed. It's most noticeable with edits that make the code invalid for a moment and then fix it right away, like adding surrounding delimiters. Clients that send didChange and then pull diagnostics immediately, such as Neovim's built-in pull diagnostics, hit this regularly.

Implementation

  • The reader thread now only converts the textDocument.uri to a URI object.
  • The parse, and the ERB delegate/textDocument/virtualState notification that depends on it, moved into a new private BaseServer#parse_document. The worker calls it in handle_incoming_message, after the cancellation check and right before process_message. Edits and requests are processed in order on the same worker, so the parse always sees every edit received before the request. The parse still runs under @global_state.synchronize.
  • Text synchronization notifications are still never parsed. Unknown documents are still ignored through Store::NonExistingDocumentError.

One change for reviewers: the parse now runs when the worker picks up the request instead of when the message is read. Requests that are cancelled while waiting in the queue are no longer parsed at all.

Automated Tests

Added ServerTest#test_requests_parse_the_document_after_queued_edits_are_applied. It sends didOpen (def foo), a didChange that adds \nend, and a textDocument/diagnostic request through the real Server#start reader loop. It asserts that no diagnostics are returned. Without the fix the test fails every time with expected a delimiter to close the parameters. With the fix it passes.

Manual Tests

The bug depends on timing, so it's easiest to reproduce with a client that pulls diagnostics right after each change:

  1. Open a Ruby file and type def foo on its own, which produces a syntax error.
  2. Quickly add the closing end, or use a surround or auto-pair action that inserts both delimiters together.
  3. Before this change, the Prism syntax error sometimes stays after the code is valid, until the next edit. After this change, diagnostics always match the current buffer.

🤖 Generated with Claude Code

Since text synchronization notifications were moved to the worker queue
(Shopify#3637), the reader thread could parse a document for an incoming request
before the didChange edits queued ahead of it were applied. The worker then
applied the edits (clearing the cache) and ran the request against the
outdated parse result, caching the stale response until the next edit. This
shows up as syntax error diagnostics that persist after the code is fixed.

Move the parse (and the ERB virtual state delegation) into the worker,
right before the request is processed, so it always reflects every edit
received before the request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant