[Load] Preserve pending tablets when parser fails during fallback - #18487
Open
Caideyipi wants to merge 1 commit into
Open
[Load] Preserve pending tablets when parser fails during fallback#18487Caideyipi wants to merge 1 commit into
Caideyipi wants to merge 1 commit into
Conversation
Member
|
The idea is reasonable: flush pending tablets before reporting a parser failure so already-converted data isn’t lost. Two things need confirmation though: 1) In the synchronous pipe path, a non-success result is marked retryable by the caller, but this change clears the conversion context after flushing, so a retry can restart from the beginning and re-insert the same tablets; 2) this also affects ordinary LOAD, where a parser failure would now become partial write + error. Consider limiting the flush to receiver-owned/terminal tasks, or mark the task terminal after flushing, and add coverage for the managed path and for retries not duplicating writes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When tree-model Load falls back from scan parsing to query parsing, a parser exception can occur after earlier Tablets have been converted but before the pending batch reaches its normal flush boundary. The exception path released the conversion context and discarded that batch.
Fix
Flush pending Tablets before returning a normal parser error. Temporary-unavailable results retain the pending batch and original parser status for the next managed-task retry; memory-pressure and interruption exceptions keep their existing retry behavior.
Tests
mvn -pl iotdb-core/datanode -am test -Dtest=LoadTreeStatementDataTypeConvertExecutionVisitorTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false -DskipITsThe separate exact-query versus wildcard-query observation is intentionally out of scope.