Skip to content

[Load] Preserve pending tablets when parser fails during fallback - #18487

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/load-preserve-pending-tablets
Open

[Load] Preserve pending tablets when parser fails during fallback#18487
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:fix/load-preserve-pending-tablets

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Problem

When tree-model Load falls back from scan parsing to query parsing, a parser exception can occur after earlier Tablets have been converted but before the pending batch reaches its normal flush boundary. The exception path released the conversion context and discarded that batch.

Fix

Flush pending Tablets before returning a normal parser error. Temporary-unavailable results retain the pending batch and original parser status for the next managed-task retry; memory-pressure and interruption exceptions keep their existing retry behavior.

Tests

  • mvn -pl iotdb-core/datanode -am test -Dtest=LoadTreeStatementDataTypeConvertExecutionVisitorTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false -DskipITs
  • Spotless check
  • Checkstyle: 0 violations

The separate exact-query versus wildcard-query observation is intentionally out of scope.

@luoluoyuyu

Copy link
Copy Markdown
Member

The idea is reasonable: flush pending tablets before reporting a parser failure so already-converted data isn’t lost. Two things need confirmation though: 1) In the synchronous pipe path, a non-success result is marked retryable by the caller, but this change clears the conversion context after flushing, so a retry can restart from the beginning and re-insert the same tablets; 2) this also affects ordinary LOAD, where a parser failure would now become partial write + error. Consider limiting the flush to receiver-owned/terminal tasks, or mark the task terminal after flushing, and add coverage for the managed path and for retries not duplicating writes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants