[wushilin]

Problem

The Go stream load SDK only retried HTTP 429/408/5xx. Doris reports almost every load failure as HTTP 200 + Status: Fail — including transient ones such as [E-235] too many versions, [E-238] too many segments, current running txns on db ... larger than limit, memory limit exceeded and txn/publish timeouts — so none of those were ever retried. The Flink connector (DorisBatchStreamLoad.load()) retries every failed load up to sink.max-retries with a fresh label_N.

Changes

• classifyResponseError: every non-success outcome (non-2xx HTTP, or 2xx whose Status is not Success/Publish Timeout) is now retriable, matching the Flink connector. The HTTP status code is no longer used to guess retriability. The only exception is Label Already Exists + ExistingJobStatus: RUNNING, which stays ambiguous and is resolved by polling get_load_state.
• Config.MaxRetries / LoaderConfig.max_retries: default 3 (same as sink.max-retries). A batch is sent at most MaxRetries+1 times, still bounded by DorisUploadTimeout. -1 disables retries.
• Fresh label per retry: <label>_<attempt>, mirroring Flink's label + "_" + retry.
• streamLoadError.Error() includes ErrorURL (like the Flink error message) and keeps both the wrapper message and the last Doris response message.
• README: new config row, rewritten failure table, "Retry policy" section.

Deliberately kept from the Go SDK: ambiguous outcomes (response lost after the request was sent, or label RUNNING) are resolved via get_load_state before any retry — VISIBLE/COMMITTED is success, ABORTED/UNKNOWN retries — so data is never loaded twice. Flink batch mode re-sends blindly in that situation.

Tests

• Updated TestClassifyResponseError table; new fakeSender tests for MaxRetries, fresh labels, -1, and that ABORTED-poll retries count against the cap.
• New mock_backend_test.go: a scripted mock Doris (_stream_load + get_load_state, dropped TCP connections, FE 307 → BE redirect) driving the real httpSender for each scenario:
  • Fail (too many versions) retried with fresh label, same body, auth kept
  • txn-limit Fail exhausts MaxRetries, last Doris message + ErrorURL surfaced
  • non-2xx retried; MaxRetries=-1 fails fast; DorisUploadTimeout bounds retries
  • connection refused retried without polling
  • dropped response + VISIBLE → success without re-send; ABORTED/UNKNOWN → retried; PREPARE/PRECOMMITTED polled until terminal; poll timeout → failure without re-send
  • Label Already Exists: FINISHED → success, RUNNING → polled (then success or retry), other → retried
  • retry after FE redirect goes back through the FE with a new label and keeps basic auth
  • concurrent batches keep independent retry state

go vet, go test ./... and go test -race ./... pass; the mock tests were run with -count=5 to check for flakiness.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UzpE493uFnJ1UQWx5N55sh