Repository navigation
fix(bigtable): remove shadow attemptSettings from MutateRowsAttemptCallable - #15045
igorbernstein2 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the retry mechanism in MutateRowsAttemptCallable by replacing RetryAlgorithm<MutateRowsRequest> with RetryAlgorithm<MutateRowsAttemptResult> and delegating retry decisions to shouldRetryBasedOnResult. It also adds new tests to verify partial success handling and per-call retry settings overrides. The feedback suggests adding a defensive null check for retryAlgorithm in the constructor, appending throws Exception to the new test method signature, and applying a timeout with specific exception handling on the .get() call to prevent tests from hanging.
| this.currentRequest = Preconditions.checkNotNull(originalRequest, "currentRequest"); | ||
| this.callContext = Preconditions.checkNotNull(callContext, "callContext"); | ||
| this.retryableCodes = Preconditions.checkNotNull(retryableCodes, "retryableCodes"); | ||
| this.retryAlgorithm = retryAlgorithm; |
There was a problem hiding this comment.
| } | ||
|
|
||
| @Test | ||
| public void perCallRetrySettingsMaxAttemptsOverrideTest() { |
There was a problem hiding this comment.
Add throws Exception to the method signature to support throwing checked exceptions (such as TimeoutException or InterruptedException) from the .get() call with a timeout.
| public void perCallRetrySettingsMaxAttemptsOverrideTest() { | |
| public void perCallRetrySettingsMaxAttemptsOverrideTest() throws Exception { |
References
- In Java test methods that declare
throws Exception, it is acceptable to letInterruptedExceptionpropagate.
| Throwable actualError = null; | ||
| try { | ||
| retryingCallable.futureCall(request, perCallContext).get(); | ||
| } catch (Throwable t) { | ||
| actualError = t.getCause(); | ||
| } |
There was a problem hiding this comment.
Use a timeout on the .get() call to prevent the test from hanging indefinitely if the asynchronous execution fails or deadlocks. Additionally, catch ExecutionException specifically so that other exceptions (like TimeoutException) can propagate and fail the test clearly.
| Throwable actualError = null; | |
| try { | |
| retryingCallable.futureCall(request, perCallContext).get(); | |
| } catch (Throwable t) { | |
| actualError = t.getCause(); | |
| } | |
| Throwable actualError = null; | |
| try { | |
| retryingCallable.futureCall(request, perCallContext).get(10, java.util.concurrent.TimeUnit.SECONDS); | |
| } catch (ExecutionException t) { | |
| actualError = t.getCause(); | |
| } |
References
- Avoid using very short timeouts (such as 200ms) when waiting for asynchronous events or mock server requests in tests, as they can cause flakiness in busy CI/CD environments. Use a larger timeout (e.g., 2000ms) to ensure reliability.
Summary
MutateRowsAttemptCallablepreviously maintained a shadowthis.attemptSettingsfield initialized viaretryAlgorithm.createFirstAttempt(null)that was only updated inhandleAttemptErrorand never inhandleAttemptSuccess, separate fromexternalFuture.getAttemptSettings()managed by GAX'sBasicRetryingFuture.This caused three issues:
handleAttemptSuccess) were followed by an RPC-level failure (handleAttemptError),this.attemptSettingslagged behind the actual attempt count tracked byexternalFuture.RetrySettingsoverrides ignored on RPC-level errors: Becausethis.attemptSettingswas created and advanced withRetryingContext = null, per-callRetrySettingsoverrides (such as a largermaxAttemptsonApiCallContext) were ignored inhandleAttemptError, prematurely marking entries as permanent failures and aborting retries at the stub's defaultmaxAttempts.isRetryable()and tracer reporting on retry exhaustion: EvaluatingTimedAttemptSettingsinsidehandleAttemptErrorflippedMutateRowsException.isRetryable()tofalseon the final attempt (inconsistent withhandleAttemptSuccessand GAX'sApiException.isRetryable()contract, and causingBasicRetryingFutureto recordtracer.attemptPermanentFailureinstead oftracer.attemptFailedRetriesExhausted).Changes
this.attemptSettingsfield fromMutateRowsAttemptCallable.handleAttemptError, check result-based retryability (retryAlgorithm.getResultAlgorithm()) withcallContext, leavingTimedAttemptSettings(maxAttemptsandtotalTimeout) tracking and enforcement to the parentRetryingFuture.MutateRowsAttemptCallableTestcovering partial entry failure followed by RPC-level failure and per-callRetrySettings(maxAttempts) overrides.