Repository navigation
test: [SDK-5481] add a fault-injection harness for request retries - #1776
Conversation
There was a problem hiding this comment.
Multi-model review
Reviewed by Claude Opus 5.5, GPT 5.6 Sol, and Grok 4.7. Test-only harness (762 lines). It models the #1775 executor backoff; against main it is expected that most failed requests wait for the next launch.
Act on
- Unknown
failroute still writes a catch-all rule (Opus, reproduced):read … <<< "$(route_rule …)"swallowsdiebecause the failure is in a subshell. Emptypathplusre.search("", path)matches every SDK request, so a typo likefail update-usr 503prints an error and then fails everything. tapalways succeeds (3/3):maestro | grep … || truehides Maestro failures, so a missed control can look like an SDK retry bug.- No cleanup traps (3/3):
startcan leave the Mac HTTP(S) proxy pointed at a dead port, and a secondstartoverwritessaved-proxywith those redirected settings.network-flapcan leave Wi-Fi off on Ctrl-C.
Consider
--add/ anyrules.jsonmtime rewrite resets in-memory--timescounters (3/3).report.pygroups by stripped method+path, so parallel or successive requests (login b/c, session + add-tag) share one backoff counter (3/3).identity-by-subis listed as a User-executor route, butEXECUTOR_ROUTErequiressubscriptions/[^/]+$, so it never appears inreport(Opus+Grok).- On #1775 the User executor is woken from the 5s poll;
report.pyonly adds poll slack for repo routes, so correct User-executor gaps getLATE(Opus).
Noted
- Bind mitmdump to
127.0.0.1except inmanualmode. - Status
0+--retry-afterrecords a floor the SDK never sees. - Flows hardcode
com.onesignal.examplewhileRETRY_TEST_BUNDLE_IDis documented.
Dismissed
- Missing in-session executor retry on this branch (Grok): expected. This PR is tooling for #1775 and documents current
mainbehavior.
Sent by Cursor Automation: PR Reviews
878ce17 to
0a51989
Compare
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ies by content Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… actions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0a51989 to
8298948
Compare
Reject unknown routes before writing a rule, fail tap when the Maestro flow fails, keep the saved proxy settings across a failed start, restore Wi-Fi when network-flap is interrupted, keep --times counts across --add, report identity-by-sub, and use RETRY_TEST_BUNDLE_ID in the flows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8298948 to
d5274cb
Compare
|
Thanks. Other items from the review:
Also rebased on |
abdulraqeeb33
left a comment
There was a problem hiding this comment.
Requesting changes.
- Re-issuing the same fail rule keeps an exhausted count, so the command injects nothing while the log still shows the rule as armed.
- The attempt after a 400, 401, 403, 404, 409, or 410 is labeled "new attempt after success", and the client retry count is not reset.
Do not subtract slack from the minimum gap. Treat only 200, 201, and 202 as success. A negative or non-finite Retry-After is 60 seconds in the SDK.
…Retry-After rules Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abdulraqeeb33
left a comment
There was a problem hiding this comment.
The three posted items are fixed. Resetting both counters on 401 and 403 is not. A held request keeps its backoff count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abdulraqeeb33
left a comment
There was a problem hiding this comment.
The bearer split is right for routes that drop an unsigned 401. Two leftovers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>


Description
One Line Summary
Add a fault-injection harness to the demo app for watching and checking SDK request retries.
Details
Motivation
Retry and backoff behavior is hard to check by hand: you need specific requests to fail with specific statuses, at the right time, and then measure the gaps between attempts. This harness does that from the demo app, and is what the manual testing in #1775 used. Split out so that PR stays focused on the SDK change.
Scope
Test tooling only, under
examples/demo/scripts/retry, plus a.gitignoreentry for its state. No SDK or demo app code changes.fault_proxy.py: a mitmproxy addon, listening on loopback only, that fails only requests carrying the SDK'sSDK-Versionheader, by path and status (includingRetry-Afterand dropped connections). The demo's own REST calls pass through.retry-test.sh: starts and stops the proxy, installs and resets the demo on its own simulator, sets fault rules, drives the UI through Maestro flows inflows/, and follows SDK logs.report.py: lists each request's attempts and checks the gaps against the expected backoff window.make_jwt.py: signs an ES256 Identity Verification JWT with openssl, claiming the aliases and email/SMS the SDK will add. WithRETRY_TEST_IV_APP_IDandRETRY_TEST_IV_KEYset,tap login <id>andtap update-jwt <id>sign one, andjwtprints one. Ported from the Android harness (test: [SDK-5535] add a fault-injection harness for operation retries OneSignal-Android-SDK#2791).pause-iamsflow: an app's in-app message can otherwise take atap.README.md: setup and scenarios, including an Identity Verification run of the backoff, offline and order scenarios.reportjudges retries by the behavior from #1775, now onmain.Testing
Unit testing
None, test tooling only.
Manual testing
Ran the scenarios in the README on an iOS 26.5 simulator against the #1775 build. Results are in that PR's manual testing section.
Identity Verification: ran the backoff, repo executor, offline and order scenarios, plus a 401 in the middle of retries followed by
update-jwt, against an app that requires it. Results are in #1775.Affected code checklist
Checklist
Overview
Testing
Final pass
🤖 Generated with Claude Code