Repository navigation
Distributed lock: one sync per project across instances (Redisson) - #15
Merged
Merged
Conversation
- Redisson client (plain library; its Boot starter targets Boot 3) on the same Redis connection details as the cache, connecting lazily so the app starts without Redis; object names prefixed (resourceapi.redisson.key-prefix) - ProjectSyncLock: per-project RLock taken by POST /sync and released when the background sync ends (held by a random owner id, as it is released on another thread); expires after resourceapi.sync.lock-lease-time (60s); a concurrent sync gets 409; if Redis is unreachable the sync runs unlocked - spring.cache.type=redis: Redisson brings a JCache provider that Boot would otherwise pick over Redis - Tests (real Redis): lock held elsewhere -> 409 and no SDLC call; slow sync -> second refused, then allowed once the first released (verified to fail without the release); abandoned lock expires; keys carry the context's prefix; Redis down -> sync still runs (verified to fail without fail-open) - Backpressure test uses two projects (the lock refuses the same one first) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
- ProjectSyncService.requestSync takes and releases the lock; the async work moves to ProjectSyncWorker, so no caller can start a sync unlocked - A lock request that fails (e.g. times out after Redis applied it) keeps its lease, so the sync's end still tries to release it - Redisson is only created for a standalone Redis; with Sentinel/Cluster the app starts and syncs run unlocked (was: startup failure) - The task executor depends on Redisson, so on shutdown running syncs release their locks before Redisson stops - No Redisson retries (a failed request took 2-3s), small connection pool (defaults were 24 idle / 64 max) - Tests: Sentinel config starts and syncs unlocked; executor shutdown order; SDLC call count checked before the next sync can call; slow-sync test waits for its last sync; backpressure test polls instead of sleeping Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
- RateLimiters: Redisson RRateLimiter token buckets shared by all instances; created with the configured rate, updated when the configuration changed, expiring after an hour unused; requests are allowed when Redis is unreachable - Calls to one SDLC system (resourceapi.rate-limit.sdlc.*, 10/s, waiting up to 1s): only real calls count, not cache hits; exceeded -> 503 + Retry-After (our own limit), logged by a background sync - POST /sync per project (resourceapi.rate-limit.sync.*, 6/min): checked after the 404 and lock checks, so refused requests don't use a permit; exceeded -> 429 + Retry-After - Tests (real Redis): third uncached call refused and not sent, permits shared with another instance and refilled, cache hits don't count, changed rate applied (verified to fail without the update), sync 429 with Retry-After, 409s don't use permits; Redis down still allows (verified to fail without fail-open) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
…ted limits - Redisson's RRateLimiter is a sliding window (a used permit returns one period later): Retry-After is now the full period (was period/permits, so clients retried into another 429); README says sliding window - A limiter is set up once per instance (trySetRate no longer on every call), and again only when Redis dropped it after expiring; it is marked set up only after that succeeded, so a failed update is retried - The set of set-up limiters is bounded (LRU, 10,000) - Rate limit properties are validated at startup (@positive permits, required period and max-wait) - README: a sync refused for a full executor (503) does use a permit - Tests: removed/expired limiter is set up again (verified to fail without it); invalid limits rejected; Retry-After is one full window Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
# Conflicts: # src/main/resources/application.properties
They are deprecated on the server config (logged as warnings at startup). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
…lient - external-projects-missing cache (30s): a project the SDLC system doesn't know is answered from the cache instead of asking again each time, and found soon after it is created there; a sync replaces it with the project. Used directly through the CacheManager, so failures are caught and logged like the cache error handler does - ClientRateLimitFilter on /api/* (FilterRegistrationBean, after the trace filter): requests per client IP across all instances (100/s, 429 + Retry-After); server.forward-headers-strategy=native takes the client IP from X-Forwarded-For only from internal proxy addresses - Tests: unknown answer remembered then found after expiry, sync clears it; per-client 429 with Retry-After and X-Trace-Id, separate per client, non-API paths unlimited; the repeated-lookup test counts calls after the first lookup, as the HTTP client may retry a GET on a stale connection Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
Startup validation reads the record fields reflectively; the native image failed to start with MissingReflectionRegistrationError on Limit.permits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
…ases # Conflicts: # src/main/java/com/company/resourceapi/config/RateLimitProperties.java
…ed 429 - "Unknown project" is cached as Spring's null value in the external-projects cache with a per-entry TTL (30s unknown, 5m found), replacing the separate cache and the hand-written lookups; a sync simply @CachePut-replaces it. - RateLimiters stops asking Redis for 10s after a failure, so an outage doesn't add a timeout to every request. - Limiters expire 10 periods after last use (at least a minute) instead of an hour, so per-IP limiters don't pile up. - The per-client filter answers through the shared exception handler; refused callers are logged at DEBUG. - README: server.tomcat.remoteip.internal-proxies for the per-IP limit. - Tests: longer negative-cache TTL, no timing-sensitive 404 assertion, delta-based call counts, unit test for the outage pause. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012MLyPXGHeCc1fvTeB8JnT2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of three Redis PRs, stacked on #14 (cache). Merge #14 first; GitHub then retargets this PR to
main.org.redisson:redisson4.8.0 library. Its Spring Boot starter is built against Boot 3.5 and Boot 4 moved the Redis auto-configuration, so the starter isn't used.DataRedisConnectionDetailsand the same timeouts, so Docker Compose and Testcontainers connections work too.docker build, starts without Redis.resourceapi.redisson.key-prefix.ProjectSyncLock.POST /api/v2/projects/{id}/synctakes a per-projectRLock(project-sync:{id}). A second sync of the same project, from any instance, gets 409 until the first ends.resourceapi.sync.lock-lease-time(60 s) if never released, e.g. when an instance dies.spring.cache.type=redis. Redisson brings a JCache provider (javax.cache:cache-api), and Boot prefers JCache over Redis when choosing the cache. Without this setting, every cache call failed with "Cannot find cache named 'sdlc-systems'".Tests (real Redis)
ProjectSyncLockIntegrationTest:CacheUnavailableTest: with Redis down, a sync still runs and completes.ProjectSyncBackpressureTest: now uses two projects, because the lock refuses the same project before the executor limit applies. It also checks that a rejected sync released its lock.Full suite: 95 tests pass.
Generated by Claude Code