Repository navigation
Replace single-key tracker storage with per-URL stores - #46
Merged
Merged
Conversation
All tracked URLs lived in one serialized array under tracker::urls, so every GET, tracked render and invalidation read (and often rewrote) the whole blob, and concurrent writes lost entries. Storage now goes through a small Store interface with two implementations, picked automatically from the static cache store: - RedisStore: a hash of URL records plus one set per tag, using single-key commands only so it works on Redis Cluster. - CacheStore: any Laravel cache store, with one key per URL, a sharded URL index and locked per-tag buckets whose stale entries are pruned on lookup. Existing data is migrated from tracker::urls automatically on first use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every tracked URL lived in one serialized array under
tracker::urls. On a site with ~11k URLs that's a 13+ MB value, and:has()checkWhat
Storage goes through a small
Storeinterface with two implementations, picked automatically from the static cache store. There's no new config.RedisStore(when the store is Redis): one hash of URL records plus one set per tag. Commands touch a single key, so it works on Redis Cluster, and writes are pipelined.CacheStore(any other Laravel cache store): one key per URL, a URL index split across 256 keys, and one key per tag listing its URLs. Writes to those shared keys are locked. Leftover entries in a tag's list are checked and removed at lookup time.Existing
tracker::urlsdata is moved over automatically on first use, behind a lock so only one process does it.The public
TrackerAPI is unchanged. This also addsTracker::findByTags(), whichGetUrlsControllernow uses instead of scanning everything.Why two drivers: the generic driver alone fixes reads and invalidation. Tracking a page, though, has to lock, read and rewrite the URL list of each tag used on nearly every page (globals, navs, partials). After a global or nav save invalidates the whole site, a re-warm queues every render on those locks. Redis adds a URL to a set (
SADD) in a single call with no lock.Numbers
11,218 URLs, 30 tags each, local Redis (no network time):
has()add()Notes
cache-tracker:listnow sorts by URL, sinceall()order is no longer the order URLs were added.it_doesnt_track_pages_already_in_the_manifest. It tracked/while the middleware trackshttp://localhost, so it only passed because of array ordering.Testing
tests/Unit/StoreTest.phpruns against both drivers. It covers stale tags, wildcards, import, flush and migration. The Redis cases are skipped when Redis isn't available.CACHE_TRACKER_TEST_REDIS=1 vendor/bin/pestruns the whole suite against Redis (it uses db 15).🤖 Generated with Claude Code