[floatdrop]

What

A fifth cache, tinylfu. A new entry waits in a small queue, as in S3-FIFO. When its turn comes, it gets into the main queue only if a frequency sketch has its key down as asked for more often of late than the entry that the main queue would evict next. The main queue gives an entry that was read another round, up to three in a row. The size of the small queue is found by Caffeine's hill climber.

Reads take no lock, as in s3fifo and sieve:

• a read counts on its entry, up to three, and tallies a hit in the slot that it pins with, in a cache line that it writes to anyway;
• only writers touch the sketch, and they pass on what readers counted when they come by an entry;
• the climber goes by the hits from the slots and the misses that the shards hand on, so it measures the share of requests answered exactly, for the whole cache.

Commits

1. internal/cslab: Lookup can tally hits in the reader's slot; Readers.Hits adds them up. s3fifo and sieve pass false.
2. internal/sketch: the count-min sketch of TinyLFU in Caffeine's layout.
3. tinylfu: the cache, the climber, a reference model, scenario tests, a fuzz target, and a load test of the climber under the race detector.
4. bench: tinylfu joins the benchmarks, and all reports are made again; README and the docs that quote them follow the new numbers.

Results (bench/results, Apple M3 Max)

	tinylfu	otter	s3fifo
mean hit ratio over the 11 traces	47.9%	47.5%	44.8%
furthest behind the best cache in any of the 88 rows	5.3 points	5.9	63
rows won outright	25	1	32

• tinylfu is ahead of otter on every replay in 42 rows and behind in 14.
• Its hit ratio varies between replays, by up to 5.6 points, because its sketch hashes with a random seed, so it is replayed 11 times, like otter.
• It reads within 6% of s3fifo at 1, 2 and 8 CPUs, and reads through faster with 8, because it misses less.
• Writes that evict get 5 to 10% less done than in s3fifo.
• Memory is about that of s3fifo, and it gives the garbage collector just as little to do.

What speaks against it, all in the README:

• It was designed on these 11 traces. On 14 others, in a simulation of the policy, it was level with S3-FIFO and W-TinyLFU.
• On Mixed at the larger capacities it is up to 4 points behind lru.
• Where every key comes back after one fixed while and never again, the climber has no slope, and it answers far less than lru (6.6% against 66.7% in a synthetic test).
• With 8 CPUs and 1% writes it was 22% slower than s3fifo. Taking out everything it does differently on that path did not close the gap, and the cause has not been found.

The README now says "If in doubt, use tinylfu" (it said s3fifo). s3fifo stays the advice for the same results on every run and for workloads without loops.

Checked

• make lint, make test (race detector), go vet and test compilation for linux/386; every commit builds and passes on its own.
• The cache is compared operation by operation with a model of the policy (which also follows every move of the climber), and fuzzed against it.
• Mutation testing: 120 mutants of the policy, the climber, the window and the slab's tally. 117 are killed; two survivors are equivalent; the last one, Clear without the window's lock, is caught by the race detector.
• Three independent reviews:
  • The first found the climber's hit-rate signal biased on skewed workloads. It used read counts capped at 15 and drifted to a large small queue: 93.9% instead of 95.5% on Zipf 1.4. That led to counting hits exactly; the drift is gone, and the climber now ends within 0.2 points of the best fixed size on those workloads.
  • The second found a biased first measurement after Clear with batched shards; fixed, and tested.
  • The third fact-checked the README and docs against the result files; its ten corrections are in.
• The library's hit ratios agree with an independent simulator of the policy in all 88 rows.

🤖 Generated with Claude Code