Skip to content

fix(SLRU): rounding leak let W-TinyLFU skip many admission checks - #387

Merged
1a1a11a merged 4 commits into
1a1a11a:developfrom
marcft:fix/slru-seg-size-remainder
Nov 14, 2026
Merged

1a1a11a merged 4 commits into
1a1a11a:developfrom
marcft:fix/slru-seg-size-remainder

Conversation

@marcft

@marcft marcft commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Hi, while working on #385 I kept seeing weird spikes in the fixed-window W-TinyLFU curves, and the reason I gave for them in that PR was wrong.

The reason was SLRU rounds both segment down, so W-TinyLFU's main ends up one slot short (unless its size is a multiple of 5). W-TinyLFU sees the free slot and puts the next object without checking the sketch. For example in some few traces, more than half the objects just skipped the sketch check at all. The spikes were the few sizes where the bug doesn't happen.

And this is the plot twist, the bug often actually helps. Fixing it makes default W-TinyLFU worse in 50 of the 76 cells from #385. On cluster26 at the smallest size the miss ratio goes from about 0.26 to 0.47.

I give you here the AI summary ⬇️

What does this PR do?

SLRU_parse_params() rounds each seg-size segment down on its own, so the segments can add up to less than the cache.

  • The leak: W-TinyLFU's main cache is SLRU with seg-size=1:4, so a main of 1,001 objects gets 200 + 800 = 1,000 slots. Once both segments are full, every promotion out of probation evicts inside find(). W-TinyLFU's next miss sees the free slot and admits the window victim without the frequency check. On a typical trace from feat(WTinyLFU): adapt the window size with a hill climber #385 (median of 26, smallest size), 22% of admissions into main skip the check.
  • The fix: call SLRU_resize(cache, cache->cache_size) after parsing. It already gives the remainder to the last segment, so parsing and resizing now share one rule. Add Merlin eviction algorithm and fix capacity remainders in SR-LRU and SLRU #343 fixed the same rounding for the default layout; this covers explicit seg-size.
  • Side effect on feat(WTinyLFU): adapt the window size with a hill climber #385: the hill climber resizes through SLRU_resize(), so it never leaked, while most fixed-window baselines did.
  • Also changes: plain SLRU with an explicit seg-size that doesn't divide the cache (seg-size=1:2:3:4 on 1 GB: 9:19:29:39 becomes 9:19:29:40).

Type of change

  • Bug fix
  • New eviction / admission / prefetch algorithm
  • New trace reader or trace format
  • Performance improvement
  • Documentation
  • Build / CI
  • Other:

How was it tested?

  • New unit test test_SLRU_seg_size_fills_cache: fails on develop (5 == 6), passes with the fix.

  • test/test_cli.sh: 428 passed, 0 failed.

  • ctest --test-dir _build --output-on-failure passes (16/16)

  • Build is warning-free (CI uses -Wall -Wextra -Werror)

Results

All runs: --ignore-obj-size=1 --num-thread=1, Release.

The leak is gone.

./bin/cachesim cluster26.oracleGeneral.sample10.zst oracleGeneral wtinylfu <size> --ignore-obj-size=1 --num-thread=1 -e window-size=0.14
size main % 5 develop this PR
1223 2 (leaks) 0.2547 0.4138
1220 0 (no leak) 0.4143 0.4143

At 1223 this PR matches develop with the segments forced to add up exactly, and sizes that were already exact are untouched (13/13 in #385). Over 208 cells on cluster26 and alibabaBlock_117, the spikes are exactly the cells with main % 5 == 0.

Effect on the 76 cells of #385 (miss ratio, pp; "same" = within 0.05 pp; negative = first one better):

comparison better / same / worse mean
this PR vs develop, fixed 1% window 4 / 22 / 50 +1.10
this PR vs develop, hill climber 10 / 43 / 23 +0.07
climber vs fixed 1%, as published in #385 35 / 9 / 32 −1.58
climber vs fixed 1%, both leak-free (this PR) 54 / 9 / 13 −2.61

Not tested: byte mode with variable object sizes, where SLRU may still overfill probation.

Checklist

  • Formatted with clang-format (or the pre-commit hook from scripts/setup_hooks.sh)
  • Added or updated tests
  • Added or updated documentation
  • New algorithms are registered in the CLI and listed in the README

SLRU_parse_params() rounds each seg-size segment down on its own, so the
segments add up to less than the cache unless the cache size divides
evenly. 1a1a11a#343 fixed this for the default layout but kept explicit
seg-size layouts as they were.

With W-TinyLFU's seg-size=1:4 main cache this is one object short unless
the main size is a multiple of 5. Once every segment is full, a hit in
probation evicts inside find(), and W-TinyLFU then admits its next
window victim without the frequency check.

Call SLRU_resize() after parsing, which gives the remainder to the last
segment. The seg-size=1:2:3:4 print in test_cli.sh changes from
9:19:29:39 to 9:19:29:40.
@marcft
marcft requested a review from haochengxia as a code owner November 13, 2026 21:37
Copilot AI balanced review requested due to automatic review settings November 13, 2026 21:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@1a1a11a 1a1a11a left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a clean fix and a really useful write-up. Reusing SLRU_resize() makes parsing and the hill climber use the same rounding rule, which is the right call. It's also safe to call that early: SLRU_resize (SLRU.c:525-534) only touches lru_max_n_bytes, seg_ratio and cache_size, so it's fine inside SLRU_parse_params before SLRU_init allocates the list heads and byte counters (SLRU.c:148-160). CI is green on 2a04d238. No blocking issues.

A few things to consider:

  1. The admission bypass can still happen in byte mode (SLRU.c:275-279, SLRU.c:551-573, WTinyLFU.c:387-389). With the segments now summing to the cache size, count mode can't leave a free slot. With variable object sizes, though, a promotion that overflows protected cools its tail into probation, and SLRU_cool(…, 0) evicts whole probation objects until probation fits. That can leave main below cache_size by up to one object's size. W-TinyLFU's occupied + obj_size + md <= cache_size check then admits any window victim small enough to fit the slack, with no sketch comparison. You already flagged byte mode as untested; it may be worth an issue, or a note in the PR, that this fix is exact only for --ignore-obj-size=1.

  2. Related, pre-existing and out of scope: SLRU_cool moves obj->obj_size between segments (SLRU.c:567-568), but promote, insert and evict use obj_size + obj_md_size. With consider_obj_metadata, each promote-then-demote leaves +md on the upper segment and -md on the lower one, so per-segment occupancy drifts away from the real contents. That's the same family of segment-accounting bug, so it could go in the same follow-up.

  3. Published numbers change. As the description says, default W-TinyLFU gets worse in 50 of 76 cells, and cluster26 goes from about 0.26 to 0.47 at the smallest size. Since anyone comparing against earlier libCacheSim W-TinyLFU results will see this jump, a CHANGELOG/README line saying the fixed-window W-TinyLFU numbers moved because of this fix would save people some confusion.

  4. Cache name nit (SLRU.c:164-186). An explicit seg-size with equal parts on a size the parts don't divide now gives unequal segments. For example, seg-size=1:1 on 101 objects used to be 50/50 and is now 50/51. That flips same_size to false, so the name goes from S2LRU(50:50) to S2LRU(49:50), and the same goes for -e print. This already happens on the default path since #343, so it's consistent, but scripts that key on cache names will see new labels. Comparing seg_ratio instead of lru_max_n_bytes for same_size and the printed percentages would keep the labels stable.

  5. Test (test/test_evictionAlgo.c, test_SLRU_seg_size_fills_cache). This catches the regression through get_n_obj. Since the bug that mattered was in W-TinyLFU, a small W-TinyLFU-level test would pin the effect you care about: a main size not divisible by 5 (e.g. total 1223, window 0.14), then assert that main is full after warm-up, or that a cold window victim is rejected once main is full.

Posted automatically as a COMMENT review on behalf of @1a1a11a.

SLRU_cool() moved obj_size between segments while promote, insert and
evict moved obj_size + obj_md_size, so with --consider-obj-metadata
each promote-then-demote left the metadata on the upper segment's
count. When no segment had room, SLRU_insert() evicted until the whole
cache fit, which could push the lowest segment past its share when
objects differ in size; it now evicts until that segment fits too.

The cache name and -e print now show the shares asked for (seg_ratio)
instead of the byte limits, so seg-size=1:1 is S2LRU(50:50) at every
cache size and seg-size=1:2:3:4 prints 10:20:30:40.

Count mode (--ignore-obj-size=1) is unchanged.
In byte mode the main cache evicts whole objects, so it can sit a
little below its size. A window victim that fit that space was admitted
without the frequency check, even long after the main cache had filled.
Remember that the main cache has filled, and compare from then on; the
hill climber clears this when it grows the main cache. A winning window
victim larger than the main victim now evicts until it fits.

Two tests push a cold object out of the window into a full main cache
with free space: from SLRU segments that did not add up (count mode;
fails on develop 4e3521a) and from evicting whole objects (byte mode;
fails without this fix).

Count mode (--ignore-obj-size=1) is unchanged.
Comment thread libCacheSim/cache/eviction/SLRU.c Fixed
-e print exits from SLRU_parse_params() without freeing the cache.
LeakSanitizer stayed quiet only while the cache pointer was still live
in the print call; since the label change no longer passes it, the
Ubuntu job reports the whole cache as leaked. Free it before exiting.

The cache name loops now stop once the name fills the buffer, as the
-e print loop already does.

@1a1a11a 1a1a11a left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following up on the byte-mode and metadata points. The SLRU_cool byte fix and the matching _SLRU_verify_lru_size change, the seg_ratio-based names, the main_full latch and the two W-TinyLFU tests all look right to me. CI is green on 497a50e6. This is a re-review of the three commits since 2a04d238.

  1. New insert loop trusts lru_n_bytes[0], but SLRU_remove never updates it (SLRU.c:316-320, SLRU.c:407-420). The old loop only checked cache->occupied_byte, which cache_remove_obj_base keeps correct. The new condition also waits for lru_n_bytes[0] + sz <= lru_max_n_bytes[0]. SLRU_remove (an existing bug) unlinks the object but never decrements lru_n_bytes/lru_n_objs. After a user remove from probation, then, lru_n_bytes[0] stays too high. If probation's list is now empty, SLRU_to_evict skips past it to higher segments, and evicting those never lowers lru_n_bytes[0]. The loop can then drain the whole cache and end up in SLRU_evict with a NULL victim. WTinyLFU_remove reaches this through main_cache->remove. The simplest fix is to make SLRU_remove decrement lru_n_bytes[id] / lru_n_objs[id] the same way SLRU_evict does. Adding a cache->n_obj > 0 guard to the loop would also be cheap insurance.

  2. main_full is only cleared when the climber grows main (WTinyLFU.c:261-263, WTinyLFU.c:397-411). Once it's set, an explicit WTinyLFU_remove (WTinyLFU.c:464) that frees space in main leaves the latch on. Every later admission is then compared, and a winning victim also evicts a main object even though there's room. That's not wrong for TinyLFU, but it's a behaviour change worth either a comment or a main_full = false in the main-cache branch of WTinyLFU_remove.

  3. Plain SLRU byte-mode numbers move too (test/test_evictionAlgo.c:205-207). The SLRU truth values change in both directions, e.g. 89624 → 89862 at the smallest size and 75388 → 75447 at the fifth. I'd guess this comes from the insert loop now evicting until probation fits. The PR description still covers only the count-mode rounding fix and says byte mode is untested. Since this now changes default SLRU results in byte mode, and W-TinyLFU admission in byte mode, it's worth updating the description and CHANGELOG so people comparing against older numbers know why they moved.

  4. Minor: in WTinyLFU_evict, the new do { evict } while (!fits) (WTinyLFU.c:426-428) is safe because WTinyLFU_can_insert already requires the object to fit main's probation segment. A one-line comment saying so would help the next reader, since the loop has no other bound.

No blocking issues besides (1). That one is a real crash path, though only reachable through explicit removes.

Posted automatically as a COMMENT review on behalf of @1a1a11a.

@1a1a11a
1a1a11a merged commit 4b66d97 into 1a1a11a:develop Nov 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants