Skip to content

Local bucketing leaks native memory (it's causing OOMs) #195

New issue
New issue
Closed
#202
#201
Closed
Local bucketing leaks native memory (it's causing OOMs)#195
#202
#201

Description

@robbiebowman-rf
robbiebowman-rf
opened on Jul 24, 2026

Bug

DevCycleLocalClient leaks native linear memory on every variableValue() call. The JVM heap stays flat while process RSS climbs ~370 bytes per evaluation, so any service doing a steady volume of evaluations eventually gets OOM-killed. We hit this regularly in production (a worker service that evaluates ~30 flags per minute).

Reproduction

Claude helped me author a pretty minimalist reproduction of this. You'll need Docker and an API key. https://github.com/robbiebowman-rf/devcycle-leak-mre

Instructions are in the repo's README.

The repro takes about 2 minutes and has some logging showing RSS space climbs while the heap stays flat.

Suspected cause

Every evaluation copies the user into the WASM bucketing module's memory to run the lookup. Normally that scratch memory should be released afterwards, but on the protobuf variable path it never is, so each call leaves a bit behind. WASM memory can only grow and never shrinks, so those leftovers pile up until the process runs out.

Looking at the decompiled LocalBucketing, getVariableForUserProtobuf allocates via newUint8ArrayParameter but never calls unpinAll() (or otherwise frees/collects), whereas other methods like generateBucketedConfig do call unpinAll(). That missing cleanup looks like the leak.

Activity

  1. JamieSinn commented on Aug 17, 2026

    @JamieSinn
    JamieSinn
    on Aug 17, 2026
    Member

    Hey @robbiebowman-rf - thanks for the report!

    This bug was a bit more detailed, in the allocation the issue was not specifically where the AI thought it was.

    Image

    The allocation was actually leaking from the config metadata path, as that was being fetched in excess beyond requirement. This data is only written once at the write time of each config, and can be cached in the Java layer instead of the WASM layer.

    I've setup #201 to fix this, and once it's tested we'll get it merged in!

    Massively appreciate the repro - it made things a lot quicker to debug!

    Thanks!

  2. robbiebowman-rf commented on Aug 18, 2026

    @robbiebowman-rf
    robbiebowman-rf
    on Aug 18, 2026
    Author

    Thanks for fixing this, Jamie!

  3. JamieSinn
    closed this as completedin #201on Aug 18, 2026

  4. kaushalkapasi commented on Aug 18, 2026

    @kaushalkapasi
    kaushalkapasi
    on Aug 18, 2026
    Contributor

    Hey @robbiebowman-rf, we've released this fix as part of v2.9.6. It should be available to download/install shortly, once the Maven/Sonatype cache updates.

  5. robbiebowman-rf commented on Aug 19, 2026

    @robbiebowman-rf
    robbiebowman-rf
    on Aug 19, 2026
    Author

    Looks like the leak is still happening, just a little slower. See new branch with updated version: https://github.com/robbiebowman-rf/devcycle-leak-mre/tree/bump-2.9.6

  6. JamieSinn commented on Aug 19, 2026

    @JamieSinn
    JamieSinn
    on Aug 19, 2026
    Member

    Can you provide a bit more info about your runtime env? I'm not able to repro with the current setup that we have. Docker or otherwise.

  7. JamieSinn commented on Aug 19, 2026

    @JamieSinn
    JamieSinn
    on Aug 19, 2026
    Member

    Here is some interesting data -

    Standard test environment (M4 Mac):
    JDK 25

    [   5s] evals=76,600 heap=25MB rss=80MB rssDelta=-952MB (-13032 B/eval)
[  10s] evals=153,730 heap=17MB rss=80MB rssDelta=-952MB (-6493 B/eval)
[  15s] evals=233,575 heap=25MB rss=80MB rssDelta=-952MB (-4274 B/eval)
[  20s] evals=313,174 heap=33MB rss=80MB rssDelta=-952MB (-3188 B/eval)
[  25s] evals=393,058 heap=33MB rss=80MB rssDelta=-952MB (-2540 B/eval)
[  30s] evals=472,148 heap=41MB rss=80MB rssDelta=-952MB (-2114 B/eval)
[  35s] evals=551,325 heap=41MB rss=80MB rssDelta=-952MB (-1811 B/eval)
[  40s] evals=631,149 heap=9MB rss=80MB rssDelta=-952MB (-1582 B/eval)
[  45s] evals=710,575 heap=9MB rss=80MB rssDelta=-952MB (-1405 B/eval)
[  50s] evals=790,642 heap=17MB rss=80MB rssDelta=-952MB (-1263 B/eval)
[  55s] evals=870,536 heap=25MB rss=80MB rssDelta=-952MB (-1147 B/eval)
[  60s] evals=950,688 heap=25MB rss=80MB rssDelta=-952MB (-1050 B/eval)
[  65s] evals=1,029,640 heap=33MB rss=80MB rssDelta=-952MB (-970 B/eval)
[  70s] evals=1,109,518 heap=33MB rss=80MB rssDelta=-952MB (-900 B/eval)
[  75s] evals=1,189,468 heap=41MB rss=80MB rssDelta=-952MB (-839 B/eval)
[  80s] evals=1,269,099 heap=41MB rss=80MB rssDelta=-952MB (-787 B/eval)
[  85s] evals=1,348,770 heap=9MB rss=80MB rssDelta=-952MB (-740 B/eval)
[  90s] evals=1,428,459 heap=17MB rss=80MB rssDelta=-952MB (-699 B/eval)
[  95s] evals=1,508,216 heap=17MB rss=80MB rssDelta=-952MB (-662 B/eval)
[ 100s] evals=1,587,912 heap=25MB rss=80MB rssDelta=-952MB (-629 B/eval)
[ 105s] evals=1,667,697 heap=25MB rss=80MB rssDelta=-952MB (-599 B/eval)
[ 110s] evals=1,747,748 heap=33MB rss=80MB rssDelta=-952MB (-571 B/eval)
[ 115s] evals=1,827,545 heap=41MB rss=80MB rssDelta=-952MB (-546 B/eval)
[ 120s] evals=1,906,251 heap=41MB rss=80MB rssDelta=-952MB (-524 B/eval)
[ 125s] evals=1,985,588 heap=41MB rss=80MB rssDelta=-952MB (-503 B/eval)
[ 130s] evals=2,065,036 heap=9MB rss=80MB rssDelta=-952MB (-483 B/eval)
[ 135s] evals=2,144,251 heap=9MB rss=80MB rssDelta=-952MB (-466 B/eval)
[ 140s] evals=2,224,109 heap=17MB rss=80MB rssDelta=-952MB (-449 B/eval)
[ 145s] evals=2,303,793 heap=17MB rss=80MB rssDelta=-952MB (-433 B/eval)
[ 150s] evals=2,383,478 heap=25MB rss=80MB rssDelta=-952MB (-419 B/eval)
[ 155s] evals=2,463,353 heap=25MB rss=80MB rssDelta=-952MB (-405 B/eval)
[ 160s] evals=2,543,162 heap=33MB rss=80MB rssDelta=-952MB (-393 B/eval)
[ 165s] evals=2,623,080 heap=41MB rss=80MB rssDelta=-952MB (-381 B/eval)
[ 170s] evals=2,703,076 heap=41MB rss=80MB rssDelta=-952MB (-369 B/eval)
[ 175s] evals=2,783,171 heap=9MB rss=80MB rssDelta=-952MB (-359 B/eval)
[ 180s] evals=2,863,523 heap=17MB rss=80MB rssDelta=-952MB (-349 B/eval)
[ 185s] evals=2,943,741 heap=25MB rss=80MB rssDelta=-952MB (-339 B/eval)
[ 190s] evals=3,023,018 heap=25MB rss=80MB rssDelta=-952MB (-330 B/eval)
[ 195s] evals=3,103,119 heap=33MB rss=80MB rssDelta=-952MB (-322 B/eval)
[ 200s] evals=3,182,658 heap=41MB rss=80MB rssDelta=-952MB (-314 B/eval)
[ 205s] evals=3,261,152 heap=41MB rss=80MB rssDelta=-952MB (-306 B/eval)
[ 210s] evals=3,340,188 heap=41MB rss=80MB rssDelta=-952MB (-299 B/eval)
[ 215s] evals=3,418,281 heap=41MB rss=80MB rssDelta=-952MB (-292 B/eval)
[ 220s] evals=3,497,319 heap=41MB rss=80MB rssDelta=-952MB (-285 B/eval)
[ 225s] evals=3,576,693 heap=41MB rss=80MB rssDelta=-952MB (-279 B/eval)
[ 230s] evals=3,655,003 heap=41MB rss=80MB rssDelta=-952MB (-273 B/eval)
[ 235s] evals=3,733,710 heap=41MB rss=80MB rssDelta=-952MB (-267 B/eval)
[ 240s] evals=3,810,728 heap=41MB rss=80MB rssDelta=-952MB (-262 B/eval)
[ 245s] evals=3,888,806 heap=33MB rss=80MB rssDelta=-952MB (-257 B/eval)
[ 250s] evals=3,967,221 heap=33MB rss=80MB rssDelta=-952MB (-252 B/eval)
[ 255s] evals=4,044,950 heap=33MB rss=80MB rssDelta=-952MB (-247 B/eval)
[ 260s] evals=4,122,604 heap=33MB rss=80MB rssDelta=-952MB (-242 B/eval)
[ 265s] evals=4,201,129 heap=33MB rss=80MB rssDelta=-952MB (-238 B/eval)
[ 270s] evals=4,278,877 heap=25MB rss=80MB rssDelta=-952MB (-233 B/eval)
[ 275s] evals=4,356,010 heap=25MB rss=80MB rssDelta=-952MB (-229 B/eval)
[ 280s] evals=4,433,227 heap=17MB rss=80MB rssDelta=-952MB (-225 B/eval)
[ 285s] evals=4,510,625 heap=17MB rss=80MB rssDelta=-952MB (-221 B/eval)
[ 290s] evals=4,588,061 heap=9MB rss=80MB rssDelta=-952MB (-218 B/eval)
[ 295s] evals=4,665,290 heap=9MB rss=80MB rssDelta=-952MB (-214 B/eval)
[ 300s] evals=4,742,510 heap=41MB rss=80MB rssDelta=-952MB (-210 B/eval)
[ 305s] evals=4,819,864 heap=41MB rss=80MB rssDelta=-952MB (-207 B/eval)
[ 310s] evals=4,897,570 heap=33MB rss=80MB rssDelta=-952MB (-204 B/eval)
[ 315s] evals=4,975,115 heap=33MB rss=80MB rssDelta=-952MB (-201 B/eval)
[ 320s] evals=5,052,579 heap=33MB rss=80MB rssDelta=-952MB (-198 B/eval)
[ 325s] evals=5,129,837 heap=25MB rss=80MB rssDelta=-952MB (-195 B/eval)
[ 330s] evals=5,206,977 heap=17MB rss=80MB rssDelta=-952MB (-192 B/eval)
[ 335s] evals=5,284,337 heap=17MB rss=80MB rssDelta=-952MB (-189 B/eval)
[ 340s] evals=5,361,249 heap=9MB rss=80MB rssDelta=-952MB (-186 B/eval)
[ 345s] evals=5,438,247 heap=9MB rss=80MB rssDelta=-952MB (-184 B/eval)
[ 350s] evals=5,515,801 heap=41MB rss=80MB rssDelta=-952MB (-181 B/eval)
[ 355s] evals=5,593,199 heap=41MB rss=80MB rssDelta=-952MB (-178 B/eval)
[ 360s] evals=5,670,564 heap=33MB rss=80MB rssDelta=-952MB (-176 B/eval)
[ 365s] evals=5,747,752 heap=33MB rss=80MB rssDelta=-952MB (-174 B/eval)
[ 370s] evals=5,825,041 heap=25MB rss=80MB rssDelta=-952MB (-171 B/eval)
[ 375s] evals=5,902,372 heap=25MB rss=80MB rssDelta=-952MB (-169 B/eval)
[ 380s] evals=5,980,074 heap=17MB rss=80MB rssDelta=-952MB (-167 B/eval)
[ 385s] evals=6,057,598 heap=17MB rss=80MB rssDelta=-952MB (-165 B/eval)
[ 390s] evals=6,135,202 heap=9MB rss=80MB rssDelta=-952MB (-163 B/eval)
[ 395s] evals=6,212,330 heap=9MB rss=80MB rssDelta=-952MB (-161 B/eval)
[ 400s] evals=6,289,808 heap=41MB rss=80MB rssDelta=-952MB (-159 B/eval)
[ 405s] evals=6,366,977 heap=41MB rss=80MB rssDelta=-952MB (-157 B/eval)
[ 410s] evals=6,444,881 heap=41MB rss=80MB rssDelta=-952MB (-155 B/eval)
[ 415s] evals=6,523,491 heap=41MB rss=80MB rssDelta=-952MB (-153 B/eval)
[ 420s] evals=6,602,510 heap=41MB rss=80MB rssDelta=-952MB (-151 B/eval)
[ 425s] evals=6,682,150 heap=9MB rss=80MB rssDelta=-952MB (-149 B/eval)
[ 430s] evals=6,761,422 heap=9MB rss=80MB rssDelta=-952MB (-148 B/eval)
[ 435s] evals=6,841,034 heap=9MB rss=80MB rssDelta=-952MB (-146 B/eval)
[ 440s] evals=6,920,644 heap=17MB rss=80MB rssDelta=-952MB (-144 B/eval)
[ 445s] evals=7,000,574 heap=25MB rss=80MB rssDelta=-952MB (-143 B/eval)
[ 450s] evals=7,080,118 heap=25MB rss=80MB rssDelta=-952MB (-141 B/eval)
[ 455s] evals=7,159,881 heap=33MB rss=80MB rssDelta=-952MB (-139 B/eval)
[ 460s] evals=7,239,762 heap=33MB rss=80MB rssDelta=-952MB (-138 B/eval)
[ 465s] evals=7,319,219 heap=41MB rss=80MB rssDelta=-952MB (-136 B/eval)
[ 470s] evals=7,399,162 heap=9MB rss=80MB rssDelta=-952MB (-135 B/eval)
[ 475s] evals=7,479,552 heap=9MB rss=80MB rssDelta=-952MB (-133 B/eval)
[ 480s] evals=7,560,008 heap=17MB rss=80MB rssDelta=-952MB (-132 B/eval)
[ 485s] evals=7,640,192 heap=25MB rss=80MB rssDelta=-952MB (-131 B/eval)
[ 490s] evals=7,719,365 heap=33MB rss=80MB rssDelta=-952MB (-129 B/eval)
[ 495s] evals=7,798,657 heap=33MB rss=80MB rssDelta=-952MB (-128 B/eval)
[ 500s] evals=7,878,031 heap=33MB rss=80MB rssDelta=-952MB (-127 B/eval)
[ 505s] evals=7,958,098 heap=41MB rss=80MB rssDelta=-952MB (-125 B/eval)
[ 510s] evals=8,037,265 heap=41MB rss=80MB rssDelta=-952MB (-124 B/eval)
[ 515s] evals=8,116,883 heap=9MB rss=80MB rssDelta=-952MB (-123 B/eval)
[ 520s] evals=8,196,068 heap=9MB rss=80MB rssDelta=-952MB (-122 B/eval)
[ 525s] evals=8,276,124 heap=17MB rss=80MB rssDelta=-952MB (-121 B/eval)
[ 530s] evals=8,355,751 heap=25MB rss=80MB rssDelta=-952MB (-119 B/eval)
[ 535s] evals=8,435,422 heap=25MB rss=80MB rssDelta=-952MB (-118 B/eval)
[ 540s] evals=8,514,456 heap=33MB rss=80MB rssDelta=-952MB (-117 B/eval)
[ 545s] evals=8,593,518 heap=33MB rss=80MB rssDelta=-952MB (-116 B/eval)
[ 550s] evals=8,671,308 heap=33MB rss=80MB rssDelta=-952MB (-115 B/eval)
[ 555s] evals=8,750,505 heap=33MB rss=80MB rssDelta=-952MB (-114 B/eval)
[ 560s] evals=8,828,927 heap=33MB rss=80MB rssDelta=-952MB (-113 B/eval)
[ 565s] evals=8,907,695 heap=41MB rss=80MB rssDelta=-952MB (-112 B/eval)
[ 570s] evals=8,986,058 heap=41MB rss=80MB rssDelta=-952MB (-111 B/eval)
[ 575s] evals=9,065,104 heap=41MB rss=80MB rssDelta=-952MB (-110 B/eval)
[ 580s] evals=9,144,303 heap=41MB rss=80MB rssDelta=-952MB (-109 B/eval)
[ 585s] evals=9,223,507 heap=9MB rss=80MB rssDelta=-952MB (-108 B/eval)
[ 590s] evals=9,300,253 heap=41MB rss=80MB rssDelta=-952MB (-107 B/eval)
[ 595s] evals=9,378,776 heap=41MB rss=80MB rssDelta=-952MB (-106 B/eval)
[ 600s] evals=9,457,690 heap=9MB rss=80MB rssDelta=-952MB (-106 B/eval)
[ 605s] evals=9,535,982 heap=9MB rss=80MB rssDelta=-952MB (-105 B/eval)
[ 610s] evals=9,614,973 heap=9MB rss=80MB rssDelta=-952MB (-104 B/eval)
[ 615s] evals=9,693,502 heap=9MB rss=80MB rssDelta=-952MB (-103 B/eval)
[ 620s] evals=9,772,587 heap=9MB rss=80MB rssDelta=-952MB (-102 B/eval)
[ 625s] evals=9,852,058 heap=17MB rss=80MB rssDelta=-952MB (-101 B/eval)

    Docker:

    [   5s] evals=99,276 heap=7MB rss=217MB rssDelta=+54MB (579 B/eval)
[  10s] evals=202,957 heap=42MB rss=236MB rssDelta=+74MB (384 B/eval)
[  15s] evals=306,717 heap=21MB rss=252MB rssDelta=+90MB (308 B/eval)
[  20s] evals=409,889 heap=52MB rss=269MB rssDelta=+106MB (273 B/eval)
[  25s] evals=511,852 heap=24MB rss=288MB rssDelta=+125MB (257 B/eval)
[  30s] evals=615,719 heap=58MB rss=311MB rssDelta=+148MB (254 B/eval)
[  35s] evals=719,439 heap=37MB rss=331MB rssDelta=+169MB (247 B/eval)
[  40s] evals=821,179 heap=8MB rss=353MB rssDelta=+190MB (243 B/eval)
[  45s] evals=923,135 heap=35MB rss=372MB rssDelta=+210MB (239 B/eval)
[  50s] evals=1,024,966 heap=61MB rss=391MB rssDelta=+229MB (234 B/eval)
[  55s] evals=1,127,207 heap=35MB rss=413MB rssDelta=+251MB (234 B/eval)
[  60s] evals=1,228,108 heap=58MB rss=429MB rssDelta=+267MB (228 B/eval)
[  65s] evals=1,329,333 heap=28MB rss=447MB rssDelta=+284MB (225 B/eval)
[  70s] evals=1,431,198 heap=55MB rss=467MB rssDelta=+305MB (224 B/eval)
[  75s] evals=1,533,962 heap=30MB rss=486MB rssDelta=+323MB (221 B/eval)
[  80s] evals=1,635,266 heap=22MB rss=477MB rssDelta=+314MB (202 B/eval)
[  85s] evals=1,736,846 heap=20MB rss=503MB rssDelta=+340MB (206 B/eval)
[  90s] evals=1,838,429 heap=18MB rss=514MB rssDelta=+352MB (201 B/eval)
[  95s] evals=1,939,017 heap=12MB rss=532MB rssDelta=+370MB (200 B/eval)
[ 100s] evals=2,040,194 heap=9MB rss=552MB rssDelta=+390MB (201 B/eval)
[ 105s] evals=2,139,426 heap=18MB rss=573MB rssDelta=+410MB (201 B/eval)

    Looks like the reporting on the OSX side does not properly track heap/rss usage.

    I've fixed this using ps and I can now see a small leak still. Interesting stackup

  8. JamieSinn
    reopened this on Aug 19, 2026

  9. robbiebowman-rf commented on Aug 19, 2026

    @robbiebowman-rf
    robbiebowman-rf
    on Aug 19, 2026
    Author

    If you checkout the branch of my reproduction repo on 2.9.6, then all you need is a DevCycle SDK key and Docker (I'm running 29.7.2, fwiw)

    Then run:

    docker build -t devcycle-leak-mre .

docker run  --name leak-mre --memory=512m \
  -e JAVA_OPTS="-Xms64m -Xmx64m" \
  -e DEVCYCLE_SERVER_SDK_KEY="$APP_DEVCYCLE_SDKKEY" \
  devcycle-leak-mre

  10. robbiebowman-rf commented on Aug 19, 2026

    @robbiebowman-rf
    robbiebowman-rf
    on Aug 19, 2026
    Author

    Yes it's definitely a slower leak than it was on 2.9.4.

  11. JamieSinn commented on Aug 19, 2026

    @JamieSinn
    JamieSinn
    on Aug 19, 2026
    Member

    @robbiebowman-rf I've found the root cause of the leak, and it's actually related to a bug in the wasmtime-java library and how it's calling the invocation function with a leak.

    I've gotten the leak massively minimized, but I don't know if we can get much more improvement than this:

    [   5s] evals=101,990 heap=26MB wasm=6MB rss=586MB rssDelta=+47MB (491 B/eval)
[  10s] evals=203,884 heap=42MB wasm=6MB rss=588MB rssDelta=+49MB (256 B/eval)
[  15s] evals=307,178 heap=10MB wasm=6MB rss=589MB rssDelta=+50MB (171 B/eval)
[  20s] evals=409,955 heap=18MB wasm=6MB rss=589MB rssDelta=+50MB (129 B/eval)
[  25s] evals=513,118 heap=26MB wasm=6MB rss=589MB rssDelta=+50MB (103 B/eval)
[  30s] evals=617,236 heap=34MB wasm=6MB rss=589MB rssDelta=+50MB (86 B/eval)
[  35s] evals=719,752 heap=10MB wasm=6MB rss=590MB rssDelta=+50MB (74 B/eval)
[  40s] evals=822,216 heap=18MB wasm=6MB rss=590MB rssDelta=+51MB (65 B/eval)
[  45s] evals=925,008 heap=26MB wasm=6MB rss=590MB rssDelta=+51MB (58 B/eval)
[  50s] evals=1,027,471 heap=34MB wasm=6MB rss=590MB rssDelta=+51MB (53 B/eval)
[  55s] evals=1,129,859 heap=42MB wasm=6MB rss=590MB rssDelta=+51MB (48 B/eval)
[  60s] evals=1,233,529 heap=42MB wasm=6MB rss=590MB rssDelta=+51MB (44 B/eval)
[  65s] evals=1,335,386 heap=18MB wasm=6MB rss=590MB rssDelta=+51MB (41 B/eval)
[  70s] evals=1,437,700 heap=26MB wasm=6MB rss=591MB rssDelta=+52MB (38 B/eval)
[  75s] evals=1,539,752 heap=26MB wasm=6MB rss=591MB rssDelta=+52MB (36 B/eval)
[  80s] evals=1,641,965 heap=34MB wasm=6MB rss=591MB rssDelta=+52MB (34 B/eval)
[  85s] evals=1,743,479 heap=34MB wasm=6MB rss=591MB rssDelta=+52MB (32 B/eval)
[  91s] evals=1,846,901 heap=42MB wasm=6MB rss=589MB rssDelta=+50MB (29 B/eval)
[  96s] evals=1,947,385 heap=10MB wasm=6MB rss=589MB rssDelta=+50MB (27 B/eval)
[ 101s] evals=2,049,291 heap=18MB wasm=6MB rss=589MB rssDelta=+50MB (26 B/eval)
[ 106s] evals=2,150,458 heap=18MB wasm=6MB rss=589MB rssDelta=+50MB (25 B/eval)
[ 111s] evals=2,251,629 heap=18MB wasm=6MB rss=590MB rssDelta=+51MB (24 B/eval)
[ 116s] evals=2,351,843 heap=18MB wasm=6MB rss=590MB rssDelta=+51MB (23 B/eval)
[ 121s] evals=2,454,416 heap=18MB wasm=6MB rss=590MB rssDelta=+51MB (22 B/eval)
[ 126s] evals=2,556,314 heap=34MB wasm=6MB rss=590MB rssDelta=+51MB (21 B/eval)
[ 131s] evals=2,657,246 heap=34MB wasm=6MB rss=590MB rssDelta=+51MB (20 B/eval)
[ 136s] evals=2,758,711 heap=34MB wasm=6MB rss=590MB rssDelta=+51MB (20 B/eval)
[ 141s] evals=2,860,105 heap=42MB wasm=6MB rss=591MB rssDelta=+51MB (19 B/eval)
[ 146s] evals=2,961,521 heap=42MB wasm=6MB rss=591MB rssDelta=+52MB (18 B/eval)
[ 151s] evals=3,063,981 heap=10MB wasm=6MB rss=591MB rssDelta=+52MB (18 B/eval)
[ 156s] evals=3,165,389 heap=18MB wasm=6MB rss=591MB rssDelta=+52MB (17 B/eval)
[ 161s] evals=3,265,226 heap=10MB wasm=6MB rss=591MB rssDelta=+52MB (17 B/eval)
[ 166s] evals=3,366,284 heap=18MB wasm=6MB rss=591MB rssDelta=+52MB (16 B/eval)
[ 171s] evals=3,467,555 heap=18MB wasm=6MB rss=591MB rssDelta=+52MB (16 B/eval)
[ 176s] evals=3,567,819 heap=18MB wasm=6MB rss=591MB rssDelta=+52MB (16 B/eval)
[ 181s] evals=3,669,949 heap=18MB wasm=6MB rss=591MB rssDelta=+52MB (15 B/eval)
[ 186s] evals=3,770,249 heap=26MB wasm=6MB rss=591MB rssDelta=+52MB (15 B/eval)
[ 191s] evals=3,870,094 heap=18MB wasm=6MB rss=592MB rssDelta=+53MB (14 B/eval)
[ 196s] evals=3,970,358 heap=18MB wasm=6MB rss=592MB rssDelta=+53MB (14 B/eval)
[ 201s] evals=4,071,525 heap=26MB wasm=6MB rss=592MB rssDelta=+53MB (14 B/eval)
[ 206s] evals=4,171,937 heap=26MB wasm=6MB rss=592MB rssDelta=+53MB (13 B/eval)
[ 211s] evals=4,273,604 heap=18MB wasm=6MB rss=592MB rssDelta=+53MB (13 B/eval)
[ 216s] evals=4,374,120 heap=26MB wasm=6MB rss=592MB rssDelta=+53MB (13 B/eval)

    Unfortunately the library is 3 years out of date of WASMTime.

    See #202 for the full fix details

  12. JamieSinn
    closed this as completedin #202on Aug 19, 2026

  13. robbiebowman-rf commented on Aug 20, 2026

    @robbiebowman-rf
    robbiebowman-rf
    on Aug 20, 2026
    Author

    That looks like it slows it to such a degree that our release cadence will restart pods well before they hit this OOM issue. Appreciate the diligence! I'll upgrade to 2.9.7 pronto

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions