Repository navigation
jitv2: stop retrying MADV_COLLAPSE once it fails for lack of memory - #801
Open
jamesbraid wants to merge 1 commit into
Open
jamesbraid wants to merge 1 commit into
jamesbraid wants to merge 1 commit into
Conversation
The compile thread calls madvise(MADV_COLLAPSE) on each fully sealed 2 MiB of the code arena, to put sealed code on hugepages and cut iTLB misses. The arena is PROT_NONE, then RW per commit, then RX per sealed page, so fault-time hugepages cannot form across those mappings and this call is the only thing that can give the arena any. On a host with fragmented free memory every call fails, slowly. On a 224-CPU shared host (93 percent of compaction attempts failing host-wide), an iris-bench run --quick --cpu r10000 made 6 collapse calls under strace -T. They took 0.004 to 1.17 s each and all returned ENOMEM, so each stalled the compile thread and collapsed nothing. A profile of an O2 PROM boot window on that host showed 46 percent of samples in the kernel's compaction spinlock; I could not reproduce that figure on a quieter day, where compaction was about 0.6 percent of samples. After the first ENOMEM, EAGAIN or EBUSY, stop calling it for that arena. A host where collapse works keeps the hugepages; a fragmented one pays for at most one failed attempt. No speed claim beyond that. Fixes: 81cd871 ("soa bitmap to aos flag for compile/valid/deny bits ...")
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The compile thread calls madvise(MADV_COLLAPSE) on each fully sealed
2 MiB of the code arena, to put sealed code on hugepages and cut iTLB
misses. The arena is PROT_NONE, then RW per commit, then RX per sealed
page, so fault-time hugepages cannot form across those mappings and
this call is the only thing that can give the arena any.
On a host with fragmented free memory every call fails, slowly. On a
224-CPU shared host (93 percent of compaction attempts failing
host-wide), an iris-bench run --quick --cpu r10000 made 6 collapse
calls under strace -T. They took 0.004 to 1.17 s each and all returned
ENOMEM, so each stalled the compile thread and collapsed nothing.
A profile of an O2 PROM boot window on that host showed 46 percent of
samples in the kernel's compaction spinlock; I could not reproduce
that figure on a quieter day, where compaction was about 0.6 percent of
samples.
After the first ENOMEM, EAGAIN or EBUSY, stop calling it for that
arena. A host where collapse works keeps the hugepages; a fragmented
one pays for at most one failed attempt. No speed claim beyond that.
Fixes: 81cd871 ("soa bitmap to aos flag for compile/valid/deny bits ...")