Skip to content

Commit eebb756

Browse files
authored
perf: stop mimalloc asking for transparent huge pages (#1391)
mimalloc madvises its 1 GiB arenas for transparent huge pages, so on Linux hosts with THP in madvise mode (the Ubuntu default) every partly used 2 MiB page stayed resident in full. A cold 10,000-model compile held about 470 MiB of huge pages at its peak. Building mimalloc with no_thp keeps those arenas on normal pages and lowers that compile's peak memory by about 85 MB, with unchanged wall and CPU time. Other platforms are unaffected.
1 parent 43bc9f7 commit eebb756

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

‎Cargo.toml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,9 @@ globset = "0.4"
8181
junction = { version = "2.1.0", default-features = false }
8282
keyring = { version = "3.6.3", default-features = false }
8383
# Python loads the extension with dlopen, so mimalloc must not use initial-exec TLS.
84-
mimalloc = { version = "0.1", default-features = false, features = ["local_dynamic_tls"] }
84+
# no_thp: mimalloc otherwise asks for transparent huge pages on its 1 GiB arenas, and each
85+
# partly used 2 MiB page then stays resident in full, which raises peak memory.
86+
mimalloc = { version = "0.1", default-features = false, features = ["local_dynamic_tls", "no_thp"] }
8587
proc-macro2 = { version = "1", features = ["span-locations"] }
8688
pyo3 = { version = "0.26", features = ["abi3-py312"] }
8789
rpassword = "7.5.4"

0 commit comments

Comments
 (0)