THP can be effective for specific workloads by lowering the TLB cache’s unhit state, but if it’s not used properly, the 2MB vs. 4Kb page size can result in significant memory waste.
The fundamental idea of THP Shrinker, developed by Meta/Facebook developers to improve Linux’s Transparent Pages (THP), is to reduce memory wastage by eliminating unused transparent pages.
Highlights
-
By separating the least-used big memory pages, the THP Shrinker mechanism Facebook is developing for the Linux kernel intends to solve this waste while still enjoying the advantages of THP for CPU performance.
-
On one of their platforms, Facebook developers discovered that the THP wasn’t being used to its full potential, even leaving each machine with almost 2.7GB of unused memory.
Facebook engineers are working to tie this THP Shrinker to CPU/IO/memory stress events and to add more tuning to it. The THP shrinker should eventually enable the engineers to directly deprecate Linux’s madvise THP mode and switch to THP enabled for all programmes, they hope.
The madevise system call-based option is replaced by THP Shrinker, which is created to prevent RAM wastage and make huge memory pages as transparent and accessible to everyone as feasible.