Repository navigation
Rebase edera/mainline onto master @ 65ba8f34ec88 - #64
Closed
edera-rebase-app[bot] wants to merge 10000 commits into
Closed
edera-rebase-app[bot] wants to merge 10000 commits into
edera-rebase-app[bot] wants to merge 10000 commits into
Conversation
Now we have the expressive vma_is_mm_backed() predicate, use it to determine whether to proceed with migration. This drops the VMA_IO_BIT test, which is safe as mm-managed mappings may not set it. hugetlb mappings are mm-backed but remain explicitly excluded, as the migrate_vma page walk has no hugetlb support. No functional change intended. Link: https://lore.kernel.org/20261013-b4-mmap-prepare-vma-flag-sanify-v6-36-1aa3c9b139f9@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Leon Romanovsky <leon@kernel.org> Cc: Paul Moore <paul@paul-moore.com> Cc: Stephen Smalley <stephen.smalley.work@gmail.com> Cc: Jaroslav Kysela <perex@perex.cz> Cc: Takashi Iwai <tiwai@suse.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Lance Yang <lance.yang@linux.dev> Cc: Usama Arif <usama.arif@linux.dev> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Doug Gilbert <dgilbert@interlog.com> Cc: James Bottomley <james.bottomley@hansenpartnership.com> Cc: Martin K. Petersen <mkp@kernel.org> Cc: Simona Vetter <simona@ffwll.ch> Cc: Helge Deller <deller@gmx.de> Cc: Sebastian Reichel <sre@kernel.org> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Peter Xu <peterx@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Rik van Riel <riel@surriel.com> Cc: Harry Yoo <harry@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Thomas Zimmermann <tzimmermann@suse.de> Cc: David Airlie <airlied@gmail.com> Cc: Will Deacon <will@kernel.org> Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Jan Kara <jack@suse.cz> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Anup Patel <anup@brainfault.org> Cc: Paul Walmsley <pjw@kernel.org> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Janosch Frank <frankja@linux.ibm.com> Cc: Claudio Imbrenda <imbrenda@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: David S. Miller <davem@davemloft.net> Cc: Andreas Larsson <andreas@gaisler.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christian Brauner <brauner@kernel.org> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Byungchul Park <byungchul@sk.com> Cc: Gregory Price <gourry@gourry.net> Cc: Ying Huang <ying.huang@linux.alibaba.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Chris Li <chrisl@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Youngjun Park <youngjun.park@lge.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Qi Zheng <qi.zheng@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Wei Xu <weixugc@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Michal Hocko <mhocko@kernel.org> Cc: Miklos Szeredi <miklos@szeredi.hu> Cc: Xu Xin <xu.xin@linux.dev> Cc: Ihor Solodrai <ihor.solodrai@linux.dev> Cc: Bert Karwatzki <spasswolf@web.de>
Every user of the VM_SPECIAL or VMA_SPECIAL_FLAGS has now been converted to predicates which explicitly express what is actually being checked for rather than the nebulous concept of possessing 'special' VMA flags. In any case 'special' is not so special a term of art in mm - it includes VDSO/VVAR mappings, special in the sense of vm_normal_folio() and probably other cases too. Therefore make things less special by eliminating these now unused flags. No functional change intended. Link: https://lore.kernel.org/20261013-b4-mmap-prepare-vma-flag-sanify-v6-37-1aa3c9b139f9@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Zi Yan <ziy@nvidia.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Leon Romanovsky <leon@kernel.org> Cc: Paul Moore <paul@paul-moore.com> Cc: Stephen Smalley <stephen.smalley.work@gmail.com> Cc: Jaroslav Kysela <perex@perex.cz> Cc: Takashi Iwai <tiwai@suse.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Lance Yang <lance.yang@linux.dev> Cc: Usama Arif <usama.arif@linux.dev> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Doug Gilbert <dgilbert@interlog.com> Cc: James Bottomley <james.bottomley@hansenpartnership.com> Cc: Martin K. Petersen <mkp@kernel.org> Cc: Simona Vetter <simona@ffwll.ch> Cc: Helge Deller <deller@gmx.de> Cc: Sebastian Reichel <sre@kernel.org> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Peter Xu <peterx@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Rik van Riel <riel@surriel.com> Cc: Harry Yoo <harry@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Thomas Zimmermann <tzimmermann@suse.de> Cc: David Airlie <airlied@gmail.com> Cc: Will Deacon <will@kernel.org> Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Jan Kara <jack@suse.cz> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Anup Patel <anup@brainfault.org> Cc: Paul Walmsley <pjw@kernel.org> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Janosch Frank <frankja@linux.ibm.com> Cc: Claudio Imbrenda <imbrenda@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: David S. Miller <davem@davemloft.net> Cc: Andreas Larsson <andreas@gaisler.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christian Brauner <brauner@kernel.org> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Byungchul Park <byungchul@sk.com> Cc: Gregory Price <gourry@gourry.net> Cc: Ying Huang <ying.huang@linux.alibaba.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Chris Li <chrisl@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Youngjun Park <youngjun.park@lge.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Qi Zheng <qi.zheng@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Wei Xu <weixugc@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Michal Hocko <mhocko@kernel.org> Cc: Miklos Szeredi <miklos@szeredi.hu> Cc: Xu Xin <xu.xin@linux.dev> Cc: Ihor Solodrai <ihor.solodrai@linux.dev> Cc: Bert Karwatzki <spasswolf@web.de>
Commit e1fb4a0 ("dax: remove VM_MIXEDMAP for fsdax and device dax") prevented fsdax and device-dax from setting VM_MIXEDMAP, as DAX no longer relies on it to direct core mm paths. The fuse DAX implementation, added later, copied the old pattern and still sets it. Fuse DAX maps pages the same way fsdax does, via dax_iomap_fault() and ultimately vmf_insert_page_mkwrite() and vmf_insert_folio_pmd(), which insert ordinary refcounted pages and so do not require VM_MIXEDMAP. Setting it only serves to mark the mapping as not mm-managed, making fuse DAX the sole DAX implementation whose mappings are unmergeable, cannot be mlock()'d, eagerly copy page tables on fork and reject MADV_DOFORK and MADV_DODUMP. It also requires vma_is_special_huge() in mm/huge_memory.c to carve DAX out of its check for mappings not managed by core mm explicitly. There is no reason for fuse DAX to keep on using this flag so drop it. Link: https://lore.kernel.org/20261013-b4-mmap-prepare-vma-flag-sanify-v6-38-1aa3c9b139f9@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Leon Romanovsky <leon@kernel.org> Cc: Paul Moore <paul@paul-moore.com> Cc: Stephen Smalley <stephen.smalley.work@gmail.com> Cc: Jaroslav Kysela <perex@perex.cz> Cc: Takashi Iwai <tiwai@suse.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Zi Yan <ziy@nvidia.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Lance Yang <lance.yang@linux.dev> Cc: Usama Arif <usama.arif@linux.dev> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Doug Gilbert <dgilbert@interlog.com> Cc: James Bottomley <james.bottomley@hansenpartnership.com> Cc: Martin K. Petersen <mkp@kernel.org> Cc: Simona Vetter <simona@ffwll.ch> Cc: Helge Deller <deller@gmx.de> Cc: Sebastian Reichel <sre@kernel.org> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Peter Xu <peterx@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Rik van Riel <riel@surriel.com> Cc: Harry Yoo <harry@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Thomas Zimmermann <tzimmermann@suse.de> Cc: David Airlie <airlied@gmail.com> Cc: Will Deacon <will@kernel.org> Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Jan Kara <jack@suse.cz> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Anup Patel <anup@brainfault.org> Cc: Paul Walmsley <pjw@kernel.org> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Janosch Frank <frankja@linux.ibm.com> Cc: Claudio Imbrenda <imbrenda@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: David S. Miller <davem@davemloft.net> Cc: Andreas Larsson <andreas@gaisler.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christian Brauner <brauner@kernel.org> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Byungchul Park <byungchul@sk.com> Cc: Gregory Price <gourry@gourry.net> Cc: Ying Huang <ying.huang@linux.alibaba.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Chris Li <chrisl@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Youngjun Park <youngjun.park@lge.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Qi Zheng <qi.zheng@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Wei Xu <weixugc@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Michal Hocko <mhocko@kernel.org> Cc: Miklos Szeredi <miklos@szeredi.hu> Cc: Xu Xin <xu.xin@linux.dev> Cc: Ihor Solodrai <ihor.solodrai@linux.dev> Cc: Bert Karwatzki <spasswolf@web.de>
vma_is_special_huge() tests whether either the VMA_PFNMAP_BIT or VMA_MIXEDMAP_BIT is set (i.e. whether the VMA is not mm-managed), but with a DAX carve-out. DAX however no longer sets VMA_MIXEDMAP_BIT, so this carve-out is no longer required. Therefore test for !vma_is_mm_managed() instead and also drop the VMA_IO_BIT check, as it is now redundant since it is enforced that mm-managed mappings may not set VMA_IO_BIT. This also eliminates another overloaded use of 'special' within mm. No functional change intended. Link: https://lore.kernel.org/20261013-b4-mmap-prepare-vma-flag-sanify-v6-39-1aa3c9b139f9@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Zi Yan <ziy@nvidia.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: Pedro Falcato <pfalcato@suse.de> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Leon Romanovsky <leon@kernel.org> Cc: Paul Moore <paul@paul-moore.com> Cc: Stephen Smalley <stephen.smalley.work@gmail.com> Cc: Jaroslav Kysela <perex@perex.cz> Cc: Takashi Iwai <tiwai@suse.com> Cc: Alexei Starovoitov <ast@kernel.org> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Andrii Nakryiko <andrii@kernel.org> Cc: Eduard Zingerman <eddyz87@gmail.com> Cc: Kumar Kartikeya Dwivedi <memxor@gmail.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Nico Pache <nico.pache@linux.dev> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Dev Jain <dev.jain@arm.com> Cc: Barry Song <baohua@kernel.org> Cc: Lance Yang <lance.yang@linux.dev> Cc: Usama Arif <usama.arif@linux.dev> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Doug Gilbert <dgilbert@interlog.com> Cc: James Bottomley <james.bottomley@hansenpartnership.com> Cc: Martin K. Petersen <mkp@kernel.org> Cc: Simona Vetter <simona@ffwll.ch> Cc: Helge Deller <deller@gmx.de> Cc: Sebastian Reichel <sre@kernel.org> Cc: John Hubbard <jhubbard@nvidia.com> Cc: Peter Xu <peterx@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Rik van Riel <riel@surriel.com> Cc: Harry Yoo <harry@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Thomas Zimmermann <tzimmermann@suse.de> Cc: David Airlie <airlied@gmail.com> Cc: Will Deacon <will@kernel.org> Cc: Aneesh Kumar K.V <aneesh.kumar@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: Matthew Wilcox (Oracle) <willy@infradead.org> Cc: Jan Kara <jack@suse.cz> Cc: Marc Zyngier <maz@kernel.org> Cc: Oliver Upton <oupton@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Anup Patel <anup@brainfault.org> Cc: Paul Walmsley <pjw@kernel.org> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Janosch Frank <frankja@linux.ibm.com> Cc: Claudio Imbrenda <imbrenda@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: David S. Miller <davem@davemloft.net> Cc: Andreas Larsson <andreas@gaisler.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christian Brauner <brauner@kernel.org> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Byungchul Park <byungchul@sk.com> Cc: Gregory Price <gourry@gourry.net> Cc: Ying Huang <ying.huang@linux.alibaba.com> Cc: Alistair Popple <apopple@nvidia.com> Cc: Chris Li <chrisl@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Youngjun Park <youngjun.park@lge.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Qi Zheng <qi.zheng@linux.dev> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Axel Rasmussen <axelrasmussen@google.com> Cc: Yuanchu Xie <yuanchu@google.com> Cc: Wei Xu <weixugc@google.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Michal Hocko <mhocko@kernel.org> Cc: Miklos Szeredi <miklos@szeredi.hu> Cc: Xu Xin <xu.xin@linux.dev> Cc: Ihor Solodrai <ihor.solodrai@linux.dev> Cc: Bert Karwatzki <spasswolf@web.de>
The comment above the VMA flag setup in hugetlbfs_file_mmap() says the VMA address alignment has already been checked by prepare_hugepage_range(). That function no longer exists: it was removed by commit eff4138 ("mm/hugetlb: remove prepare_hugepage_range()"), and the check now lives in hugetlb_get_unmapped_area(). The comment also mentions ia64, which was removed from the tree by commit cf8e865 ("arch: Remove Itanium (IA-64) architecture"). Point the comment at the current function and drop the ia64 reference. The rest of the comment, which tells future edits to add any error return only after VM_HUGETLB has been set, is still accurate and is left untouched. No functional change. Link: https://lore.kernel.org/tencent_AA61551D1F50E61F46C8542F5C8874512C05@qq.com Signed-off-by: Zhenghui Hao <zhenghui.hao@qq.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: David Hildenbrand <david@kernel.org>
Patch series "mm: implement and use vma_has_anon_rmap(), silence KCSAN". Provide a function to abstract the common task of checking whether a VMA has an anonymous reverse mapping associated with it. In the first patch, const-ify vma_assert_stabilised() and related functions so that vma_has_anon_rmap() can reference a const vma pointer. In the second patch, introduce vma_has_anon_rmap(). Finally in the third patch, update comments referencing anon_vma to instead reference the anon rmap to abstract this conceptually to avoid confusion. There are still other functions which reference anon_vma directly, those can be addressed in a follow up. This patch (of 3): The vma pointers are not modified in any of these functions so make it official by const-ify them. Propagate this throughout the call stack. No functional change intended. Link: https://lore.kernel.org/20260917-vma-is-faulted-v3-0-5c22314a72e7@kernel.org Link: https://lore.kernel.org/20260917-vma-is-faulted-v3-1-5c22314a72e7@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Zi Yan <ziy@nvidia.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Pedro Falcato <pfalcato@suse.de> Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org> Reviewed-by: Lance Yang <lance.yang@linux.dev> Cc: Alistair Popple <apopple@nvidia.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Barry Song <baohua@kernel.org> Cc: Byungchul Park <byungchul@sk.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Chris Li <chrisl@kernel.org> Cc: Dev Jain <dev.jain@arm.com> Cc: Gregory Price <gourry@gourry.net> Cc: Guilherme Giacomo Simoes <trintaeoitogc@gmail.com> Cc: Harry Yoo <harry@kernel.org> Cc: "Huang, Ying" <ying.huang@linux.alibaba.com> Cc: Jann Horn <jannh@google.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Muchun Song <muchun.song@linux.dev> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Peter Xu <peterx@redhat.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Rik van Riel <riel@surriel.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: xu xin <xu.xin16@zte.com.cn>
Provide a function to abstract the common task of checking whether
a VMA has an anonymous reverse mapping associated with it.
If the VMA is attached, a VMA or mmap lock must be held when calling this
function.
For an attached, anonymous, VMA:
Transition | VMA/mmap Lock state
-----------------------------|-------------------------------------------
No anon rmap to anon rmap | Write lock/read lock + mm->page_table_lock
Anon rmap to no anon rmap | Write lock
vma_has_anon_rmap() never provides a false positive (the lock precludes
it), but if only a read lock is held, a negative result must be re-checked
with mm->page_table_lock held.
A VMA obtains an anonymous reverse mapping when first faulted or forked
and it is removed when it is freed.
Detached VMAs cannot be concurrently manipulated as they are removed from
the maple tree so require no guarantees.
Use data_race() to silence KCSAN about non-existent data races between
concurrent vma->anon_vma read/write on optimistic fault tests.
Update the core VMA merge/split, rmap, mremap, KSM, fork,
khugepaged and fault preparation callers which test vma->anon_vma directly
to use vma_has_anon_rmap() instead.
Finally, update comments that reference anon_vma to reference the anon rmap
instead.
Since the lockless read in reusable_anon_vma() is doing more than checking
whether the VMA has anon rmap - it is returning the anon_vma to be used on
fault - do not alter it.
There is one odd one out - file_backed_vma_is_retractable() - which holds
neither a VMA nor mmap lock and is stabilised by the file rmap lock only,
so simply add a comment to explain why it's necessary.
Link: https://lore.kernel.org/20260917-vma-is-faulted-v3-2-5c22314a72e7@kernel.org
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reported-by: Guilherme Giacomo Simoes <trintaeoitogc@gmail.com>
Closes: https://lore.kernel.org/all/20260829100034.423064-1-trintaeoitogc@gmail.com/
Closes: https://lore.kernel.org/all/20260909115723.528501-1-trintaeoitogc@gmail.com/
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Barry Song <baohua@kernel.org>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Chengming Zhou <chengming.zhou@linux.dev>
Cc: Chris Li <chrisl@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: Harry Yoo <harry@kernel.org>
Cc: "Huang, Ying" <ying.huang@linux.alibaba.com>
Cc: Jann Horn <jannh@google.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Kairui Song <kasong@tencent.com>
Cc: Kemeng Shi <shikemeng@huaweicloud.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Nhat Pham <nphamcs@gmail.com>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Peter Xu <peterx@redhat.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Rik van Riel <riel@surriel.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: xu xin <xu.xin16@zte.com.cn>
Cc: Zi Yan <ziy@nvidia.com>
Now that vma_has_anon_rmap() abstracts whether a VMA has an anonymous reverse mapping, remove references to anon_vma and instead reference the anon rmap. The anon_vma is an implementation detail and should be treated as such. Do not update mm/rmap.c which implements the anon_vma mechanism as it is reasonable to directly reference it there. No functional change intended. [akpm@linux-foundation.org: fix comment, per Zi Yan] Link: https://lore.kernel.org/20260917-vma-is-faulted-v3-3-5c22314a72e7@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Pedro Falcato <pfalcato@suse.de> Reviewed-by: Lance Yang <lance.yang@linux.dev> Cc: Alistair Popple <apopple@nvidia.com> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Barry Song <baohua@kernel.org> Cc: Byungchul Park <byungchul@sk.com> Cc: Chengming Zhou <chengming.zhou@linux.dev> Cc: Chris Li <chrisl@kernel.org> Cc: Dev Jain <dev.jain@arm.com> Cc: Gregory Price <gourry@gourry.net> Cc: Guilherme Giacomo Simoes <trintaeoitogc@gmail.com> Cc: Harry Yoo <harry@kernel.org> Cc: "Huang, Ying" <ying.huang@linux.alibaba.com> Cc: Jann Horn <jannh@google.com> Cc: Joshua Hahn <joshua.hahnjy@gmail.com> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Kiryl Shutsemau <kas@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Matthew Brost <matthew.brost@intel.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Muchun Song <muchun.song@linux.dev> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Oscar Salvador <osalvador@suse.de> Cc: Peter Xu <peterx@redhat.com> Cc: Rakie Kim <rakie.kim@sk.com> Cc: Rik van Riel <riel@surriel.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Shakeel Butt <shakeel.butt@linux.dev> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: xu xin <xu.xin16@zte.com.cn> Cc: Zi Yan <ziy@nvidia.com>
When updating a VMA, vma_prepare() unconditionally removes it from its
rmap interval trees under the rmap lock, and vma_complete() reinserts it
before releasing the lock.
This is wholly unnecessary if its page offset (file rmap) or anonymous
page offset (anon rmap) is unchanged.
So, track whether they will change in the newly introduced
vp->file_pgoff_unchanged and vp->anon_pgoff_unchanged fields, and use them
to determine whether to remove the VMA or not.
The rmap lock keeps things safe as no rmap walks can concurrently occur
during the operation.
Additionally, some architectures (arm, parisc, nios2, csky) have dcache
flush rmap walkers which take only flush_dcache_mmap_lock(), which is
likewise held across the operation.
If the VMA remains in the tree, it's necessary to keep the augmented
rb_subtree_last field updated to reflect its changed range.
Provide mapping_rmap_tree_[pre, post]_update() and anon_rmap_tree_[pre,
post]_update_vma() (replacing the existing logic in the anonymous case) to
handle both the changed and unchanged cases.
For the anon rmap case, with CONFIG_DEBUG_VM_RB set, avc->cached_vma_last
is also updated when propagating in place.
When performing a VMA shrink or a split where the VMA is the lower one,
the page offset cannot change, so set the flags unconditionally in these
cases.
When merging VMAs the page offset is unchanged only in some cases, so
update init_multi_vma_prep() to set the flags only if the page offsets
remain the same.
Finally, while we're here, also update expand_upwards() similarly.
These changes ultimately result in less rmap lock contention.
Pan Deng reported results using the UnixBench/excel benchmark on a
2-socket 192 core, 384 thread x86-64 system for v7.3-rc4 with/without the
patch applied:
Execl Throughput, index score:
avg %stdev min max
v7.3-rc4 3511.5 0.44% 3494.5 3543.4
+ patch 4069.0 0.48% 4047.4 4109.5 (+15.9%)
Average wait on file rmap lock in ms, 5 runs per kernel:
avg %stdev min max
v7.3-rc4 9.470 2.91% 9.070 9.820
+ patch 8.420 3.34% 8.100 8.770 (-11.1%)
Profiling data obtained during the operation highlighted the file rmap lock
as the primary source of contention.
Link: https://lore.kernel.org/20260930-speed-up-inplace-rmap-v2-1-ac1aa19708aa@kernel.org
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Pan Deng <pan.deng@intel.com>
Reviewed-by: Rik van Riel <riel@surriel.com>
Acked-by: Lance Yang <lance.yang@linux.dev>
Tested-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Pedro Falcato <pfalcato@suse.de>
Cc: David Hildenbrand <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Harry Yoo <harry@kernel.org>
Cc: Jann Horn <jannh@google.com>
Each row of a memcg-aware shrinker's debugfs count file contains the cgroup ID followed by one object count per NUMA node. memcg_shrinker.py uses only the first count when sorting and displaying entries. This undercounts objects on NUMA systems. An entry with counts of 0 and 23 is treated as zero and omitted from the output, even though its total is 23. Sum all count fields after the cgroup ID so that the displayed totals and sort order account for every node. Single-node input retains the same result. Link: https://lore.kernel.org/179074810112.139422.9274126921568167859@gmail.com Fixes: d261ea2 ("tools: add memcg_shrinker.py") Signed-off-by: Ren Tamura <ren.tamura.oss@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Assisted-by: LLM Cc: Roman Gushchin <roman.gushchin@linux.dev>
Since block swap I/O is batched in struct swap_iocb, swap_add_folio() only appends a folio to the pending batch when its slot directly follows the previous one, and submits the pending batch otherwise. Batches submitted inside the readahead loop go through the plug, where the block layer can still merge them, but the final batch is submitted after blk_finish_plug() and always becomes a separate request. So submit the final batch before finishing the plug. Link: https://lore.kernel.org/20261001085730.2029773-1-alex@ghiti.fr Fixes: dda8fb6 ("mm/swap: also use struct swap_iocb for block I/O") Signed-off-by: Alexandre Ghiti <alex@ghiti.fr> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Joshua Hahn <joshua.hahnjy@gmail.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Chris Li <chrisl@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Nhat Pham <nphamcs@gmail.com> Cc: Baoquan He <baoquan.he@linux.dev> Cc: Barry Song <baohua@kernel.org> Cc: Youngjun Park <youngjun.park@lge.com>
The HUGETLB SUBSYSTEM F: entries list include/linux/hugetlb.h explicitly, but other hugetlb headers are not covered, so get_maintainer.pl does not attribute them to the hugetlb maintainers. Enumerating headers by hand is also fragile: any header not listed (such as hugetlb_cgroup.h) falls outside the subsystem. Joe Perches suggested using a glob F: entry, which covers all current and future hugetlb headers in one line and avoids the gap. Link: https://lore.kernel.org/20261003060029.2152489-1-lienze@kylinos.cn Signed-off-by: Enze Li <lienze@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Suggested-by: Joe Perches <joe@perches.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: SJ Park <sj@kernel.org> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de>
The final #endif that closes the _LINUX_HUGETLB_CGROUP_H include guard is left without a comment. Add the macro name to the comment for consistency and easier navigation. Link: https://lore.kernel.org/20261003060029.2152489-2-lienze@kylinos.cn Signed-off-by: Enze Li <lienze@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Muchun Song <muchun.song@linux.dev> Reviewed-by: SJ Park <sj@kernel.org> Cc: Oscar Salvador <osalvador@suse.de>
mrelease_test doubles the child allocation when process_mrelease() returns ESRCH. When size is already MAX_SIZE_MB, the current <= check still allows another retry, causing the allocation to grow from 1024 MB to 2048 MB. Use < instead of <= so the largest allocation attempted remains MAX_SIZE_MB. LLM used in discovering bug. Changes were made and reviewed manually. Link: https://lore.kernel.org/CANOyQmFzsssM_BXHUDrV+UuVD5SZMBmSkg3UQnmw9Ns1PV7GCQ@mail.gmail.com Fixes: 3377614 ("selftests: vm: add process_mrelease tests") Signed-off-by: Aveline Noir <jm5905938@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Reviewed-by: SJ Park <sj@kernel.org> Assisted-by: LLM Cc: Shuah Khan <shuah@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Mike Rapoport <rppt@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Christian Brauner <brauner@kernel.org> Cc: Muhammad Usama Anjum <usama.anjum@arm.com>
kmsan_vmalloc_to_page_or_null() rejects addresses outside the regular vmalloc and module ranges, including the shadow and origin addresses passed by its callers. As a result, iounmap does not free the metadata backing pages. Also, the first iteration of the teardown loop unmaps the entire metadata range, preventing subsequent iterations from finding their pages even if the address lookup is fixed. Factor the page-table walk out of vmalloc_to_page() into __vmalloc_to_page(), keeping the address check in the public wrapper. Use the unchecked helper in kmsan_vmalloc_to_page_or_null() and check for NULL before converting the returned page to a PFN. Mark __vmalloc_to_page() as __always_inline to avoid an additional function call in vmalloc_to_page(). Move metadata teardown into kmsan_iounmap_pages(). Free the backing blocks while their mappings are still available, then unmap each metadata range once and flush its TLB entries. Link: https://lore.kernel.org/20261002200508.546-1-dmytrokoziuk68@gmail.com Fixes: b073d7f ("mm: kmsan: maintain KMSAN metadata for page operations") Signed-off-by: Dima Koziuk <dmytrokoziuk68@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Suggested-by: Alexander Potapenko <glider@google.com> Link: https://lkml.iu.edu/2609.3/12748.html Cc: Marco Elver <elver@google.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Uladzislau Rezki (Sony) <urezki@gmail.com>
Looking further at kmsan_ioremap_page_range(), I found three cases where error cleanup leaks metadata blocks. 1. If the first iteration fails, clean is zero and cleanup is skipped, leaking any allocations that succeeded in that iteration. 2. If shadow mapping succeeds but origin mapping fails, the shadow pointer has already been cleared. Removing its mapping loses the backing block. 3. On failures after completed iterations, cleanup removes the earlier metadata mappings without freeing their backing blocks. The cleanup needed here is the same as for iounmap, so it makes sense to reuse kmsan_iounmap_pages(). Track the end of installed mappings with mapped_end and advance it after each successful shadow mapping. This includes the current shadow block if origin mapping subsequently fails, while the helper skips the missing origin mapping. Run cleanup whenever err is non-zero. Free allocations that have not been mapped directly, and use the shared helper to unmap and free the installed metadata, including blocks from completed iterations. Link: https://lore.kernel.org/20261002200508.546-2-dmytrokoziuk68@gmail.com Fixes: fdea03e ("mm: kmsan: handle alloc failures in kmsan_ioremap_page_range()") Signed-off-by: Dima Koziuk <dmytrokoziuk68@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Alexander Potapenko <glider@google.com> Cc: Marco Elver <elver@google.com> Cc: Dmitry Vyukov <dvyukov@google.com> Cc: Uladzislau Rezki (Sony) <urezki@gmail.com>
…e_noflush() __vmap_pages_range_noflush() currently only warns when page_shift is smaller than PAGE_SHIFT, then continues with the invalid shift. Return -EINVAL instead so the bogus argument is caught immediately. Link: https://lore.kernel.org/20261004071639.119857-1-jiangwenxiaomi@gmail.com Signed-off-by: Wen Jiang <jiangwen6@xiaomi.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Suggested-by: Uladzislau Rezki (Sony) <urezki@gmail.com> Reviewed-by: Barry Song <baohua@kernel.org> Reviewed-by: Uladzislau Rezki (Sony) <urezki@gmail.com> Reviewed-by: Vishal Moola (Fractile) <vishal.moola@gmail.com> Cc: Dev Jain <dev.jain@arm.com>
The per-node hugepages section writes the free_hugepages and surplus_hugepages file names with stray trailing quotes. Quote the real names as inline literals instead. Link: https://lore.kernel.org/20261003161125.40485-1-lienze@kylinos.cn Signed-off-by: Enze Li <lienze@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: David Hildenbrand <david@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net>
Every test is expected to reserve the HugeTLB pages it needs. hugetlb-read-hwpoison was not converted, so on a system without pre-reserved huge pages the MAP_POPULATE mmap() fails with ENOMEM and every test case is skipped. Reserve the pages with hugetlb_setup_default(). Each chunk size runs two HWPOISON tests and a poisoned huge page can't be reused, so reserve two pages per chunk size. The pool size is restored on exit. The poisoned pages remain allocated until reboot. With the patch, all 12 cases pass on a system with no huge pages reserved (tested on Fedora, x86_64, CONFIG_MEMORY_FAILURE=y). Link: https://lore.kernel.org/20261004205458.119608-1-jaeyeon.lee.dev@gmail.com Signed-off-by: Jaeyeon Lee <jaeyeon.lee.dev@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Tested-by: Sarthak Sharma <sarthak.sharma@arm.com> Reviewed-by: Sarthak Sharma <sarthak.sharma@arm.com> Assisted-by: LLM Cc: David Hildenbrand <david@kernel.org> Cc: Shuah Khan <shuah@kernel.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com>
On systems/VMs with a large number of CPUs (e.g. 4096), hugetlb_init() computes num_fault_mutexes = roundup_pow_of_two(8 * num_possible_cpus()) = 32768. The subsequent kmalloc for the mutex table can exceed MAX_PAGE_ORDER when struct mutex is enlarged by debug options like CONFIG_DEBUG_MUTEXES and CONFIG_DEBUG_LOCK_ALLOC, resulting in: WARNING: at __alloc_frozen_pages_noprof (order > MAX_PAGE_ORDER) kernel BUG at mm/hugetlb.c (BUG_ON(!hugetlb_fault_mutex_table)) Kernel panic - not syncing: Fatal exception Switch to kvmalloc_objs() so the allocation falls back to vmalloc when the contiguous physical allocation is too large. The table is only accessed by index, so virtual contiguity is sufficient. Link: https://lore.kernel.org/20261006160128.31447-1-dheerajkumar.srivastava@amd.com Fixes: 8382d91 ("mm, hugetlb: improve page-fault scalability") Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: David Hildenbrand <david@kernel.org> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: Vasant Hegde <vasant.hegde@amd.com> Cc: Bharata B Rao <bharata@amd.com>
Patch series "support kselftest on nommu platform", v5. This patchset adds an ability to execute kselftest on nommu platforms. Currently there are several issues if we wish to run kselftests on nommu targets: - it cannot compile/build test binaries because the current files mainly assume to build with glibc, - some of the tests are not able to run on nommu targets as there are no fork(2) syscall. The first issue can be avoided if we can build static PIE binaries (if targets support it), but in our case (build on ubuntu/glibc and run on alpine/musl-libc), it fails to invoke due to lack of the GNU ifunc mechanism. Thus, we need to cross-compile with musl toolchain, which needs to be solved the first issue. The second issue is the lack of fork(2) syscall on those platforms. Especially the test harness helper (kselftest_harness.h) uses the syscall, which cannot be simply with vfork(2). `timeout` command used in `runner.sh` never works for nommu platform as it uses fork(2). nommu component in the mm subsystem has several known issues and having test cases should help this situation, thus this patchset is very first step toward enriching test environment which has not been well tested for a while. The test cases is implemented based on the document (Documentation/admin-guide/mm/nommu-mmap.rst). Initial shape for this patches is implemented as an opt-in TARGET, not built/invoked automatically if users don't specify this nommu target. This patch (of 2): Architectures lacking MMU don't support fork(2) syscall and only vfork(2) is available with limitations. Thus, we cannot run kselftest on nommu architecture as is. This commit addresses this issue with the following changes: - Detect NOMMU at runtime from the MmapCopy field in /proc/meminfo and avoid timeout, which requires fork(2). - Document how to build and run the NOMMU tests, including the procfs prerequisite for runtime detection. Build and run the tests with: $ make ARCH=um O=build kselftest-all TARGETS=nommu $ make ARCH=um O=build kselftest-install TARGETS=nommu $ ./build/kselftest/kselftest_install/run_kselftest.sh -p -c nommu Link: https://lore.kernel.org/20261006085642.3852152-1-thehajime@gmail.com Link: https://lore.kernel.org/20261006085642.3852152-2-thehajime@gmail.com Signed-off-by: Hajime Tazaki <thehajime@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Closes: https://sashiko.dev/#/patchset/20260813063401.1786548-1-thehajime%40gmail.com Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Shuah Khan <shuah@kernel.org> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Kees Cook <kees@kernel.org> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Will Drewry <wad@chromium.org> Cc: Mark Brown <broonie@kernel.org> Cc: Brendan Jackman <brendan.jackman@linux.dev> Cc: Hangbin Liu <liuhangbin@gmail.com> Cc: "Ricardo B. Marliere" <rbm@suse.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Lorenzo Stoakes <ljs@kernel.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Daniel Palmer <daniel@thingy.jp>
Introduce a kselftest utility to validate memory mapping capabilities under nommu kernels, aligned with Documentation/admin-guide/mm/nommu-mmap.rst. The test implements basic checks into a generic architecture-agnostic test matrix applicable across nommu targets. It evaluates: 1. MAP_FIXED allocation rejections. 2. Standard MAP_PRIVATE and MAP_ANONYMOUS allocation resilience. 3. Regular file mappings via standard filesystem storage. 4. Block device subsystem mappings (gracefully skipping if node is missing). 5. Shared vs Private backing discrepancies under nommu conditions. 6. mremap limits, ensuring non-expandable restrictions behave properly. Link: https://lore.kernel.org/20261006085642.3852152-3-thehajime@gmail.com Signed-off-by: Hajime Tazaki <thehajime@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Closes: https://sashiko.dev/#/patchset/20260825015945.141739-1-thehajime@gmail.com Assisted-by: Gemini:Pro [AI_Reviewer] [Sashiko_Linter] Assisted-by: cubic.dev:unspecified Cc: David Hildenbrand <david@kernel.org> Cc: "Liam R. Howlett" <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Mike Rapoport <rppt@kernel.org> Cc: Suren Baghdasaryan <surenb@google.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Shuah Khan <shuah@kernel.org> Cc: Ricardo B . Marlière <rbm@suse.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Brendan Jackman <brendan.jackman@linux.dev> Cc: Hangbin Liu <liuhangbin@gmail.com> Cc: Jonathan Corbet <corbet@lwn.net> Cc: Kees Cook <kees@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Will Drewry <wad@chromium.org> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Daniel Palmer <daniel@thingy.jp>
Patch series "mm/damon: fix a minor bug, clean code, and add a test". Patch 1 from Jaeyeon Lee fixes a minor overflow issue in DAMOS. Patch 2 from Gregory Price (Meta) cleans up the code for handling of hugetlb pages. Patch 3 from Liew Rui Yan adds a kunit test for a recently fixed bug. This patch (of 3): damon_hot_score() calculates the age of a region by multiplying r->age and the aggregation interval. On 32-bit architectures, unsigned long is 32 bits, so the multiplication overflows once the age exceeds about 4295 seconds. Then old regions are scored as if they were young. Fix it by doing the calculation in 64-bit arithmetic using div_u64(). Also make age_in_sec u64, so the result is not truncated back to 32 bits. The issue was discovered [1] by Sashiko. Link: https://lore.kernel.org/20261006065209.41507-1-sj@kernel.org Link: https://lore.kernel.org/20261006065209.41507-2-sj@kernel.org Link: https://lore.kernel.org/20260623011652.1354-1-sj@kernel.org [1] Fixes: 198f0f4 ("mm/damon/vaddr,paddr: support pageout prioritization") Signed-off-by: Jaeyeon Lee <jaeyeon.lee.dev@gmail.com> Signed-off-by: SJ Park <sj@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: SJ Park <sj@kernel.org> Assisted-by: LLM
Hugetlb folio acquisition is open-coded in three places: - damon_hugetlb_mkold() - damon_young_hugetlb_entry() - damon_va_probe_hugetlb_entry() DAMON already has damon_get_monitor_folio(), which paddr uses to get monitored LRU and hugetlb folios and to filter them through damon_folio_acceptable(). These paths bypass that check. Use that helper in these paths too, so that DAMON acquires hugetlb folios and applies its acceptance policy in one place. No functional change intended. All callers reach these paths with a present hugetlb PTE under the page table lock, so the folio is always a mapped hugetlb folio and the helper always returns it. Link: https://lore.kernel.org/20261006065209.41507-3-sj@kernel.org Signed-off-by: Gregory Price (Meta) <gourry@gourry.net> Signed-off-by: SJ Park <sj@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: SJ Park <sj@kernel.org>
…last region Commit b3723b5 ("mm/damon/core: fix unconditionally skip last region") fixed a minor bug. Add a KUnit test case to cover this behavior and prevent regressions in the future. Link: https://lore.kernel.org/20261006065209.41507-4-sj@kernel.org Signed-off-by: Liew Rui Yan <aethernet65535@gmail.com> Signed-off-by: SJ Park <sj@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: SJ Park <sj@kernel.org> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: David Gow <davidgow@davidgow.net>
If hugetlb_add_to_page_cache() in memfd_alloc_folio() fails with -EEXIST, a concurrent fault has already instantiated the folio in the page cache, and the reservation now belongs to that folio. Calling hugetlb_unreserve_pages() in that case incorrectly removes the region backing the cached folio and releases one reservation more than it should, leaving that folio in the page cache with no region recording it. That leaves resv_huge_pages one page short until that folio is removed from the page cache, so available_huge_pages() reports a page that is not actually free and the pool can grant one reservation more than it can back. Applications using hugetlb memfds can then fail to allocate or fault in a page they already reserved. They see -ENOMEM or -ENOSPC from the allocation or fault path even though the reservation and HugePages_Free still look healthy. So hold the hugetlb fault mutex from hugetlb_reserve_pages() until the error-path unreserve completes to make the reserve, allocate and instantiate steps atomic against concurrent faults. With the mutex held from the start, a concurrent fault can no longer consume the reservation between reserve and allocate/instantiate. If a fault completed before the mutex was taken, it has already added the region for that index, so hugetlb_reserve_pages() returns 0 and the error path leaves the region in place. Link: https://lore.kernel.org/20260927094757.31665-1-hongfu.li@linux.dev Link: https://lore.kernel.org/20260903030134.7407-1-hongfu.li@linux.dev Fixes: 717cf93 ("mm/memfd: reserve hugetlb folios before allocation") Signed-off-by: Hongfu Li <lihongfu@kylinos.cn> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Cc: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Hugh Dickins <hughd@google.com> Cc: Muchun Song <muchun.song@linux.dev> Cc: Oscar Salvador <osalvador@suse.de> Cc: Vivek Kasireddy <vivek.kasireddy@intel.com> Cc: <stable@vger.kernel.org>
snapshot_set_swap_area() unpins the previously selected swap device and pins the new one in two separate swap_lock critical sections. In the gap between them, swapoff() observes SWP_HIBERNATION cleared, bypasses the guard, and tears down the device, reopening the race the SWP_HIBERNATION pin was meant to close. The window is reachable on any SNAPSHOT_SET_SWAP_AREA call after the snapshot device is opened for hibernation, and on any retry after the resume path's first selection. Add repin_hibernation_swap_type(), which looks up the new device, clears the old SWP_HIBERNATION flag and sets the new one under a single swap_lock acquisition. The same-device case is short- circuited so userspace can re-select the same swap area without tripping WARN_ON_ONCE and -EBUSY. Switch snapshot_set_swap_area() to the new helper. A failed lookup now preserves the previous pin instead of dropping it, so a bad SNAPSHOT_SET_SWAP_AREA leaves the prior selection intact. The open and release paths keep using pin_hibernation_swap_type() and unpin_hibernation_swap_type(). The race was identified during AI-assisted review of the SWP_HIBERNATION pinning series. [akpm@linux-foundation.org: coding-style cleanups] Link: https://lore.kernel.org/20260430195651.287659-1-devnexen@gmail.com Fixes: 8e6e0d845823 ("mm/swap, PM: hibernate: fix swapoff race in uswsusp by pinning swap device") Signed-off-by: David Carlier <devnexen@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Assisted-by: Codex (gpt-5-codex) Cc: Baoquan He <bhe@redhat.com> Cc: Barry Song <baohua@kernel.org> Cc: Chris Li <chrisl@kernel.org> Cc: Kairui Song <kasong@tencent.com> Cc: Kemeng Shi <shikemeng@huaweicloud.com> Cc: Len Brown <lenb@kernel.org> Cc: Nhat Pham <nphamcs@gmail.com> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: DaeMyung Kang <charsyam@gmail.com> Cc: Youngjun Park <youngjun.park@lge.com>
Pretty much every pr_* message in this file starts with "HugeTLB: ", typed
out by hand, and one of them even drifted to lowercase ("hugetlb:").
hugetlb_vmemmap.c already handles this with pr_fmt, so let's do the same
here instead of repeating the prefix over and over.
The output stays the same for almost everything; the only tweaks are that
the lowercase message now matches the others, and the ratelimited "killed
due to inadequate hugepage pool" warning picks up the prefix like
everything else.
Link: https://lore.kernel.org/20261010092333.137126-1-lienze@kylinos.cn
Signed-off-by: Enze Li <lienze@kylinos.cn>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Muchun Song <muchun.song@linux.dev>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: David Hildenbrand <david@kernel.org>
Cc: Enze Li <enze.li@gmx.com>
…ut_pte_range() During stress testing with stress-ng on a 256-CPU system, a softlockup was observed when madvise(MADV_COLD) was called on a 31GB memory region backed by transparent hugepages. The region contains 15935 THP PMD entries, and the softlockup was triggered after 14485 of them had been processed. The THP branch of madvise_cold_or_pageout_pte_range() has no scheduling point. walk_pmd_range() invokes it once per 2MB PMD entry with no cond_resched() anywhere in the loop. Each iteration calls pmdp_invalidate() which issues a synchronous TLB shootdown (dsb ish) waiting for all CPUs to acknowledge. This operation can be time-consuming under heavy system load, causing the accumulated time across all PMD entries to exceed the softlockup threshold (20s) and panic when softlockup_panic=1. On a CONFIG_PREEMPT_NONE=y kernel, there is no other opportunity to reschedule during this entire traversal. The regular PTE path already has an unconditional cond_resched() at the function tail (introduced by commit 9c276cc ("mm: introduce MADV_COLD") and commit 1a4e58c ("mm: introduce MADV_PAGEOUT") when MADV_COLD and MADV_PAGEOUT were first introduced) and a conditional one inside the loop (commit b2f557a ("mm/madvise: add cond_resched() in madvise_cold_or_pageout_pte_range()")). The THP branch never received equivalent treatment. Fix by adding cond_resched() at the huge_unlock exit path after spin_unlock(ptl) and reclaim_pages(), which is the single exit point for all goto paths in the THP branch. Crash log: watchdog: BUG: soft lockup - CPU#63 stuck for 23s! [stress-ng-vm:1665914] ... Call trace: pmdp_invalidate+0x48/0x68 madvise_cold_or_pageout_pte_range+0x84c/0xb00 walk_pmd_range.isra.0+0x1ac/0x3b0 walk_pud_range+0x120/0x190 walk_pgd_range+0x170/0x1b8 __walk_page_range+0x184/0x198 walk_page_range_vma+0x94/0x128 madvise_cold+0xd8/0x198 madvise_vma_behavior+0x234/0x638 madvise_walk_vmas.constprop.0+0xdc/0x210 madvise_do_behavior+0xa8/0x328 do_madvise+0xd8/0x140 __arm64_sys_madvise+0x2c/0xe0 Link: https://lore.kernel.org/20261010024042.4070048-1-albin_yang@163.com Signed-off-by: Wei Yang <albinwyang@tencent.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Gregory Price (Meta) <gourry@gourry.net> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: <stable@vger.kernel.org>
__discard_anon_folio_pmd_locked() clears a lazyfree THP's PMD before it determines whether the folio can be discarded, and restores it if the folio is dirty or has extra references. It is done under only the PMD lock and the anon_vma read lock inside try_to_unmap_one() and no mmap lock is held. The temporarily none PMD causes UAF. A parallel munmap() or mremap() is supposed to serialize with it on the PMD lock, but zap_pmd_range() (called by munmap()) and move_page_tables() (called by mremap()) do not take the lock when they see the temporarily none PMD and skip the PMD. There is no more page table walk after the none PMD is restored: - munmap(): free_pgtables() waits in unlink_anon_vmas() for the anon_vma lock, then free_pmd_range() drops the restored PMD as bad without removing the rmap, so the folio keeps a stale mapcount. - whole-VMA MREMAP_DONTUNMAP: dontunmap_complete() unlinks the source VMA from its anon_vma while the restored PMD still maps the folio. - plain mremap(): the THP is not moved but unmapped, so the THP's data is lost. In the first two cases, the folio stays mapped, but no VMA that maps it is linked to its anon_vma. When the anon_vma is freed, it is still reachable when one rmap walks the folio. Racing lazyfree reclaim against MREMAP_DONTUNMAP as an unprivileged user reproduces a KASAN use-after-free in folio_lock_anon_vma_read() as reported in the Closes link. One solution is to use an invalidated PMD instead of a cleared one until the discard is committed, but it needs arch fixes first. 1. GUP-fast still follows an invalidated PMD on riscv, LoongArch and s390. 2. sparc64 miscounts thp_pte_count when an invalidated PMD is set again or cleared. To keep the fix simple for backport, stop discarding lazyfree THPs at PMD level instead. try_to_unmap_one() splits the PMD and discards the folio at PTE level. unmap_huge_pmd_locked() and __discard_anon_folio_pmd_locked() are left in place, unused, to minimize this fix for backport and the invalidated PMD approach is planned as a follow-up series to use them again. Link: https://lore.kernel.org/20261012-lazyfree-stable-fix-v2-1-6387d6e30063@nvidia.com Fixes: 735ecdf ("mm/vmscan: avoid split lazyfree THP during shrink_folio_list()") Signed-off-by: Zi Yan <ziy@nvidia.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reported-by: Kyle Zeng <kylebot@openai.com> Closes: https://lore.kernel.org/r/20261009165214.40212-2-kylebot@openai.com Reviewed-by: Kyle Zeng <kylebot@openai.com> Reviewed-by: Lance Yang <lance.yang@linux.dev> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org> Tested-by: JongHyuk Kim(MalHyuk) <malhyuk97@gmail.com> Link: https://lore.kernel.org/CADQRuqvXQ=YE=7T3oR66vTxE33pi=ygEeZKPTOU=cALZL0Figw@mail.gmail.com Assisted-by: LLM Cc: Rik van Riel <riel@surriel.com> Cc: Liam R. Howlett <liam@infradead.org> Cc: Vlastimil Babka <vbabka@kernel.org> Cc: Harry Yoo <harry@kernel.org> Cc: Jann Horn <jannh@google.com> Cc: <stable@vger.kernel.org>
Deflate still breaks a parked block into pages and asks Xen to populate
each with its own order-0 extent, which lets Xen back the block with any
512 frames it has.
Populate a parked block as a single order-9 extent instead, so that Xen
allocates one machine-contiguous 2 MiB run for it:
- On PVH Xen maps that run with a single 2 MiB EPT entry rather than
512 4 KiB ones.
- On PV there is no EPT, and the guest cannot use 2 MiB mappings. The
guest maps the run in itself, with the batched contiguous-run
helper, from the base machine frame Xen reports.
The p2m leaves of each block are preallocated while walking the block
list, before the populate hypercall: mapping the block afterwards cannot
allocate, and once Xen has populated an extent there is no way to hand
it back. Failing there only shortens the batch.
A block is only an option here too. balloon_deflate() falls back to the
page at a time path when a block makes no progress: Xen refuses an
order-9 extent when the host has no free 2 MiB run, or when the domain
is less than 2 MiB below its max_pages ceiling, and neither should stop
the guest growing by what it can still get. Credits smaller than a block
take the page at a time path as well.
Blocks are populated at most BALLOON_BLOCK_BATCH at a time, which bounds
the work done under balloon_mutex; the OOM notifier can only trylock it.
Backends: PV, PVH
Direction: deflate (frames move from Xen to the guest)
Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
The OOM notifier deflates through increase_reservation(), which works a page at a time and reaches parked blocks only by breaking them up. Deflate through balloon_deflate() instead, so that an OOM deflate populates a whole block as a single order-9 extent where Xen can supply one, and still falls back to single pages where it cannot, including the last 2 MiB below the domain's max_pages ceiling. Backends: PV, PVH Direction: deflate (frames move from Xen to the guest) Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
xen_init_time_common() points x86_platform.calibrate_tsc at xen_tsc_khz() but leaves calibrate_cpu native, so tsc_early_init() -> determine_cpu_tsc_frequencies() still calls native_calibrate_cpu_early() for cpu_khz. That tries CPUID 0x16, then MSR_PLATFORM_INFO, and then falls through to quick_pit_calibrate() -- which on a PVH guest waits out its full 50ms timeout on a PIT that is not there, prints "Fast TSC calibration failed", and arrives at a value the hypervisor had already been asked for a moment later. It is 48ms of every boot, and it is the largest single item in one: a 512 MiB PVH guest reached "Run /init as init process" at 0.330s, and 0.118s once this and the xenbus boot wait were out of the way. There is nothing for the calibration to find out here. Xen reports the TSC frequency, tsc_khz already comes from it, and cpu_khz differing from tsc_khz under a hypervisor is not information -- determine_cpu_tsc_frequencies() would overwrite cpu_khz with tsc_khz anyway once they disagree by more than 10%. Note that tsc_early_khz= does not help: it only short-circuits calibrate_tsc, and the 50ms is spent in calibrate_cpu. Set in xen_init_time_common() so PV and PVH/HVM guests both get it, which is where calibrate_tsc is already set for the same reason. Signed-off-by: Alex Zenla <alex@edera.dev>
boot_wait_for_devices() sleeps HZ/10 between checks, so on a CONFIG_HZ=100 kernel it waits a tenth of a second to notice something that has already happened. What it is waiting for is not slow: a toolstack has every backend sitting in XenbusStateInitWait before it unpauses the guest, so the wait covers the frontend half of the handshake, which finishes a few milliseconds after the late_initcall starts looking. Measured on a PVH guest, the last frontend connected at 0.232s and init started at 0.311s -- 79ms of sleeping past the event. Frontends move to XenbusStateConnected (and blkfront sets is_ready) from their otherend_changed callback, which runs from the frontend bus's backend_changed() watch. Bump a sequence count and wake a wait queue there, and have wait_for_devices() sleep on it instead of for a fixed interval. The sequence is sampled before each check, so a change that lands between the check and the sleep still wakes the waiter. The HZ/10 interruptible sleep stays as the timeout, so state changes that do not come through the watch are noticed no later than before, and the 30/270-second budgets and their reporting are unchanged. Signed-off-by: Alex Zenla <alex@edera.dev>
Port the nightly upstream rebase from the Xen tree to the kernel. Every night at 03:23 UTC, Claude rebases edera/mainline onto torvalds master, following .automation/skills/upstream-rebase/SKILL.md: it replays the series, resolves conflicts, fixes what the replay broke, builds, audits upstream fixes that meet the series, and writes a report. The model's own account of its result decides nothing. It runs with a read-only token; its branch leaves the job as a git bundle and is judged by scripts taken from the commit the run started from: - verify-upstream-rebase.sh (unchanged from Xen) proves the series sits on the upstream tip, is linear, kept every commit, and moved the tree by exactly the upstream delta; - kernel-build.sh builds x86_64 and arm64 vmlinux with the Hyper-V, Xen dom0/backend, PV-IOMMU, NUMA affinity and OpenPaX options built in, and fails if olddefconfig drops any of them. If both pass and the report asks for no decision, edera/mainline is force-pushed with a lease pinned to the tip the rebase started from, and the old tip is kept as a backup branch. Otherwise the result goes up as a pull request carrying the report, and approving it lands it (rebase-land.yml). A night with no result opens an issue. Every job clones blobless: the jobs need the kernel's commit graph but only the contents of the files the series and upstream touch.
The rebase runs from a scheduled or dispatched event and needs its own federation rule rather than the organization-wide one, so read it from vars.REBASE_FEDERATION_RULE_ID.
edera/6.18-lts carries its own rebase automation, but scheduled workflows run only from the default branch. Add a workflow here whose only job is to dispatch edera/6.18-lts's kernel-nightly.yml each night.
OPENPAX currently defaults to y, which means OpenPaX gets enabled for any configuration that picks up the new symbol, including ones that never asked for it. Default it to n so it has to be selected explicitly. While at it, make OPENPAX_SOFTMODE depend on OPENPAX. Soft mode does nothing without OpenPaX enabled, so there is no reason for it to be selectable (and default y) on its own. Signed-off-by: Alex Zenla <alex@edera.dev> (cherry picked from commit 514cd60)
edera/6.18-lts proposes a backport of each pull request merged into edera/mainline, driven by its own workflows. Add a workflow here whose only job is to dispatch that, skipping pull requests labelled no-backport and the nightly's rebase pull requests.
xen-pciback seizes a hidden device in an fs_initcall, but the Xen IOMMU registered from a device_initcall. Registering probes every PCI device that already exists, so a hidden device was found with a driver bound and the IOMMU core warned about a late probe at driver bind. Register from subsys_initcall_sync instead, so the IOMMU is in place before pciback binds anything. Signed-off-by: Alex Zenla <alex@edera.dev>
Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
All of the extra memory Linux reserves at boot goes to the mapping pool, so dom0's balloon starts empty. Growing dom0 at runtime then means memory hotplug, which places the new range at the first free physical address: on PVH that is the host's own map, where nothing guarantees the range is not a device's MMIO window. Keep the part of the extra memory dom0 may grow into on the balloon instead. Xen's ceiling for dom0 (dom0_mem=...,max:) sizes that part, so a host booted without a max: keeps today's behaviour. Every region gives up the same fraction, which spreads the growth over the NUMA nodes on PVH and leaves each node's mapping pool its share, and the last memory section stays with the pool so that early grant mappings never fall back to holes in the physical map. Signed-off-by: Alexander M. Merritt <alexander@edera.dev>
A request that does not fit in its ring sleeps in io_wait_event_killable() until the backend frees space, but the event handler wakes the ring with wake_up_interruptible(), which only wakes TASK_INTERRUPTIBLE sleepers. A TASK_KILLABLE waiter therefore sleeps until it is killed, which hangs a guest that has enough large requests in flight to fill a ring. Wake the ring with wake_up() so killable waiters see the freed space. Fixes: 9523fea ("net/9p: Switch to wait_event_killable()") Signed-off-by: Alex Zenla <alex@edera.dev>
The frontend's resume handler only warned that suspend and resume are unsupported. After a restore its rings, grant references and event channels are gone, yet the mounted client kept writing requests into them, so every request on the mount hung. The next InitWait from the backend then allocated a second frontend instance for the same device, which no client used. On resume, collect any replies the backend left in the rings, free the rings, fail the requests that are still unanswered with -EIO, and publish Initialising so that a backend which still serves the old rings starts over. When the backend reaches InitWait, allocate and publish new rings for the existing frontend, keeping its clients. Requests that arrive while the frontend is not connected wait until it is, and a rw_semaphore keeps them off the rings while a resume replaces them. Signed-off-by: Alex Zenla <alex@edera.dev>
A forked, restored or imported domain resumes against a new backend whose fid table is empty, while the guest's 9P clients still hold the fids of the session they were suspended with. With the transport reconnected every request on those fids still fails, so even an idle mount comes back unusable. When the backend advertises feature-session-transfer, wait at suspend for the requests in flight to be answered, then ask the backend for its fid table with Tsession_save and keep the pages in guest memory, where they survive into whatever domain the guest resumes in. Once a restored frontend's new backend connects, replay the pages with Tsession_restore before any request can use the rings. The pages are opaque to the frontend; the backend validates every entry as it would a walk or open. Signed-off-by: Alex Zenla <alex@edera.dev>
The toolstack has to know whether a guest can carry its 9P session into a new backend before it exports the domain, because a guest that cannot comes back with every open file on its mounts unusable. Until now only the backend advertised the feature, and the frontend's support was invisible from dom0. Publish feature-session-transfer = 1 in the frontend's own xenstore directory, next to version and num-rings. Signed-off-by: Alex Zenla <alex@edera.dev>
arch/x86/xen/enlighten.c includes xen/interface/memory.h for XENMEM_maximum_reservation, but memory.h uses xen_ulong_t, domid_t and GUEST_HANDLE without including where they are defined, xen/interface/xen.h. Other configurations reach those through an earlier include. The zone-tiny configuration does not, and fails to build with CONFIG_WERROR=y: include/xen/interface/physdev.h:41:5: error: unknown type name 'xen_ulong_t' Include xen/interface/xen.h first, as arch/arm/xen/enlighten.c does. Signed-off-by: Alex Zenla <alex@edera.dev>
A node whose pool is short called fill_list() even when other nodes still held free pages. fill_list() takes new address space from iomem_resource, which on a PVH dom0 can overlap MMIO that Xen has identity mapped. A later grant map onto such a gfn makes Xen refuse the p2m change and crash dom0. Carve a new region only when every node's pool is short, and otherwise take pages from the preferred node first, then the others. Return pages to the pool of their own node when the PV p2m allocation fails.
…ode variant Borrowing across nodes made xen_alloc_unpopulated_pages_node() best effort, so a caller that asked for a node could get pages of another. The node variant now takes pages only from the requested node's pool and fails with -ENOMEM when it is short. It no longer carves a region. xen_alloc_unpopulated_pages() has no node requirement: it prefers the local pool, borrows from other nodes, and carves a region only when every pool is short. xenbus falls back to it when the node's pool is short, and gnttab_alloc_pages_node() uses it for NUMA_NO_NODE.
GitHub starts no workflow for a review of a pull request it cannot compute a merge ref for, and a rebase often conflicts with the tree it replaces: approving #48 started nothing, and it had to be landed by hand. The land workflow can now also be run with a pull request number. It then reads the head, branch and base from the pull request, requires the same rebase/edera-mainline/... branch into edera/mainline, and lands on the most recent approval of the current head from a reviewer whose latest review is still an approval. Everything after that, the write-access check, the tip check, the backup and the leased push, is unchanged.
When a backport pull request merges into edera/6.18-lts, delete its branch and retry the mainline pull requests waiting for it. A backport/edera-6.18-lts/pr-<N> branch is the automation's own and is of no use once its pull request has merged, but delete_branch_on_merge stays off so feature branches survive, and the merged ones pile up. It is deleted only if it still points at the commit that merged. The retry was meant to happen already, from lts-backport-merged.yml on edera/6.18-lts, but that has never run: pull_request_target always takes the workflow from the default branch. This copy of that path lives on edera/mainline and runs edera/6.18-lts's retry-backports.sh.
edera/6.18-lts audits itself for Edera commits from edera/mainline that it lacks (kernel-nightly.yml with audit=true), but scheduled workflows run only from the default branch. Add a workflow here whose only job is to dispatch that audit every Monday.
The acceptance build judged x86_64 against defconfig with the Edera options forced on, which is not the kernel we ship. Merge the latest configs/x86_64/host.config from edera-dev/linux-kernel-oci over defconfig first, the way linux-kernel-oci itself applies it (the file is a fragment against x86_64_defconfig and omits SMP, NET and the rest of what matches it), and log the commit it came from. The Edera options are still forced on as built-ins afterwards: the host config builds OpenPaX out and 9P/Xen, virtio-gpu and the Hyper-V network drivers as modules, which a vmlinux build would never compile. AGP joins that list on x86_64 because host.config has it as a module, which caps DRM at m. arm64 is unchanged.
A rebase whose independent build failed went straight to a pull request, and in practice that was a run Claude never finished: the Bash tool's two-minute default timeout cannot hold a kernel build, so the build went to the background and the session ended "waiting for the x86 build" (#47, and #35 before it). Nobody saw the error until a reviewer did. Give the rebase step a 90-minute Bash timeout, and tell the skill to run builds in the foreground and to keep fixing and rebuilding until both pass rather than treat a failing build as an outcome to report. If the independent build still fails, a new read-only fix job hands Claude its branch and the build logs for one repair round, following a new "Repairing a failed build" section of the skill. A repaired branch is verified and built again from scratch before publish judges it; the checker now runs on whichever branch is final. Publish takes the repaired branch and report when there is one, says so in the pull request, and keeps the repair round's report when it repaired nothing. The build job moves to a reusable rebase-build.yml so the first build and the rebuild are the same job, and it now keeps each build's log as an artifact for the repair round to read. A repair changes downstream commits, so a repaired branch still shows drift and still goes to review rather than being pushed: what changes is that the reviewer gets a branch that builds.
The previous commit moved the build job into rebase-build.yml and calls it twice, but the file itself never made it into the commit: the kernel's .gitignore ignores dot-directories, so it has to be added with -f. Without it, upstream-rebase.yml calls a workflow that does not exist.
3 of 4 tasks
Author
|
Superseded by #66. |
edera-rebase-app
Bot
deleted the
rebase/edera-mainline/20261028-cde3385b5531-65ba8f34ec88
branch
October 29, 2026 05:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nightly rebase of
edera/mainline(cde3385b5531) ontomaster(65ba8f34ec88), driven by Claude. Not pushed automatically because the checker found drift. Run log.Important
Do not use the merge button: this branch replaces
edera/mainlinerather than merging into it. Approve the pull request to land it; the land workflow force-pushesedera/mainlineto the approved head, pinned tocde3385b5531, and keeps a backup.Independent checks
Builds: success (x86_64 vmlinux on the Edera host config and arm64 vmlinux, both with the Edera features built in; per-build results in the run).
cde3385b5531b96061a7c857864c07de0c7d65ba8f34ec88Commits whose hunk context moved (own +/- lines identical)
ea62ac9f1b0f iommu/xen: Add Xen PV-IOMMU driver→1d647e07cc78bd1c0dba57c3 sysctl: wire up PaX softmode setting→d754599a4025849ed61740ea use maximum supported mmap rnd entropy by default→2a8ce6240c51Drift (changed lines: upstream delta vs. actual change)
Claude's report
Summary
Rebased
edera/mainline(cde3385b5531) onto torvalds master65ba8f34ec88(v7.2-rc5-38657-g65ba8f34ec88), from merge base864c07de0c7d(v7.2-rc5-26646-g864c07de0c7d). That is 11366 upstream non-merge commits (a merge window). All 97 downstream commits were replayed ontorebase-result(b96061a7c857). There were 2 conflicts, both mechanical, and no commits were dropped.verify-upstream-rebase.shpasses base, linearity and commit-count, and says 3 commits changed only in hunk context. It flags Downstream drift, and all of that drift is thepax_tableblock inkernel/sysctl.cmoving to the new spot upstream gave the sysctl tables (see Conflicts). So the checker will report drift, and this branch will go to review as a PR. x86_64 and arm64 builds both pass.One thing about the run itself. At commit 46/97 (
init: add pax_softmode= kernel command-line option), git failed to fetch a blob from the promisor remote (upload-pack: not our ref), so that pick was recorded as done without being applied. Once the blob was available I applied the same commit with a plaingit cherry-pick 8c8f3ee28fcdand continued the rebase. The checker shows this commit replayed with identical content.Needs a decision
None.
Conflicts resolved
sysctl: wire up PaX softmode setting(bd1c0dba57c3→d754599a4025),kernel/sysctl.c. It collided with443bea2f71c5 sysctl: consolidate CONFIG_SYSCTL into a single block in kernel/sysctl.c(andacd19f2c56f2). Upstream movedsysctl_subsys_tableandsysctl_init_bases()into the single#ifdef CONFIG_SYSCTLblock and dropped the inner#ifdef CONFIG_SYSCTLaroundsysctl_writes_strict. I kept upstream's layout and put the downstreampax_tabledefinition (still underCONFIG_OPENPAX_SOFTMODE) right before the relocatedsysctl_subsys_table, with theregister_sysctl_init("kernel/pax", pax_table)call inside the relocatedsysctl_init_bases(). The commit's own+lines are byte-identical to before. The drift the checker reports is this block appearing at its new position.use maximum supported mmap rnd entropy by default(849ed61740ea→2a8ce6240c51),arch/Kconfig. It collided with8cdae75cebe0 arch/Kconfig: fix dead conditions by removing dead options. Upstream removedARCH_MMAP_RND_BITS_DEFAULT/ARCH_MMAP_RND_COMPAT_BITS_DEFAULTand theirdefault X if Xlines, which never took effect: no arch set them, and an int used as a condition is always false. I kept the removal and applied the downstream change ofdefault *_MINtodefault *_MAXfor both symbols. Behaviour is the same as before. In the built configs, x86_64 hasARCH_MMAP_RND_BITS=32(=MAX) andCOMPAT=16(=MAX). arm64 stays at 28 becausearch/arm64/Kconfigsets its own default first. That was already true before this rebase, and upstream did not change it.Dropped downstream commits
None.
Build fixes
None.
Upstream commits that meet the series
57 upstream commits touch files the series also touches. Where they matter:
443bea2f71c5,acd19f2c56f2: the sysctl consolidation. This is the conflict above.242bac52294e sysctl: Disallow partial updates for erroneous sysctl vectors,d08ab285b096,2bfbb09b505c: these change the internals of the proc_* handlers.pax_tableuses plainproc_dointvecon a single int, so its behaviour is unchanged.8cdae75cebe0: the arch/Kconfig conflict above. In the final tree, the removed symbols appear nowhere in arch/.6eecb67b69af xen/grant-table: stop setting PG_private on pages for grant mapping: downstreamgnttab_alloc_pages_node(fd6c9cba12d4) still callsgnttab_pages_set_private(), which upstream changed, and does not touch PG_private itself. A grep of the downstream diff finds noPagePrivate/SetPagePrivate/PG_privateanywhere. The fix is intact.1ae140d0f272 iommu: Make iommu-dma helpers private,fdd81f15fc43,1c8f6308e557 iommu/dma: Catch scatterlist length overflows,641a9b9cc697,95d2fec40c48: none of these overlap the one downstream hunk indma-iommu.c(skipping PCI window reservation under the Xen PV-IOMMU iniova_reserve_iommu_regions).xen-iommu.calready includesiommu-priv.h. Downstream code does not usedma_opt_mapping_sizeor any of the iommu-dma helpers that were made private. The sg-overflow checks are intact.2690d4156b51 iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest: this hooksx86_init.iommu.iommu_init = hv_iommu_initinsidehyperv_init()only. The downstream Xen-PV-dom0-nested-on-Hyper-V path runshyperv_init_nested_on_xen()instead, so the Hyper-V pvIOMMU is never installed there, and the Xen PV-IOMMU keeps the platform.HYPERV_PVIOMMUnow defaults to y alongsideXEN_IOMMUin the x86_64 build, and the build passes.118bdac10546 Drivers: hv: Add logical device ID registry for vPCI devices:hv_pci_probe()now registers each bus in an rhashtable thathv_common_init()sets up. The nested-on-Xen bring-up callshv_common_init(), so the table exists before any vPCI bus probes. The downstream pci-hyperv hunks (root IRQ domain, vector, multi-MSI anduse_callswhen nested on Xen) sit apart from the new register/unregister calls.d0ec95a8a4e7 hv_netvsc: Advertise the SR-IOV capability for CoCo VMs: this is innegotiate_nvsp_ver(). The downstream netvsc change is innetvsc_dma_map/unmap, so they do not interact.51e766bcd35e,b82ed0e3a1da(init/main argv/envp fixes),aabe6e1b36ce,958c01049e0a,677f83795fdf,084100a1b579: none of these overlap the downstreampax_softmodedefinition and__setup.pax_softmode=is a__setup, not an early param, so the bootconfig change does not affect it.10823ced2f18,8ebd2eb87f50(KHO rename),5728d3523f12,681f3b68f435,537ec2b15fdd,28d5df7b7274touchkernel-parameters.txt. The downstream doc commit only addspax_softmode=.68cb098eadda,7135a05838e8,dc0b17144d62(fixes),72a993449cb0(VM_SPECIAL removed),c911e7ebebf5,94c86564538e,9cae38dc11da,47e725702b0b,38564710f443,e59c9a971a9f,f85eb0656c40, and the rest of the VMA series. None of the downstream OpenPaX code uses removed or renamed symbols: a grep of the downstream diff finds noVM_SPECIAL,is_vm_hugetlb_pageorhugetlb_inline.map_deny_write_exec(), which carries the OpenPaX MPROTECT check, is still called frommprotect_fixupand first thing inmmap_region(). Themm_struct.pax_flagsfield and the brk randomization hunks apply unchanged. I could not buildtools/testing/vmahere because the runner has no liburcu (urcu.h), so the downstream stubs in that harness were not compile-tested against upstream's reworkeddup.h/stubs.h.bbe5d3b3d402,35e340a47888,69d38c792c53,abaa0835b4f3,264d8fd2794f,2e83e2ce34ff,c2d4ccfbba3f,0bf031ba8c49,42eed9bb9ca4: these only meet the series in shared files (drivers/iommu/Kconfig/Makefile,mm/util.c,init/main.c,arch/Kconfig), and none of them overlap the downstream hunks.Builds