summaryrefslogtreecommitdiff
path: root/include
diff options
context:
space:
mode:
authorMatthew Wilcox (Oracle) <willy@infradead.org>2020-08-14 17:30:23 -0700
committerLinus Torvalds <torvalds@linux-foundation.org>2020-08-14 19:56:56 -0700
commit1378a5ee451a5e87d0d8dd6356a0b7844db231f6 (patch)
tree557c87581ebd5347545d5bf86c5b6711f846b341 /include
parent14a36a435343931296277f4a3c5adf4e29a8f46d (diff)
mm: store compound_nr as well as compound_order
Patch series "THP prep patches". These are some generic cleanups and improvements, which I would like merged into mmotm soon. The first one should be a performance improvement for all users of compound pages, and the others are aimed at getting code to compile away when CONFIG_TRANSPARENT_HUGEPAGE is disabled (ie small systems). Also better documented / less confusing than the current prefix mixture of compound, hpage and thp. This patch (of 7): This removes a few instructions from functions which need to know how many pages are in a compound page. The storage used is either page->mapping on 64-bit or page->index on 32-bit. Both of these are fine to overlay on tail pages. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: William Kucharski <william.kucharski@oracle.com> Reviewed-by: Zi Yan <ziy@nvidia.com> Cc: David Hildenbrand <david@redhat.com> Cc: Mike Kravetz <mike.kravetz@oracle.com> Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Link: http://lkml.kernel.org/r/20200629151959.15779-1-willy@infradead.org Link: http://lkml.kernel.org/r/20200629151959.15779-2-willy@infradead.org Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'include')
-rw-r--r--include/linux/mm.h5
-rw-r--r--include/linux/mm_types.h1
2 files changed, 5 insertions, 1 deletions
diff --git a/include/linux/mm.h b/include/linux/mm.h
index e7602a3bcef1..10b1e6e5f885 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -922,12 +922,15 @@ static inline int compound_pincount(struct page *page)
static inline void set_compound_order(struct page *page, unsigned int order)
{
page[1].compound_order = order;
+ page[1].compound_nr = 1U << order;
}
/* Returns the number of pages in this potentially compound page. */
static inline unsigned long compound_nr(struct page *page)
{
- return 1UL << compound_order(page);
+ if (!PageHead(page))
+ return 1;
+ return page[1].compound_nr;
}
/* Returns the number of bytes in this potentially compound page. */
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 0277fbab7c93..496c3ff97cce 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -134,6 +134,7 @@ struct page {
unsigned char compound_dtor;
unsigned char compound_order;
atomic_t compound_mapcount;
+ unsigned int compound_nr; /* 1 << compound_order */
};
struct { /* Second tail page of compound page */
unsigned long _compound_pad_1; /* compound_head */