Enable Huge Page in Linux

This is a post summarizing my effort to enable huge pages in Linux. I want to use huge page to test out potential performance improvement for graph applications such as PageRank and BFS. (The post is still getting updated)

UPDATE:

It is possible that orthogonal to the post below, that the operating system has enabled Transparent Huge Pages. In that case, even if meminfo showed that page size is 4kb, the application could still be using 2MB huge pages. Be VERY CAREFUL about that. To check whether transparent huge pages (TPH) is enabled or not, use the following command

cat /sys/kernel/mm/transparent_hugepage/enabled

Huge Page is a feature supported by many Linux Kernels. Typically, we have 4 kb page sizes, but we can potentially use 2 MB page size and even up to 1GB (requires special hardware support for 1GB).  Using huge page can reduce TLB misses and therefore save computation time spent in computing the physical memory address.

A good place to start is the Wikipedia page

https://en.wikipedia.org/wiki/Page_(computer_memory)#Huge_pages

This is potentially useful for graph applications because they incur large amount of random memory access on different pages. For a 1 million nodes graph, the rank and reciprocal data can take as much as 16 MB (two doubles one for rank, the other for reciprocal). This data can incur a lot of miss on 4 kb page size.

I first tried to do it in mac OS X, but there doesn’t seem to have a support for that. I googled through the web and there is not much information on how to do this explicitly.  The post below doesn’t seem to think

http://stackoverflow.com/questions/28038151/osx-set-page-size 

The posts mentioned in there provide a good background on Huge Pages (the five parts in the article are well written and highly recommended)

http://lwn.net/Articles/374424/ 

The most read post seems to be from linux documentation here

https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt

Following this post, you can first do

cat /proc/meminfo

to see if any huge pages are allocated. If not, we need some privileges. The safest way seems to be

Advertisements
This entry was posted in Uncategorized and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s