This is a post summarizing my effort to enable huge pages in Linux. I want to use huge page to test out potential performance improvement for graph applications such as PageRank and BFS. (The post is still getting updated)
It is possible that orthogonal to the post below, that the operating system has enabled Transparent Huge Pages. In that case, even if meminfo showed that page size is 4kb, the application could still be using 2MB huge pages. Be VERY CAREFUL about that. To check whether transparent huge pages (TPH) is enabled or not, use the following command
Huge Page is a feature supported by many Linux Kernels. Typically, we have 4 kb page sizes, but we can potentially use 2 MB page size and even up to 1GB (requires special hardware support for 1GB). Using huge page can reduce TLB misses and therefore save computation time spent in computing the physical memory address.
A good place to start is the Wikipedia page
This is potentially useful for graph applications because they incur large amount of random memory access on different pages. For a 1 million nodes graph, the rank and reciprocal data can take as much as 16 MB (two doubles one for rank, the other for reciprocal). This data can incur a lot of miss on 4 kb page size.
I first tried to do it in mac OS X, but there doesn’t seem to have a support for that. I googled through the web and there is not much information on how to do this explicitly. The post below doesn’t seem to think
The posts mentioned in there provide a good background on Huge Pages (the five parts in the article are well written and highly recommended)
The most read post seems to be from linux documentation here
Following this post, you can first do
to see if any huge pages are allocated. If not, we need some privileges. The safest way seems to be