Setting Huge Pages in Linux

If your Linux based database server has huge amount of physical memory, it will be a good idea to enable huge pages so that memory could be used efficiently by Oracle instance. Setting huge pages means that you have bigger sized (2 MB) memory blocks in memory to allocate to Oracle SGA. Bigger memory block size would mean fewer number of total memory blocks, and this is where managing memory becomes efficient by the OS. I would recommend to use/enable huge pages if host’s physical memory size goes beyond 128G. But you can also set huge pages even for a small amount of memory.

Huge pages for Oracle SGA can be used only if SGA_TARGET and SGA_MAX_SIZE parameters are in use. MEMORY_TARGET and MEMORY_MAX_TARGET do not support using huge pages. Therefore, make sure that SGA size is set using SGA_TARGET and SGA_MAX_SIZE.

In this example, I will explain how to set huge pages for total SGA size of 24G (for all instances running on this host).

For huge pages, each memory page size is 2 MB (2048 K) which we can see using follow command.

#cat /proc/meminfo |grep Huge HugePages_Total: 200 HugePages_Free: 200 HugePages_Rsvd: 120 HugePages_Surp: 0 Hugepagesize: 2048 kB

For 24G SGA_MAX_SIZE, first we need to convert SGA size to into MB and then divide it by 2. The resulting value should be the value of huge pages that we should set. (24*1024)/2 => 12288

To calculate this value, you can also use a script available in MOS doc 401749.1. I would suggest increasing the resulting value a little more. For example, I would make this calculated value of 12288 to 12300 which is 12 pages (24 MB) more than actual calculated value.

To set huge pages, add/edit values in /etc/sysctl.conf and also in /etc/security/limits.conf (or in a file under /etc/security/limits.d directory which you are using to set limits)

In /etc/sysctl.conf file and add following entry.

vm.nr_hugepages=12300

You can execute following command to have new value take effect. System reboot after all setting is needed anyway which will have new value take effect.

#sysctl –p

Add/edit following in /etc/security/limits.conf (or in a file under /etc/security/limits.d directory which you are using to set limits). The value in this file is specified in KB. Therefore, for this example of 12300 huge pages, the value should be 12300*2048=25190400. This value can also be set to even a higher value (just less than actual amount of physical RAM). This value is just to specify how much maximum memory segment can be used by “oracle” user. 

oracle soft memlock 25190400 oracle soft memlock 25190400

If SGA parameters have not been set yet, set them now and completely remove meory_target and memory_max_target from the init file.

Make sure that parameter use_large_pages is also set to TRUE. The default value is always TRUE

SQL> show parameter use_large_pages NAME                                 TYPE        VALUE ------------------------------------ ----------- ------------------------------ use_large_pages                      string      TRUE

Reboot the server.

After reboot of the system and startup of database, you can see following lines in the alert log file of the database that confirms that huge pages are in use now.

Starting ORACLE instance (normal) ************************ Large Pages Information ******************* Per process system memlock (soft) limit = 24 GB Total Shared Global Region in Large Pages = 12 GB (100%) Large Pages used by this instance: 6145 (12 GB) Large Pages unused system wide = 2046 (4092 MB) Large Pages configured system wide = 12300 (24 GB) Large Page size = 2048 KB ********************************************************************