Friday, June 28, 2019

Poor Database Performance After Setting Huge Pages

I recently faced a problem whereby full memory of system got occupied after the database startup and eventually system was hung and reboot was inevitable. The recent change was to setting huge pages on the host. Eventually we found the reason that “soft memlock” was not set (oracle software owner name was wrongly spelled when specifying memlock in /etc/security/limits.conf file). Bu default “soft memlock” was set to a default value 64KB and none of huge pages was in use by the Oracle (SGA). As a result, SGA was allocated from the remaining memory (Huge pages would remain unused, yet allocated) and huge pages were merely consuming memory without any use. Following message in alert log let us understand the problem

Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Per process system memlock (soft) limit = 64 KB

Total Shared Global Region in Large Pages = 0 KB (0%)

Large Pages used by this instance: 0 (0 KB)
Large Pages unused system wide = 2046 (4092 MB)
Large Pages configured system wide = 12300 (24 GB)
Large Page size = 2048 KB

Total System Global Area size is 24 GB. For optimal performance, prior to the next instance restart:
1. Increase the number of unused large pages by at least 1 (page size 2048 KB, total size 2048 KB) system wide to get 100% of the System global Area allocated with large pages.
2. Large pages are automatically locked into physical memory. Increase the per process memlock (soft) limit to at least 24 GB to lock 100% system Global Area’s large pages into the physical memory

I found following article to investigate the usage of huge pages.
Using ipcs –m command we can find out how much is being used by whom. As can be seen in the following output form one of my hosts, 5th column shows the amount of huge pages in bytes used by different user processes.

ipcs -m|awk '{ print $5}'|awk '{a+=$0}END{print a}'
To solve this problem, we need to make sure that memlock setting are properly done as explained in this article.

No comments:

Post a Comment

Popular Posts - All Times