4th
2012
Something is deeply broken in OS X memory management (Lion performance problems part 3)
(Mountain Lion update: possibly fixed.)
I’ve been doing some investigation with Perry Metzger into what appears to be a huge problem with memory management in Lion (I, but not everybody, have seen this going back to Snow Leopard, though it’s worse in Lion. This may have something to do with the fact that I was running the 64-bit kernel on SL and most other people weren’t.)
The main symptoms that we’ve encountered:
- Frequent beachballs, particularly when switching applications and sometimes even tabs.
- General overall slowness and poor UI responsivness.
- Specific and drastic slowdowns on every Time Machine run.
- High memory utilization in Safari Web Content, mds, and kernel_task processes.
- Large numbers of page outs even with a good deal of available RAM.
- High amounts of RAM marked as inactive which is not readily freed back to other applications, with page outs favored.
The core issue seems to be that the virtual memory manager is bad at managing which pages should be freed from the inactive state and which ones should be paged out to disk (and, consequently, back from disk). There seem to be at least two distinct problems here, though it’s difficult to tell for sure without proper instrumentation - 1) program data is not well prioritized to remain in physical RAM over buffer data, and 2) the garbage collection algorithm may require that all of a program’s data be in physical RAM before collection can happen, causing extensive paging.
High disk activity from time machine and spotlight (mds) seems to cause a lot of pages of program memory to get thrashed out to disk to make room for the disk buffer pages. This manifests itself as basically every process waiting for the disk, an incredible performance drop across the board, frequent beach balls, very high amounts of inactive memory which doesn’t seem to get freed, and high numbers of page outs even with memory available. I was able to get some temporary benefit out of running the purge command to blow away the disk cache (see part 1 and part 2), but the problem returned after not much time. I’ve seen several blog posts recommending disabling the dynamic pager in Snow Leopard, but thought that was a fairly drastic step until Perry recommended it.
The results here are dramatic - after rebooting, both of my machines (mac pro with 32GB and macbook pro with 8GB) are running viscerally and noticeably faster. Many operations which used to have a delay became instantaneous - task switching, viewing thumbnails in Lightroom, switching windows/tabs in Safari. I didn’t realize until it went away just how much I’d been waiting for the most mundane things. Even under very high load, the entire system is very responsive now.
The only downside I can see is that things would likely start crashing if physical RAM actually runs out, but I’ve noticed that the footprint of all of my apps seems to be much lower. Just keep an eye on RAM (it helps to run Activity Monitor and make the dock icon show free memory - you can configure that by right clicking on the dock icon).
Ostensibly, this is “risky”, but so far the effects only seem beneficial. Likely the worst thing that will happen is that the OS will kernel panic if it completely runs out of memory, though it may not even be that bad - there seems to be some speculation that the system may turn on the dynamic pager automatically if system memory gets too low; I haven’t encountered that, though I have noticed that if I top out physical RAM, the page out count does go up. I have no immediate explanation for that.
I can’t recommend this as a long term solution - Apple should fix this - but in the meantime, it seems to be having great results for me. Here’s how much of a difference it is: I was going to turn it back on to collect some comparative numbers for this post, but I couldn’t bear to cripple my machine again.
To deactivate the dynamic pager:
sudo launchctl unload -w /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist
followed by a reboot. You can turn it on again with
sudo launchctl load -wF /System/Library/LaunchDaemons/com.apple.dynamic_pager.plist
Some references: