Ask Your Question

Revision history [back]

Yes, this is almost certainly storage-related - but what's not clear is whether your VM kept running after seeing this error or not. Are there any errors on the host kernel log? What kind of backing storage are you using - is it just local disks or is something else connected?

If your VM does actually continue running, you could just be filling up the cache faster than it can be cleared. By default Linux uses up to 40% of the available memory for file system caching. After this mark has been reached the file system flushes all outstanding data to disk causing all following IOs going synchronous. For flushing out this data to disk this there is a time limit of 120 seconds by default. In the case here the IO subsystem is not fast enough to flush the data withing 120 seconds. As IO subsystem responds slowly and more requests are served, System Memory gets filled up resulting in the above error, thus serving HTTP requests.

You can check this by changing vm.dirty_ratio and vm.dirty_backgroud_ratio

someuser@servercore [/home/someuser]$ sudo sysctl -w vm.dirty_ratio=10
someuser@servercore [/home/someuser]$ sudo sysctl -w vm.dirty_background_ratio=5
someuser@servercore [/home/someuser]# sudo sysctl -p