masakari notification on process failure

asked 2019-04-25 06:42:15 -0500

sanjay khuntia gravatar image

updated 2019-04-25 06:45:43 -0500

Hi all, I have setup a multi host set up using devstack stein with 1 controller node and 2 compute nodes with ubuntu 16 VMs. I have enabled Masakari service on my controller local.conf file like below -

#Enable Masakari
enable_plugin masakari ${GIT_BASE}/openstack/masakari $USE_BRANCH
#enable_plugin masakari-dashboard ${GIT_BASE}/openstack/masakari-dashboard $USE_BRANCH
enable_plugin masakari-monitors ${GIT_BASE}/openstack/masakari-monitors $USE_BRANCH
enable_plugin python-masakariclient ${GIT_BASE}/openstack/python-masakariclient $USE_BRANCH

I have set up Masakari Segments adding 2 compute nodes in it with AUTO recovery under the 'Instance-ha' section in horizon dashboard. I have to mimic process, VM and compute failure to check whether Masakari provides proper notification in each failure case.

To test process failure, I have created 2 instances in one compute. From root user, I killed the instance 1's qemu process on that particular host. However masakari did not provide notification for this process failure in horizon instance-ha notifications if it is monitoring the underlying processes on controller and computes.

Please let me know if I am missing anything in configuring Masakari and how Masakari notifications really works to provide useful information to the end user.

thank you for your pointers and replies.

best regards,

edit retag flag offensive close merge delete

Comments

In case of masakari-processmonitor, it only monitors processes as mentioned in the process_list.yaml which by default monitors libvirt-bin, nova-compute, instancemonitor, hostmonitor and sshd processes.

To test process failure, you should terminate any of the above processes.

tvpatil gravatar imagetvpatil ( 2019-04-29 16:53:03 -0500 )edit

masakari-processmonitor will attempt to restart the terminated process. If it fails to restart the process, then only it will send the notification to masakari.

tvpatil gravatar imagetvpatil ( 2019-04-29 17:03:24 -0500 )edit