AMQP not returning nova notifications
I'm trying to fetch Nova notifications from OpenStack for a monitoring application. When I try launch and instance/rename/delete, I only get the port.create/update/delete notifications. The environment is RDO Mitaka. In another environment, with Mirantis and Liberty, I do get reasonable notifications. What am I doing wrong here?
My Python code follows:
from kombu.mixins import ConsumerMixin
from kombu.log import get_logger
from kombu import Queue, Exchange
class EventHandler:
def instance_add(self, notification): print("instance_add")
def instance_delete(self, notification): print("instance_delete")
def instance_update(self, notification): pass
def instance_down(self, notification): pass
def instance_up(self, notification): pass
def region_add(self, notification): pass
def region_delete(self, notification): pass
def region_update(self, notification): pass
logger = get_logger(__name__)
class Worker(ConsumerMixin):
event_queues = [
Queue('nova',
Exchange('nova', 'topic', durable=False),
durable=False, routing_key='#')
]
def __init__(self, connection):
self.connection = connection
self.handler = EventHandler()
self.notification_responses = {
"compute.instance.create.end": self.handler.instance_add,
"compute.instance.delete.end": self.handler.instance_delete,
"compute.instance.rebuild.end": self.handler.instance_update,
"compute.instance.update": self.handler.instance_update,
"servergroup.create": self.handler.region_add,
"servergroup.delete": self.handler.region_delete,
"servergroup.update": self.handler.region_update,
"servergroup.addmember": self.handler.region_update,
"compute.instance.shutdown.start": self.handler.instance_down,
"compute.instance.power_off.start": self.handler.instance_down,
"compute.instance.power_on.end": self.handler.instance_up,
"compute.instance.suspend.start": self.handler.instance_down,
"compute.instance.suspend.end": self.handler.instance_up
}
def get_consumers(self, Consumer, channel):
return [Consumer(queues=self.event_queues,
accept=['json'],
callbacks=[self.process_task])]
def process_task(self, body, message):
if "event_type" in body:
self.handle_event(body["event_type"], body)
message.ack()
def handle_event(self, type, notification):
if type not in self.notification_responses:
return ""
return self.notification_responses[type](notification)
if __name__ == '__main__':
from kombu import Connection
from kombu.utils.debug import setup_logging
# setup root logger
setup_logging(loglevel='DEBUG', loggers=[''])
with Connection('amqp://guest:guest@10.56.20.81:5672//') as conn:
try:
print(conn)
worker = Worker(conn)
worker.run()
except KeyboardInterrupt:
print('Stopped')
Are notifications enabled for Nova? If you're using Neutron then the port updates are probably from Neutron itself, but may be placed on the queue to alert nova. See if the notification_driver has been configured for Nova.
In /etc/nova/nova.conf I had notify_on_state_change=<none> In Mirantis Liberty env. I had notify_on_state_change=vm_state
Changed this to match in the Devstack Mitaka env. but still no good :-( Left it for now. Will try again after I have Mirantis Mitaka in place. Will report back then.
Check the notification_driver in the RDO environment and compare it to the others. Also see if there are messages getting stuck in the queue:
rabbitmqctl list_queues name consumers messages|grep -i notification
on another note, do want services NOT to consume so your script can pull data?
RDO: notification_driver=nova.openstack.common.notifier.rabbit_notifier,ceilometer.compute.nova_notifier Mirantis-Liberty, which works, doesn't have this setting at all.
I'm not sure what you mean about services not consuming. Do you mean that if my script consumes other services can't?
Yes, this is also possible. If the exchange U r connecting is not fanout than rabbit will dispatch messages in round robin fashion between your consumer and default one.