Ask Your Question

How does openstack controller know if compute node is down

asked 2016-07-26 15:40:26 -0500

Sen Wang gravatar image

I am working on a openstack auto heal function. We can know if a compute node is off from command-line interface by using command like nova service-list / nova hypervisor-list or through equivalent REST Call, but how does controller know it from the first place.

Controller and compute communicate with each other from RabbitMQ. Is there any 'keep alive' messages being passing through in RabbitMQ. Or when compute node is down, will any messages be published in RabbitMQ.

edit retag flag offensive close merge delete

1 answer

Sort by » oldest newest most voted

answered 2016-08-25 08:58:36 -0500

sunnyarora gravatar image

Below is detailed answer for your query:

There are two important components here : One is Scheduler Service and another is messaging queue service(for ex: qpid, RabbitMQ). These two aspects keep controller fully updated on whole openstack environment.

OpenStack Controller saves all the configurations in database which is by default mysql, so Controller knows all the information. A Nova component named nova-scheduler running as a controller service will decide where to place VM among all available hosts.

nova-scheduler interacts with other components through queue and central database repo. For scheduling, queue is the essential communications hub. All compute nodes (also known as hosts in terms of OpenStack) periodically publish their status, resources available and hardware capabilities to nova-scheduler through the queue. nova-scheduler then collects this data and uses it to make decisions when a request comes in.

image description

Above picture shows us the general idea of how the scheduler does its main job. The whole process divides into two phases. Filtering phase will generate a list of suitable hosts by applying filters. So Automatically the Host which are Down (not running openstack services) will be automatically filtered out. Weighting phase will sort the hosts according to their weighted cost scores, which are given by applying some cost functions. The sorted list of hosts is candidates to fulfill the user's request. How many hosts in this list will be used depends on the number of instances requested in one request.

So By nova-scheduler does below tasks

user requests are processed sequentially (FIFO scheduling)

  • nova-scheduler doesn't provide any dynamic priority strategy algorithm

requests dispatched by the AMQP based message broker

  • RabbitMQ / Apache Qpid™

  • AMQP (Advanced Message Queueing Protocol) is an open internet protocol for reliably sending and receiving messages. It makes it possible for everyone to build a diverse, coherent messaging ecosystem Request not satisfied (e.g. resource not available) fails and will be lost


Refer here for below info

AMQP is the messaging technology chosen by the OpenStack cloud. The AMQP broker, either RabbitMQ or Qpid, sits between any two Nova components and allows them to communicate in a loosely coupled fashion. More precisely, Nova components (the compute fabric of OpenStack) use Remote Procedure Calls (RPC hereinafter) to communicate to one another; however such a paradigm is built atop the publish/subscribe paradigm so that the following benefits can be achieved:

Decoupling between client and servant (such as the client does not need to know where the servant reference is).

Full a-synchronism between client and servant (such as the client does not need the servant to run at the same time of the remote call).

Random balancing of remote calls (such as if more servants are up and running, one-way calls are transparently dispatched to the first available servant). Nova uses direct, fanout, and topic-based exchanges.

Yes the User request message gets published at qpid/rabbitmq.

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools



Asked: 2016-07-26 15:40:26 -0500

Seen: 1,910 times

Last updated: Aug 25 '16