Ask Your Question
1

Failed to start libvirt during kolla ocata AIO installation

asked 2017-03-24 06:16:16 -0600

9lives gravatar image

updated 2017-03-28 02:32:57 -0600

Dear Stackers,

We have tried to install AIO kolla ocata version. Everything goes well until start the nova-libvirt container. could you help to figure out what is the root cause and show the solution thanks a million!

Env:

  • Docker host: Linux openstack-compute 3.13.0-62-generic #102-Ubuntu SMP Tue Aug 11 14:29:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
  • docker version: 1.13

  • kolla version: ocata stable

  • openstack_release_version:4.0.1

  • kolla_base_distro:centos

The error message from the fluentd container log is as following:

2017-03-24 10:55:34.829+0000: 2176: warning : networkStateInitialize:692 : DBus not available, disabling firewalld support in bridge_network_driver: internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied
2017-03-24 10:55:35.017+0000: 2176: error : virDBusGetSystemBus:109 : internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied
2017-03-24 10:55:35.017+0000: 2176: error : virStateInitialize:775 : Initialization of NWFilter state driver failed: internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied
2017-03-24 10:55:35.017+0000: 2176: error : daemonRunStateInit:962 : Driver state initialization failed
2017-03-24 10:59:01.526+0000: 3072: info : libvirt version: 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <http://bugs.centos.org>, 2017-03-03-02:09:45, c1bm.rdu2.centos.org)
2017-03-24 10:59:01.526+0000: 3072: info : hostname: openstack-compute
2017-03-24 10:59:01.526+0000: 3072: error : virDBusGetSystemBus:109 : internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied
2017-03-24 10:59:01.526+0000: 3072: warning : networkStateInitialize:692 : DBus not available, disabling firewalld support in bridge_network_driver: internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied
2017-03-24 10:59:01.681+0000: 3072: error : virDBusGetSystemBus:109 : internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied
2017-03-24 10:59:01.681+0000: 3072: error : virStateInitialize:775 : Initialization of NWFilter state driver failed: internal error: Unable to get DBus system bus connection: Failed to connect to socket /var/run/dbus/system_bus_socket: Permission denied

The ansible log snippets is

...
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/core/commands/command.py
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260 `" && echo ansible-tmp-1490352085.17-274695573590260="` echo $HOME/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260 `" ) && sleep 0'
<localhost> PUT /tmp/tmpdVDdlq TO /root/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260/command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260/ /root/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260/command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260/command.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1490352085.17-274695573590260/" > /dev/null 2>&1 && sleep 0'
fatal: [localhost -> localhost]: FAILED! => {
    "attempts": 20,
    "changed": false,
    "cmd": [
        "docker",
        "exec",
        "kolla_toolbox",
        "openstack",
        "--os-interface",
        "internal ...
(more)
edit retag flag offensive close merge delete

Comments

My first guess is the issue is from running centos containers in ubuntu host. While containers are made to be distributed, some services as libvirt have to connect and have privileges on the bare host. Probably the issue is that libvirt cannot create a connection with the host.

Eduardo Gonzalez gravatar imageEduardo Gonzalez ( 2017-03-26 17:52:54 -0600 )edit

my recommendation is to use whether centos or ubuntu for both containers and host.

Eduardo Gonzalez gravatar imageEduardo Gonzalez ( 2017-03-26 17:55:53 -0600 )edit
1

Thanks for the comments. Do you mean in kolla that we need docker host OS distro match the container OS distro?Where should the kvm vm be hosted, on bare host or in the container? Thanks!

Vic

9lives gravatar image9lives ( 2017-03-26 22:27:16 -0600 )edit
1

per ur recommendation,i have switched to use ubuntu 14.04 as docker host and ubuntu image for kolla, however, the nova-libvirt container failed to start for segment fault issue.

9lives gravatar image9lives ( 2017-03-26 22:38:19 -0600 )edit
1

I'd suggest filing a bug then. As far as I understand, this should be working.

rabel-b1 gravatar imagerabel-b1 ( 2017-03-27 04:54:55 -0600 )edit

1 answer

Sort by » oldest newest most voted
3

answered 2017-03-28 03:26:32 -0600

9lives gravatar image

After some debugging, we found the root cause is the apparmor is on that prevents the nova_libvirt connect to dbus.

Here is what we found in /var/log/syslog

Mar 28 01:56:36 openstack-compute kernel: [7959635.540048] type=1400 audit(1490680596.900:12553): apparmor="DENIED" operation="connect" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/libvirtd" name="run/dbus/system_bus_socket" pid=42154 comm="libvirtd" requested_mask="rw" denied_mask="rw" fsuid=0 ouid=0

Solution: Just remove the apparmor follow the following method:

sudo /etc/init.d/apparmor stop
sudo update-rc.d -f apparmor remove
sudo apt-get --purge remove apparmor apparmor-utils libapparmor-perl libapparmor1

Hope that helps!

Vic

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2017-03-24 06:16:16 -0600

Seen: 557 times

Last updated: Mar 28 '17