libvirt createWithFlags API is slowly sometimes in openstack juno

asked 2015-06-15 03:09:13 -0600

tianmaofu gravatar image

updated 2015-06-15 03:41:35 -0600

When i boot a instance in openstack juno(libvirt backend is kvm, storage backend is ceph). it takes a very long time, so i add log to see which function takes so long. and i found in the function _create_domain(self, xml=None, domain=None, instance=None, launch_flags=0, power_on=True) in libvirt.driver.py takes 30 seconds more, and in this function, the libvirt createWithFlags API takes most of the time.

this is the source code of function _create_domain in juno:

def _create_domain(self, xml=None, domain=None,
                   instance=None, launch_flags=0, power_on=True):
    """Create a domain.

    Either domain or xml must be passed in. If both are passed, then
    the domain definition is overwritten from the xml.
    """
    err = None
    try:
        if xml:
            err = _LE('Error defining a domain with XML: %s') % xml
            domain = self._conn.defineXML(xml)

        if power_on:
            err = _LE('Error launching a defined domain with XML: %s') \
                      % domain.XMLDesc(0)
            domain.createWithFlags(launch_flags)

        if not utils.is_neutron():
            err = _LE('Error enabling hairpin mode with XML: %s') \
                      % domain.XMLDesc(0)
            self._enable_hairpin(domain.XMLDesc(0))
    except Exception:
        with excutils.save_and_reraise_exception():
            if err:
                LOG.error(err)

    return domain

there is my one test log below(i just simplely make it log error):

2015-06-15 14:34:41.126 30743 ERROR nova.virt.libvirt.driver [req-1a0670ec-0958-43c9-b2d5-2a613f34ae55 None] =============test time of createWithFlags
2015-06-15 14:35:10.429 30743 ERROR nova.virt.libvirt.driver [req-1a0670ec-0958-43c9-b2d5-2a613f34ae55 None] =============test time of createWithFlags.end

and the libvirt instance log in /var/log/libvirt/qemu/instance-**.log( i delete some middle section to let you see the log start time and end time):

2015-06-15 06:34:43.819+0000: starting up
...
...
2015-06-15 06:34:43.826+0000: 6133: debug : virCommandHandshakeChild:371 : Waiting on parent for handshake complete on 75
2015-06-15 06:34:43.886+0000: 6133: debug : virFileClose:90 : Closed fd 74
2015-06-15 06:34:43.886+0000: 6133: debug : virFileClose:90 : Closed fd 75
2015-06-15 06:34:43.886+0000: 6133: debug : virCommandHandshakeChild:391 : Handshake with parent is done
Warning: option deprecated, use lost_tick_policy property of kvm-pit instead.
char device redirected to /dev/pts/31 (label charserial1)

From the log, it seems the libvirt deal the domain creating in less then 1 seconds, but in openstack, it return after almost 30 seconds, why and what's wrong with it??

I do this test for many times, and in sometimes libvirt createWithFlags API return in less than 1 seconds, but most of the time the API takes 20-40 seconds.

Is the someone can explain it and make the API more faster and stable??

if you want some more information, please tell me

edit retag flag offensive close merge delete

Comments

I have the same issue, but I'm not sure if this commit for Liberty is going to hide the problem

https://github.com/openstack/nova/com...

electrocucaracha gravatar imageelectrocucaracha ( 2015-09-01 17:47:18 -0600 )edit