Ask Your Question

KVM over KVM- Openstack

asked 2015-06-04 01:09:19 -0500

cloudQuestions gravatar image

updated 2015-06-04 05:02:53 -0500

I hope this is a simple question.

I've installed OpenStack following the official guide for Juno. All the 6 nodes(Controller, compute, network object storage 1/2 and bloq storage) are Virtual Machines of the same server(its a quite powerfull server) virtualized with KVM.

Now, i want to use KVM on the compute node for the instances of openstack, is this possible?

I'm asking this because instances wont boot using KVM, but they will if I use QEMU.


Aditional info:

Compute node(L1):

lsb_release -a (all the nodes)
    Distributor ID: Ubuntu
    Description:    Ubuntu 14.04.2 LTS
    Release:        14.04
    Codename:       trusty

lsmod | grep kvm_intel
kvm_intel             143590  0
kvm                   452088  1 kvm_intel

$ file /dev/kvm      
/dev/kvm: character special

$ cat /sys/module/kvm_intel/parameters/nested

Host server(L0):

lsb_release -a

Distributor ID: Ubuntu
Description:    Ubuntu 14.04.2 LTS
Release:        14.04
Codename:       trusty

 $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs
$ cat /sys/module/kvm_intel/parameters/enable_apicv
 $ cat /sys/module/kvm_intel/parameters/ept

$ cat /sys/module/kvm_intel/parameters/nested

This is what happens when i launch an instance with virt_type= KVM image description

Virsh capabilities of the compute node:


      <topology sockets='12' cores='1' threads='1'/>
      <feature name='hypervisor'/>
      <feature name='osxsave'/>
      <feature name='vmx'/>
      <cells num='1'>
        <cell id='0'>
          <memory unit='KiB'>30883820</memory>
          <cpus num='12'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
            <cpu id='1' socket_id='1' core_id='0' siblings='1'/>
            <cpu id='2' socket_id='2' core_id='0' siblings='2'/>
            <cpu id='3' socket_id='3' core_id='0' siblings='3'/>
            <cpu id='4' socket_id='4' core_id='0' siblings='4'/>
            <cpu id='5' socket_id='5' core_id='0' siblings='5'/>
            <cpu id='6' socket_id='6' core_id='0' siblings='6'/>
            <cpu id='7' socket_id='7' core_id='0' siblings='7'/>
            <cpu id='8' socket_id='8' core_id='0' siblings='8'/>
            <cpu id='9' socket_id='9' core_id='0' siblings='9'/>
            <cpu id='10' socket_id='10' core_id='0' siblings='10'/>
            <cpu id='11' socket_id='11' core_id='0' siblings='11'/>
      <baselabel type='kvm'>+108:+113</baselabel>
      <baselabel type='qemu'>+108:+113</baselabel>

    <arch name='alpha'>
      <machine maxCpus='4'>clipper</machine>
      <domain type='qemu'>

    <arch name='armv7l'>
      <machine maxCpus='1'>borzoi</machine>
      <machine maxCpus='4'>virt</machine>
      <machine maxCpus='4'>midway</machine>
      <machine maxCpus='1'>tosa</machine>
      <machine maxCpus='1'>cheetah</machine>
      <machine maxCpus='4'>vexpress-a15</machine>
      <machine maxCpus='1'>realview-pb-a8</machine>
      <machine maxCpus='1'>collie</machine>
      <machine maxCpus='1'>n800</machine>
      <machine maxCpus='4'>highbank</machine>
      <machine maxCpus='1'>kzm</machine>
      <machine maxCpus='1'>sx1-v1</machine ...
edit retag flag offensive close merge delete


Which OS you running on VMs ?

Set value virt_type=kvm in nova.conf on Compute Node, try restart openstack-nova-compute
What happens ?
dbaxps gravatar imagedbaxps ( 2015-06-04 02:57:59 -0500 )edit

Hi, when virt_type on nova-compute.conf is set to kvm, and i try to boot an instance qemu-system-x86 takes 100% of the cpu, cpus "allocated" by the instance will stuck (sometimes CPU#0 stuck for 22s! shows)

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 03:06:17 -0500 )edit

Which OS you running on VMs ?
Which OS you running on baremetal ?
What kind of Virtualization you use on baremetal ?

dbaxps gravatar imagedbaxps ( 2015-06-04 03:14:59 -0500 )edit

I edit my question , everything use Ubuntu 14.04, baremetal virtualization -> kvm (qemu-kvm) i manage the vms with the virt-manager

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 03:19:58 -0500 )edit

VFS: cannot find ext4 file system is system error having nothing in common with virt_type nova settings

dbaxps gravatar imagedbaxps ( 2015-06-04 06:30:57 -0500 )edit

2 answers

Sort by » oldest newest most voted

answered 2015-06-04 02:15:00 -0500

uts9 gravatar image

updated 2015-06-04 02:17:20 -0500

In Ubuntu, run

echo 'options kvm_intel nested=1' >> /etc/modprobe.d/qemu-system-x86.conf


echo “options kvm-intel nested=1″ >> /etc/modprobe.d/kvm-intel.conf
edit flag offensive delete link more


Hi, i think i've done that allready ill edit my question

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 02:26:01 -0500 )edit

Is possible to share any error message or log entry? Also please share the output of $virsh capabilities

uts9 gravatar imageuts9 ( 2015-06-04 02:52:05 -0500 )edit

Hi, i wanna share the output but i cant share files yet, and if i copy paste the text , everything is a mess in the preview of the edit

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 03:44:58 -0500 )edit

is there a way of sharing large chunk of text?

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 03:47:33 -0500 )edit

You can use pastebin or hastebin

uts9 gravatar imageuts9 ( 2015-06-04 03:55:28 -0500 )edit

answered 2015-06-04 01:21:59 -0500

dbaxps gravatar image

Nested KVM implementation on RH systems :-

1. Ensure nested virt is persistent across reboots by adding it as a
   config directive:
    $ cat /etc/modprobe.d/dist.conf
    options kvm-intel nested=y
2. Reboot the host.
3. Check if the Nested KVM Kernel module option is enabled:
    $ cat /sys/module/kvm_intel/parameters/nested
4. Before you boot your L1 guest (i.e. the guest hypervisor that runs
   the nested guest), expose virtualization extensions to it. The
   following exposes all the CPU features of host to your guest
$ virt-xml <VM-NAME>  --edit  --cpu host-passthrough,clearxml=yes

Start your L1 guest (i.e. guest hypervisor):

$ virsh start <VM-NAME>  --console

1. Ensure KVM extensions are enabled in L1 guest by running the below
$ file /dev/kvm      
    /dev/kvm: character special

You might enable Shadow VMCS, APIC Virtualization on the physical host (L0):

  $ cat /sys/module/kvm_intel/parameters/enable_shadow_vmcs
    $ cat /sys/module/kvm_intel/parameters/enable_apicv
   $ cat /sys/module/kvm_intel/parameters/ept
edit flag offensive delete link more


  1. Ensure nested virt is persistent across reboots by adding it as a config directive: $ cat /etc/modprobe.d/dist.conf

dist.conf does not exist on the server, do i have to create it manually?

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 01:28:44 -0500 )edit

And in my case virt-xml command is missing the only "xml" i have is virt-xml-validate

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 01:39:42 -0500 )edit

Create /etc/modprobe.d/dist.conf manually. Follow
Just editing XML VM's definition. Works on any systems

dbaxps gravatar imagedbaxps ( 2015-06-04 03:01:40 -0500 )edit

I guess dist.conf is only for RH. correct me if i'm wrong.

uts9 gravatar imageuts9 ( 2015-06-04 03:36:14 -0500 )edit

Hi, Ive done everything, intances boot and goes to runing stage(as with qemu), but they are stuck cant access even the logs from Horizon (this happen before too )

cloudQuestions gravatar imagecloudQuestions ( 2015-06-04 03:43:12 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools



Asked: 2015-06-04 01:09:19 -0500

Seen: 1,869 times

Last updated: Jun 04 '15