Revision history [back]

click to hide/show revision 1
initial version

machine creation really slow

Hello,

we have a 3 node setup of openstack havanna, using the automated rackspace cookbooks.

Everything work correctly but I have noticed that when creating new mahcines with a flavour that has a big HDD (400Gb for example) it will take like 20 minutes too boot up the machine.

Looking at the /var/lib/nova/<instance_id> I can see the "disk" file growing but it's incredibly slow!! it does about 3Mb/s.

I played with the image configuration in nova but the machines are still slow as hell.

The nodes have 4 2Tb HDD with LVM.

This is an iotop of one of the nodes using about 7Mb/s, as you can see it's only doing writes because it's creating the HDD files for the machines:

  778 be/3 root        0.00 B/s    7.88 K/s  0.00 % 21.97 % [jbd2/dm-0-8]
24194 be/4 libvirt-    0.00 B/s 1613.74 B/s  0.00 %  6.77 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000266 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
24105 be/4 libvirt-    0.00 B/s 1613.74 B/s  0.00 %  5.73 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000266 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
  449 be/4 libvirt-    0.00 B/s   14.97 K/s  0.00 %  4.89 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-0000027e -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
  655 be/4 libvirt-    0.00 B/s   14.97 K/s  0.00 %  4.61 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-0000027e -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
  574 be/4 libvirt-    0.00 B/s   10.24 K/s  0.00 %  4.43 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000274 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
24084 be/4 libvirt-    0.00 B/s 1613.74 B/s  0.00 %  4.19 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000266 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
24085 be/4 libvirt-    0.00 B/s 1613.74 B/s  0.00 %  4.09 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000266 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
24074 be/4 libvirt-    0.00 B/s 1613.74 B/s  0.00 %  3.93 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000264 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
24258 be/4 libvirt-    0.00 B/s   14.38 K/s  0.00 %  3.83 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000264 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
24228 be/4 libvirt-    0.00 B/s  404.22 K/s  0.00 %  3.67 % qemu-system-x86_64 -machine accel=kvm:tcg -name instance-00000264 -S -machi~en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

This is a test with hddparm on the same HDD:

/dev/mapper/controller1--vg-root:
 Timing cached reads:   18934 MB in  2.00 seconds = 9477.53 MB/sec
 Timing buffered disk reads: 380 MB in  3.01 seconds = 126.39 MB/sec

Configurations in nova.conf that could affect this:

libvirt_inject_key=false
libvirt_inject_password=false
libvirt_inject_partition=1
disk_cachemodes="file=none"
connection_type=libvirt
compute_driver=libvirt.LibvirtDriver
libvirt_type=kvm
# Inject the ssh public key at boot time (default: true)
libvirt_inject_key=false
libvirt_inject_password=false
libvirt_inject_partition=1
force_raw_images=false

Any idea of what could be the problem?