Error mounting with libguestfs (guestfs_launch failed)
I am having issues getting any image at all to boot in our new OpenStack environment. The latest freshly built image (using packer.io) boots but I cannot SSH in. Upon investigating it looks like Nova/KVM/qemu cannot mount the image to inject SSH keys, or setup the networking, hostname etc. Here's the log entry that gets produced in /var/log/nova/compute.log:
Error mounting /var/lib/nova/instances/ea6dc4a1-1746-4c18-90b5-96c226acf68f/disk with libguestfs (guestfs_launch failed.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
and/or run 'libguestfs-test-tool'.))
After Googling around for the issue the only other instances of this error I can find are due to incorrect SELinux labels. However all of our machines have SELinux disabled so this cannot be the issue. I have looked into the following nova options:
libvirt_inject_partition = -1
libvirt_inject_password=true
libvirt_inject_key=true
Explicitly setting these options across the compute nodes has made no difference at all and the error still occurs.
Also running libguestfs-test-tool as the message indicates shows 0 errors.
Images are built as the latest CentOS 6.5 64bit using Packer.
Mounting using guestfish --ro -a /var/lib/nova/instances/UUID/disk -i works without error and I can work throught the filesystem etc as I desire. When I try mount rw however I get the following error:
libguestfs: error: /usr/libexec/qemu-kvm exited with error status 1.
To see full error messages you may need to enable debugging.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
libguestfs: error: guestfs_launch failed.
See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
and/or run 'libguestfs-test-tool'.
Attempting a rw mount again with debugging turned on produces the following output: http://slexy.org/view/s20LP9ri7Q
I have checked the console.log of a Cirros image I booted and I am seeing the following error (clipped to remove anything unnecessary):
info: initramfs: up at 1.25
[ 1.467464] end_request: I/O error, dev vda, sector 0
[ 1.471458] Buffer I/O error on device vda, logical block 0
[ 1.471458] lost page write due to I/O error on vda
***** WARNING: Resize failed, attempting to revert ******
[ 1.493892] end_request: I/O error, dev vda, sector 0
[ 1.497887] Buffer I/O error on device vda, logical block 0
[ 1.497887] lost page write due to I/O error on vda
/dev/vda: Input/output error
Error closing /dev/vda
***** Appears to have gone OK ****
FAILED: failed to resize
Re-reading the partition table ...
info: initramfs loading root from /dev/vda1
info: /etc/init.d/rc.sysinit: up at 1.59
[ 1.601350] end_request: I/O error, dev vda, sector 16067
[ 1.605344] Buffer I/O error on device vda1, logical block 1
[ 1.605344] lost page write due to I/O error on vda1
[ 1.617854] EXT3-fs (vda1): I/O error while writing superblock
................removed.................
[ 6.832592] end_request: I/O error, dev vda, sector 49351
[ 6.836584] Buffer I/O error on device vda1, logical block 16643
[ 6.836584] lost page write due to I/O ...