I have an Openstack cluster with 1 controller and 2 compute nodes at https://www.cloudlab.us/ (CloudLab) and I am trying to enable live-migration with GlusterFS shared storage following the steps listed http://yongluo2013.github.io/openstack-vm-live-migration/ (here). I have setup the GlusterFS Volume and it is mounted at /var/lib/nova/instances.

root@cp-2:/users/Jasim9# gluster volume info

Volume Name: nova-gluster-vol
Type: Replicate
Volume ID: ae2baad8-0363-4391-8f24-86593648f7f9
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: cp-1:/brick1/sdb
Brick2: cp-2:/brick1/sdb
Options Reconfigured:
performance.readdir-ahead: on

as can be seen in df on compute nodes:

root@cp-1:/users/Jasim9# df -hT
Filesystem                                     Type            Size  Used Avail Use% Mounted on
udev                                           devtmpfs        126G     0  126G   0% /dev
tmpfs                                          tmpfs            26G   34M   26G   1% /run
/dev/sda1                                      ext3             16G  2.9G   12G  20% /
tmpfs                                          tmpfs           126G     0  126G   0% /dev/shm
tmpfs                                          tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs                                          tmpfs           126G     0  126G   0% /sys/fs/cgroup
ops.clemson.cloudlab.us:/proj/live-migrate-PG0 nfs             100G  512K  100G   1% /proj/live-migrate-PG0
ops.clemson.cloudlab.us:/share                 nfs              97G  1.2G   88G   2% /share
tmpfs                                          tmpfs            26G     0   26G   0% /run/user/0
cgmfs                                          tmpfs           100K     0  100K   0% /run/cgmanager/fs
/dev/sda4                                      ext4            896G   73M  850G   1% /var/lib/nova
tmpfs                                          tmpfs            26G     0   26G   0% /run/user/20001
/dev/sdb                                       xfs             932G  865M  931G   1% /brick1
cp-1:/nova-gluster-vol                         fuse.glusterfs  932G  865M  931G   1% /var/lib/nova/instances

root@cp-2:/users/Jasim9# df -hT
Filesystem                                     Type            Size  Used Avail Use% Mounted on
udev                                           devtmpfs        126G     0  126G   0% /dev
tmpfs                                          tmpfs            26G   34M   26G   1% /run
/dev/sda1                                      ext3             16G  2.9G   12G  20% /
tmpfs                                          tmpfs           126G     0  126G   0% /dev/shm
tmpfs                                          tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs                                          tmpfs           126G     0  126G   0% /sys/fs/cgroup
ops.clemson.cloudlab.us:/proj/live-migrate-PG0 nfs             100G  512K  100G   1% /proj/live-migrate-PG0
ops.clemson.cloudlab.us:/share                 nfs              97G  1.2G   88G   2% /share
tmpfs                                          tmpfs            26G     0   26G   0% /run/user/0
cgmfs                                          tmpfs           100K     0  100K   0% /run/cgmanager/fs
/dev/sda4                                      ext4            896G   73M  850G   1% /var/lib/nova
tmpfs                                          tmpfs            26G     0   26G   0% /run/user/20001
/dev/sdb                                       xfs             932G  885M  931G   1% /brick1
cp-2:/nova-gluster-vol                         fuse.glusterfs  932G  885M  931G   1% /var/lib/nova/instances

I have added the necessary commands in /etc/libvirt/libvirtd.conf on both compute nodes:

listen_tls = 0
listen_tcp = 1
auth_tcp = “none”

and added

# options passed to libvirtd, add "-l" to listen on tcp
libvirtd_opts="-l"

to /etc/default/libvirt-bin and the flags in /etc/nova/nova.conf on both compute nodes:

live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE

Have created two VM instances (test2 on cp-1 test on cp-2):

root@ctl:/users/Jasim9# nova list
+--------------------------------------+-------+--------+------------+-------------+----------------------------+
| ID                                   | Name  | Status | Task State | Power State | Networks                   |
+--------------------------------------+-------+--------+------------+-------------+----------------------------+
| a9e80a50-f849-44e9-9e04-2a5a89ed7f43 | test  | ACTIVE | -          | Running     | flat-lan-1-net=10.11.10.10 |
| b0de71ab-75df-4c21-9648-e69c47bd43e2 | test2 | ACTIVE | -          | Running     | flat-lan-1-net=10.11.10.11 |
+--------------------------------------+-------+--------+------------+-------------+----------------------------+

but when trying to live-migrate them to other host ... (more)