Revision history [back]

click to hide/show revision 1
initial version

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns that key for client.cinder.

But, if i lauch an instance with ephemeral storage, the storage is created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but in the secret tag ephemeral is set to no. Could this cause the problems??

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns that key for client.cinder.

But, if i lauch launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> </uuid> <usage type="ceph"> type="ceph"> <name>client.cinder secret</name>
</usage> </secret></secret>

XXX is the correct key but in the secret tag ephemeral is set to no. Could this cause the problems??

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value' secret-get.value UUID' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns for client.cinder.

But, if i launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but in the secret tag ephemeral is set to no. Could this cause the problems??

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value UUID' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns for client.cinder.

But, if i launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml' secret-dumpxml UUID' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but what nakes ne wonder is that in the secret tag ephemeral is set to no. Could this cause the problems??

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value UUID' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns for client.cinder.

But, if i launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml UUID' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but what nakes makes ne wonder is that in the secret tag ephemeral is set to no. Could this cause the problems??

------- Edit2 ------- Ok forget the last question this 'ephemeral' only relates to the key itself.

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value UUID' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns for client.cinder.

But, if i launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml UUID' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but what makes ne wonder is that in the secret tag ephemeral is set to no. Could this cause the problems??

------- Edit2 ------- -------

Ok forget the last question this 'ephemeral' only relates to the key itself.

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value UUID' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns for client.cinder.

But, if i launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml UUID' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but what makes ne wonder is that in the secret tag ephemeral is set to no. Could this cause the problems??

------- Edit2 -------

Ok forget the last question this 'ephemeral' only relates to the key secret itself.

Thx Stefan

Live-migration only works for instances launched by 'Boot from image (create new volume)'

Hi,

got a running openstack installation with ceph as theonly configured storage backend. Today i needed live migration to move around a couple of instances. So i tried:

nova live-migration <instance-id> <new-host-name>

During that i recognized that instances launched with the option 'Boot from image (create new volume)' could easily be live-migrated. But when it comes to the live-migration of an instance launched with the option 'boot from image' only i get the following error when i try to live migrate this instance.

ERROR (BadRequest): node03 is not on shared storage: Live migration can not be used without shared storage. (HTTP 400) (Request-ID: req-d9d3193b-8c0d-4325-89e4-1df42182adc6)

Whereby 'node03' is the name of the compute node the machine i try to migrate is currently running on. I suspect that i have some miss-configuration, can someone point me what and where?

---------- Edit ----------

Ok i checked the whole thing twice. All compute nodes have the appropriate storage.client.cinder.keychain file in /etc/ceph (the cluster is named as storage) and the key in libvirt injected. At least 'virsh secret-list' returns an UUID for ceph client.cinder secret. 'virsh secret-get.value UUID' returns the key, which is the same as in the keychain and that 'ceph --cluster auth list' returns for client.cinder.

But, if i launch an instance with ephemeral storage, the storage is still created locally not in ceph. The only thing that that makes me wonder is that 'virsh secret-dumpxml UUID' returns

<secret ephemeral="no" private="no">
<uuid> XXX </uuid> <usage type="ceph"> <name>client.cinder secret</name>
</usage> </secret>

XXX is the correct key but what makes ne wonder is that in the secret tag ephemeral is set to no. Could this cause the problems??

------- Edit2 -------

Ok forget the last question this 'ephemeral' only relates to the secret itself.

------- Edit3 -------

nova.conf

[DEFAULT] dhcpbridge_flagfile=/etc/nova/nova.conf dhcpbridge=/usr/bin/nova-dhcpbridge logdir=/var/log/nova state_path=/var/lib/nova lock_path=/var/lock/nova force_dhcp_release=True libvirt_use_virtio_for_bridges=True connection_type=libvirt root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf ec2_private_dns_show_ip=True api_paste_config=/etc/nova/api-paste.ini enabled_apis=ec2,osapi_compute,metadata

vif_plugging_is_fatal: false vif_plugging_timeout: 0

rabbit_hosts = rabbit1:5672,rabbit2:5672,rabbit3:5672 rabbit_port = 5672 rabbit_use_ssl = false rabbit_userid = XXX rabbit_password = XXX rabbit_virtual_host = / rabbit_durable_queues = False rabbit_ha_queues = true

auth_strategy = keystone

my_ip = 192.168.25.12

vnc_enabled = True vncserver_listen = 0.0.0.0 vncserver_proxyclient_address = vncserver novncproxy_base_url = http://vncserver:6080/vnc_auto.html

network_api_class = nova.network.neutronv2.api.API security_group_api = neutron linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver firewall_driver = nova.virt.firewall.NoopFirewallDriver

service_neutron_metadata_proxy = true

[database] connection = mysql://nova:XXX@database/nova

[keystone_authtoken] auth_uri = http://keystone:5000/v2.0 identity_uri = http://keystone:35357 admin_tenant_name = service admin_user = nova admin_password = XXX revocation_cache_time = 10

[glance] host = glance.wcloud.uni-koblenz.de

[neutron] url = http://neutron:9696 auth_strategy = keystone admin_auth_url = http://keystone:35357/v2.0 admin_tenant_name = service admin_username = neutron admin_password = XXX service_metadata_proxy = True metadata_proxy_shared_secret = XXX

[libvirt] libvirt_images_type = rbd libvirt_images_rbd_pool = vms libvirt_images_rbd_ceph_conf = /etc/ceph/storage.conf libvirt_rbd_user = cinder libvirt_rbd_secret_uuid = XXX

libvirt_inject_password = false libvirt_inject_key = false libvirt_inject_partition = -2

libvirt_migration_uri=qemu+tcp://%s/system libvirt_live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST

rdb_user = cinder rdb_secret_uuid = XXX

Thx Stefan