Ask Your Question

Using Cinder to store VM images (SAN with Fibre Channel)

asked 2015-11-03 10:13:39 -0500

Herr-Herner gravatar image

We are working on a Liberty setup that makes use of an IBM DS6000 that is connected to 2 Brocade Fibre Switches. Each Compute Node has an HBA installed that is connected to each of the Fibre Switches. We want to configure Nova in that way that the VM images are stored in the SAN and not locally on the Compute Nodes' HDDs. The traffic should go entirely through the Fibre Channel network without any ethernet access.

What is the right approach here? Is there any way possible to tell Nova to use Cinder as storage backend for the VM images? Our goal is the accelerate live migration so that the VM's HDDs must not be transfered from one Compute Node to the other.

I have only found this reference here:

I am sure that it is not enough to just install some additional packages. We are planning to use Ubuntu Server 14.04. as OS.

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2015-11-03 11:19:15 -0500

capsali gravatar image

updated 2015-11-04 05:33:41 -0500

So you want the images from whom you boot instances to be stored on the SAN? Glance handles images in openstack. Take a look at link text

Set up cinder to use SAN and then setup glance to use cinder as a backend to store images.

Based upon your input i got it all wrong. You can mount a shared file system under /var/lib/nova on all compute nodes and can live migrate without block migrate. We currently are using something like this.

We don't use nova;s ephemeral storage, only block volumes but nova still keeps files under /var/lib/nova, instance state, etc.

We use ceph as shared storage. So we mounted a ceph filesystem on all compute nodes under /mnt/nova and changed nova_state from /var/lib/nova to /mnt/nova (we created all subfolders of nova folder).

So sure, you can mount a shared filesystem under /var/lib/nova and take advantage of live migration. But to support live migration you need to have passwordless ssh acces on all compute nodes, make libvirt listen to tcp and add an entrie in nova.conf under [libvirt] for live migration to work!

Hope i got it right this time! :)

edit flag offensive delete link more

answered 2015-11-03 14:08:06 -0500

Herr-Herner gravatar image

I think my wording is not ok... What we want is a shared volume between our Compute Node, so that the VM data disks located on each server under /var/lib/nova/instances gets shared. We want to avoid that during live migration the disk must be copied from the source node to the target node.

Is it possible to create a shared volume with a cluster filesystem e.g. OCFS2 and mount via FC to /var/lib/nova/instances on every Compute Node?

edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2015-11-03 10:13:39 -0500

Seen: 2,059 times

Last updated: Nov 04 '15