How to always create root disks as volumes on cinder?

asked 2018-11-06 12:42:26 -0600

EvanFoster gravatar image

Hey folks!

I've got an interesting problem here. I've been beating my head against this for a few weeks and haven't been able to find anything concrete.

I'm using Kubernetes and trying to integrate it with Openstack so I can get automatic VM provisioning. The cloud driver for Kubernetes takes the following information:

  • Flavor
  • Image
  • Network
  • Floating IP pool

And a few other things that aren't that important. Here's where my issue is.

My compute nodes have basically no disks. I have a large, very fast SAN and I want all block storage on that SAN. When you create a flavor or image, you can specify a root disk size, but this root disk is created locally on the compute node. This means that I have around 20 GB for all VMs running on any given compute node.

So, is there any way that I can force all root disks to be created as Cinder volumes? I can do this manually, but I need it to automatically happen all the time.

The process I want was described to me like this on IRC:

[11:33] <larsks> So what you really want is for openstack, when booting an instance, is to (a) create a volume for you and (b) copy over the data from the boot image and then (c) boot the instance from that volume?

I'm running Openstack Ocata. My Cinder node is using LVM as its backend.

edit retag flag offensive close merge delete


Did you get an answer from Lars?

How do you create those VMs now? Do you use Magnum, do you create them manually, or do you have another mechanism?

In principle, using a volume as an instance's root disk is a trivial problem. In Horizon, it's even the default.

Bernd Bausch gravatar imageBernd Bausch ( 2018-11-06 18:10:05 -0600 )edit

Hey Bernd,

I'm using Rancher to manage k8s. Rancher uses docker-machine on the back end to create VMs on OpenStack. docker-volume doesn't allow you to specify anything other than an image and a flavor.

What I want isn't possible, so I'm going to use Ceph for my root disks and Cinder otherwise.

EvanFoster gravatar imageEvanFoster ( 2018-11-06 19:08:06 -0600 )edit

If docker-machine allowed me to create volumes then this would be possible, but you can't specify volume creation using an image and flavor alone. You have to create a volume from an image using disk size from the flavor and then run that instance from that volume.

EvanFoster gravatar imageEvanFoster ( 2018-11-06 19:10:38 -0600 )edit

So it seems the problem is docker-machine, or the OpenStack driver for docker-machine.

Yes, of course you can use LUNs from your SAN or any other remote storage solution as disks for instances' ephemeral storage.

Bernd Bausch gravatar imageBernd Bausch ( 2018-11-06 21:26:59 -0600 )edit

And to answer the question in your OP: The API to create a new instance has an optional block device mapping parameter. If you leave it out, ephemeral storage is used - but it's up to config variable instances_path where that ephemeral storage is actually located. E.g. NFS, Ceph, ....

Bernd Bausch gravatar imageBernd Bausch ( 2018-11-06 21:42:05 -0600 )edit