How does disk access work with instance backed by cinder volume.

asked 2018-11-20 22:55:06 -0600

John Smith gravatar image

updated 2018-11-21 09:51:47 -0600

Hi, I created a VM using horizon. I picked source as image and I set a volume size of 120GB.

When the VM runs the disk it is visible as /dev/vda and this disk is created on block storage correct?

So when this instance is running on the computer node, if it's a single disk, which part of it is running on the compute and which part is on the block storage? Or in this case it's all running on block storage?

But when reads/writes happen on the VM on the compute node are they not cached locally first and then written to the Cinder block storage?

Update 2018/11/21: Using KVM, Cinder with iSCSI on 10 Gigabit

edit retag flag offensive close merge delete


what's your storage backend?

rduncan-t gravatar imagerduncan-t ( 2018-11-21 05:06:38 -0600 )edit

KVM, Cinder with iSCSI over 10 gigabit network.

John Smith gravatar imageJohn Smith ( 2018-11-21 09:47:01 -0600 )edit

1 answer

Sort by » oldest newest most voted

answered 2018-11-21 00:27:19 -0600

updated 2018-11-21 01:02:48 -0600

By default, Horizon uses a volume as the root file system. You can however configure Horizon so that the default is not a volume, but ephemeral storage backed by a file, logical volume or RBD on the compute node. You can also chamge the default by clicking a switch when launching the instance.

If you use the default, indeed /dev/vda or /dev/sda is created as a block storage volume. In this case, there are no regular files on the compute node that implement the VM’s root device. No compute node storage is used; it’s 100% volume.

Whether it’s cached on the compute node depends on the volume driver and perhaps the hypervisor. Many drivers connect the compute node to the volume backend with iSCSI, for example. In this case, I believe caching takes place on the compute node, except if the hypervisor opend the device file with the O_DIRECT flag. I don’t know what hypervisors usually do and if that’s configurable.

EDIT: If you use a libvirt hypervisor, it is configurable with disk_cachemodes in nova.conf.

The RBD driver, on the other hand, connects the VM to the Ceph cluster using its own protocol, which probably doesn’t use the compute node’s cache.

Of course, the VM has its own cache.

edit flag offensive delete link more


Using KVM.

John Smith gravatar imageJohn Smith ( 2018-11-21 09:51:24 -0600 )edit

That means libvirt, and the disk_cachemodes parameter applies.

Bernd Bausch gravatar imageBernd Bausch ( 2018-11-21 16:14:22 -0600 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2018-11-20 22:54:44 -0600

Seen: 192 times

Last updated: Nov 21 '18