Ask Your Question

Where is my root volume stored?

asked 2016-10-11 13:05:54 -0500

jjandco gravatar image

When I provision an instance and say it's from an image in glance that's stored in qcow2 format, does Cinder go ahead and spin up another volume and copy it over to this as persistent storage?

edit retag flag offensive close merge delete

3 answers

Sort by » oldest newest most voted

answered 2016-10-14 09:05:30 -0500

Fenuks gravatar image

When you create and instance, there are 2 kinds of disks: Root and Ephemeral. Default flavors use only root disks. Root disks are stored in /var/lib/nova/instances/<instanceid> on the host where that instance was spinned up (nova compute node), by default. That parameter is controlled by 'instances_path' variable in nova.conf.
When you provision and instance of, for example, m1.tiny flavor from a glance image, nova will create a qcow disk file from that image with the size of 1GB, place it in '$instance_path/instanceID' and spin up the instance from there.

Cinder is a different service providing persistent attachable disks, that will not be destroyed when you delete the instance.

edit flag offensive delete link more


Ah I see, thanks Fenuks! This makes sense

So with that said, does the compute host need to have a RAID controller or some form of boot from SAN config to preserve the running VM's? I come from a VMware background where nobody runs VM's on local storage (always a SAN).

jjandco gravatar imagejjandco ( 2016-10-21 10:10:58 -0500 )edit

@jjandco: Usually Openstack is configured with CephFS cluster filesystem as storage backend. For each storage there'd be a different pool and that pool would be shared between nodes. In case of Cinder and Nova it will allow live migrations between compute nodes, as it will only have to move memory.

Fenuks gravatar imageFenuks ( 2016-10-24 06:05:49 -0500 )edit

@Fenuks is this always the case? What if an operator isn't using CephFS?

jjandco gravatar imagejjandco ( 2016-10-24 12:48:35 -0500 )edit

@jjandco: You can configure it without CephFS, it is recommended, but not mandatory. You can also use any other way of sharing storage space — SAN, iSCSI, another server with NFS shares, etc. Just make sure it is mounted and checked before nova starts and is fast enough.

Fenuks gravatar imageFenuks ( 2016-10-24 21:57:28 -0500 )edit

answered 2016-10-12 05:01:20 -0500

updated 2016-10-12 05:02:40 -0500

if you have an instance which as nova storage as back-end you could snapshot this instance, and this snapshot become a new glance images, after this you could create a new cinder volume using the previously created glance image: volumes -> new volume -> from volume source select image -> select the glance image

HTH Amedeo

edit flag offensive delete link more


I see but that's not what I'm asking.

Say I provision a m1.tiny instance, which has a 1GB root volume. Does this 1GB live on Cinder (at whatever storage provider, either SAN Or Storage Node) or does it live on the compute node?

jjandco gravatar imagejjandco ( 2016-10-12 15:35:47 -0500 )edit

What do you mean by "volume"? A Cinder volume lives in Cinder. An instance's ephemeral storage sits on the Compute node (but of course, the Compute node may use an external storage system or NFS etc). If you boot from a Glance image, you probably mean ephemeral storage.

Bernd Bausch gravatar imageBernd Bausch ( 2016-10-13 01:35:17 -0500 )edit

So like I said, say I provision a VM using an image from glance. This VM spins up, I go ahead and put it on a host and SSH In, start installing stuff. Where does this virtual disk live? On the compute hosts local storage?

jjandco gravatar imagejjandco ( 2016-10-21 09:48:17 -0500 )edit

Normally on the compute node. There is a Nova option instances_path that determines the directory.

Bernd Bausch gravatar imageBernd Bausch ( 2016-10-24 23:34:03 -0500 )edit

answered 2016-10-13 16:51:59 -0500

dkt26111 gravatar image

No, cinder won't be involved if you launch an instance from a glance image. A copy of the glance image is made and stored on the compute node. If you are running kvm hypervisor, you can run 'virsh list' and the 'virsh dumpxml INSTANCENAME' to see the xml definition of the instance. This contains the path to the image stored on the compute node. The VM will boot off image stored in that path.

edit flag offensive delete link more


You can actually launch an instance from an image and use a volume for storage. For example nova boot ... -block-device source=image,dest=volume,id=<imageuuid>.

Bernd Bausch gravatar imageBernd Bausch ( 2016-10-24 23:36:18 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2016-10-11 13:05:54 -0500

Seen: 2,564 times

Last updated: Oct 14 '16