Cinder storage question

asked 2020-08-04 20:32:50 -0500

happy75 gravatar image

Hi Openstack experts, I have some questions with regards to using LVM as the backend storage for Nova/Cinder. I have setup cinder as per openstack official website instructions where I created a new cinder volume volume group on the storage node (i.e. I have setup a separate storage node to have the cinder volume service and cinder target service on there). When I created the new instances from horizon, I selected "create new volume" for the instances. I noticed that it (1) creates a file under /var/lib/nova/instances/ and (2) a file within the volume group "cinder volumes" on the storage node My questions are: In my scenario, does my instance actually exist on compute:/var/lib/nova/instances or within the cinder volumes volume group on the storage node? What exactly (i.e. OS or data) are stored under compute:/var/lib/nova/instances and what exactly (i.e. OS or data) are stored under the volume group cinder volumes on the storage node? it creates a new volumes which is attached to the instances I created (I can actually see the new volumes in horizon), does that mean that volume can be detached (provided that I shutdown the instance where the volume was initially attached to) and re-attach to another instance?


edit retag flag offensive close merge delete


The LVM backend uses LVM volumes (not a "file" as you say) to represent Cinder volumes. You can see those LVM volumes with the lvs command.

When a volume is attached to an instance, the storage node creates an iSCSI target which the compute node imports. The VM uses this iSCSI device as a disk.

Bernd Bausch gravatar imageBernd Bausch ( 2020-08-05 06:54:18 -0500 )edit

In your case, the VM's root disk should be such an iSCSI device. I don't know what file is created under /var/lib/nova/instances; can you run the file command on it and report the result?

Bernd Bausch gravatar imageBernd Bausch ( 2020-08-05 06:55:21 -0500 )edit

thanks Bernd! On the compute node, under /var/lib/nova/instances/, there are subdirectories with names such as f233424535-9def-n523-34ldbbdfsdfs and inside those subdirectories, it's the console.log file so that means the OS and data for the instances are all stored under the cinder volumes?!

happy75 gravatar imagehappy75 ( 2020-08-05 19:29:23 -0500 )edit

does that mean if i am to have two storage nodes (for redundancy of instances - i.e. to have instances on two physical servers), do I just setup the storage node by installing the cinder volume and target service on there and then create a new volume group with the name cinder-volumes?

happy75 gravatar imagehappy75 ( 2020-08-05 19:32:04 -0500 )edit

If I don't specify it (i.e. if I only select the option to "create new volume" in horizon) - How does Nova/Cinder know which storage node (coz in my scenario i will have 2 storage nodes) to create the new volume in?

happy75 gravatar imagehappy75 ( 2020-08-05 19:33:38 -0500 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2020-08-06 19:42:15 -0500

/var/lib/nova/instances/$UUID contains information about the instance with that UUID. console.log contains the startup messages of an instance. If the instance uses ephemeral storage (i.e., not volume storage) for its root filesystem, one of the files in this directory may be the instance's root filesystem, often in qcow2 format. However, even ephemeral storage can be configured differently, for example in an LVM volume on the compute node. There are several configuration options.

To manually set up two storage nodes that use LVM backends, follow the installation guide. In short, you are right - you install Cinder-Volume, target software and a few supporting services on storage nodes.

You can then map the storage nodes to two different volume types so that users can select the node by selecting the type. If users don't specify a type, Cinder will use the default type you have configured. If you don't configure a default type, see below.

If the two storage nodes have no different types, the storage node on which a new volume is created depends on factors like the number of volumes, or gigabytes reserved. I am not 100% certain; see the Cinder Scheduler documentation.

edit flag offensive delete link more


thanks Bernd so just to clarify - does that mean I can have two separate storage nodes with the same volume group name cinder-volumes? And if that's the case when user create the instance with the new volume, Openstack will put the new volume wherever it thinks is the best ?

happy75 gravatar imagehappy75 ( 2020-08-10 23:54:19 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2020-08-04 20:32:50 -0500

Seen: 117 times

Last updated: Aug 06