Ask Your Question

separate storage node

asked 2020-05-29 01:24:55 -0500

happy75 gravatar image

HI Openstack experts,

I understand that with Openstack, there are different storage services options (e.g. cinder, swift, ceph etc). From a design perspective, is it better to have a separate node for storage (if we are using external storage)? Or it wouldn't make much difference (e.g. from performance perspective) if we are using the controller node as the storage node as well?


edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2020-06-03 13:20:57 -0500

chalans gravatar image

To complete the answers :

LVM use ISCSI portals on storage Node usually with DAS Storage . Then Cinder expose it over bloc mode on instance.

If you have already a storage Array, better to use the constructor Driver (Netapp, Dell-EMC, Hitachi) for a direct FC access to your Lun => best performance and fiability . LVM was for testing and should not be use in a production environnement .

Best storage with OpenStack is CEPH (my opinion ;)

edit flag offensive delete link more


thanks for the suggestions

happy75 gravatar imagehappy75 ( 2020-06-05 01:12:13 -0500 )edit

May i ask why you think ceph is the best storage - is it because it is the "unified storage solution" where it works for file, block and object storage? whereas swift is only object storage and cinder is only block storage?

happy75 gravatar imagehappy75 ( 2020-06-08 20:15:38 -0500 )edit

answered 2020-05-29 07:58:28 -0500

First, you need to know what kind of storage you want for your cloud. Cinder, Swift and Ceph have totally different purposes. Cinder provides volumes, comparable to disk array LUNs, which you can attach to instances. Swift is object storage; usually good for large, mostly read-only objects such as archives or videos, which can be accessed without instances. Ceph is not an OpenStack component but can be used as a backend of Nova, Glance and Cinder, and can be used as a standalone object store.

A Cinder storage node is only needed if you use LVM as backend. If your Cinder backends are disk arrays or Ceph, you don't need a separate Cinder storage node; instead, you run cinder-volume on the controllers.

You can run Swift on the controllers, but if you have big plans for your object storage, you will dedicate separate servers to the Swift cluster. Same for Ceph. In fact, you may already have a Ceph cluster and just use it as a backend for OpenStack services.

edit flag offensive delete link more


thanks for your prompt & detailed response! Can i ask why a separate storage node needed if we use LVM as a backend? But i thought you can still use LVM on disk arrays ?

happy75 gravatar imagehappy75 ( 2020-05-29 08:17:59 -0500 )edit

Also the other thing that I am a bit confused is if I use cinder as the storage service for my cloud because i need persistent storage, does that mean I should be creating all my instances using the cinder volumes (block storage) and not the default ephemeral storage which is created by Nova?

happy75 gravatar imagehappy75 ( 2020-05-29 08:20:20 -0500 )edit

if that's the case that means the operating system of the instances would also be on the cinder volumes?

happy75 gravatar imagehappy75 ( 2020-05-29 08:21:50 -0500 )edit

A separate node is not needed for LVM. It can be better for performance reasons, and generally to separate duties.

If you create your instances with ephemeral storage, their root disks won't be persistent. If that is OK, do it like that.

When you boot from a volume, the root disk is a volume.

Bernd Bausch gravatar imageBernd Bausch ( 2020-05-29 10:06:02 -0500 )edit

okay thanks Bernd. So does that mean (1) if I create an instance using Nova, the instance is created in /var/lib/nova/instances/ on the compute node. (2) If I create an instance using Cinder, it can be booted from a Cinder volume in LVM on controller node)?

happy75 gravatar imagehappy75 ( 2020-06-02 20:24:48 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2020-05-29 01:24:55 -0500

Seen: 130 times

Last updated: Jun 03 '20