Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

How is the implementation of native glusterfs in cinder?

Hello there,

I've set up a multi-node environment of openstack using packstack for the company I work for, the main idea behind it is to test it and see the benefit of it for our internal IT operations.

The multi-node setup is made up of 4 servers, 1 that acts as the controller, database, message broker, neutron server, image server and storage, and 3 nodes running nova-compute services and openvswitch agents.

All the hosts are fedora 19 up to date because we wanted to test glusterfs native integration with cinder (libgfapi)

Install/deploy method was packstack and went without issues.

Currently there's only one cinder volume created without any particularities, basically gluster vol create GlusterCinder0 replica 3 vm1:/gluster/vol0 vm2:/gluster/vol0 vm3:/gluster/vol0

The packstack answer file was configured to use gluster and the gluster volume address

Everything went just fine with one exception, I don't see the native implementation in play

After googling I read that I needed to activate the feature by editing qemu_allowed_storage_drivers in nova.conf which I did (restarted cinder-volume and nova services and even all the hosts)

I checked the cinder_shares files which seems ok and currently cinder+glusterfs is working (I can create volumes, attach them to instances, delete volumes, etc...) what seems not to be working is the "native" implementation, by that I mean: I execute "mount" on all the servers and all the mount points used by cinder show fuse.glusterfs as mount type

I can create images directly in the gluster volume using qemu-img create and using native gluster backend (e.g: gluster://ip/vol/image size) so I know that on libvirt side everything seems to work as expected (qemu is version 1.4+, gluster is 3.4.1)

Could you point me to what could I be missing on openstack's side? Maybe I'm not seeing where I should or fuse.glusterfs as mount type is somehow normal, or some better test I could use to verify gluster usage by cinder... anything will be appreciated.

That's all, thanks for reading and sorry for the long post, just wanted to made it clear.