Ask Your Question
2

juno + centos 7 + ceph: how to add rbd support (nova)

asked 2015-01-27 05:09:08 -0500

Alexey Shafranovich gravatar image

Glance and cinder works fine, but kvm refuses to start new instance because rbd is not supported for volume. Is there any way to add this support? If this configuration should work, what can I check?

PS As i understand rbd support should present in kernel and in qemu-kvm

edit retag flag offensive close merge delete

3 answers

Sort by ยป oldest newest most voted
2

answered 2015-01-27 10:57:51 -0500

dbaxps gravatar image

updated 2015-01-28 03:22:49 -0500

Set up /usr/libexec/qemu-kvm to support "rbd" on CentOS 7

[boris@juno1dev x86_64]$ ls -l
total 12424
-rwxrwxr-x. 1 boris boris     471 Jan 28 11:51 inst
-rw-rw-r--. 1 boris boris  186560 Jan 28 12:04 libcacard-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris  153156 Jan 28 12:04 libcacard-devel-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris  152832 Jan 28 12:04 libcacard-tools-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris  253524 Jan 28 12:04 qemu-guest-agent-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris  627020 Jan 28 12:04 qemu-img-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris 1908828 Jan 28 12:04 qemu-kvm-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris  322188 Jan 28 12:04 qemu-kvm-common-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris 8944284 Jan 28 12:04 qemu-kvm-debuginfo-1.5.3-60.el7.centos.10.x86_64.rpm
-rw-rw-r--. 1 boris boris  148104 Jan 28 12:04 qemu-kvm-tools-1.5.3-60.el7.centos.10.x86_64.rpm
[boris@juno1dev x86_64]$ date
Wed Jan 28 12:08:57 MSK 2015
[boris@juno1dev x86_64]$ ./inst
[sudo] password for boris: 
Preparing...                          ################################# [100%]
Updating / installing...
   1:qemu-img-10:1.5.3-60.el7.centos.1################################# [ 11%]
   2:libcacard-10:1.5.3-60.el7.centos.################################# [ 22%]
   3:qemu-kvm-common-10:1.5.3-60.el7.c################################# [ 33%]
   4:qemu-kvm-10:1.5.3-60.el7.centos.1################################# [ 44%]
   5:libcacard-devel-10:1.5.3-60.el7.c################################# [ 56%]
   6:libcacard-tools-10:1.5.3-60.el7.c################################# [ 67%]
   7:qemu-kvm-tools-10:1.5.3-60.el7.ce################################# [ 78%]
   8:qemu-kvm-debuginfo-10:1.5.3-60.el################################# [ 89%]
   9:qemu-guest-agent-10:1.5.3-60.el7.################################# [100%]

[boris@juno1dev x86_64]$ /usr/libexec/qemu-kvm --drive format=?
Supported formats: vvfat vpc vmdk vhdx vdi sheepdog sheepdog sheepdog rbd raw host_cdrom host_floppy host_device file qed qcow2 qcow parallels nbd nbd nbd iscsi gluster gluster gluster gluster dmg cow cloop bochs blkverify blkdebug

[boris@juno1dev x86_64]$ qemu-img --help | grep "Supported formats"
Supported formats: null-aio null-co host_cdrom cloop dmg host_floppy raw host_device file vvfat parallels qcow nbd iscsi qed tftp ftp ftps https http qcow2 vdi sheepdog blkdebug vhdx rbd quorum vmdk gluster blkverify vpc bochs

Attempt to add kernel module :-
Original Link http://blog.mit.bme.hu/meszaros/en/no...

[root@juno1dev Downloads]# yum install kmod-rbd-3.10-0.1.20140702gitdc9ac62.el7.x86_64.rpm \
> kmod-libceph-3.10-0.1.20140702gitdc9ac62.el7.x86_64.rpm
Loaded plugins: fastestmirror, langpacks, priorities
Examining kmod-rbd-3.10-0.1.20140702gitdc9ac62.el7.x86_64.rpm: kmod-rbd-3.10-0.1.20140702gitdc9ac62.el7.x86_64
Marking kmod-rbd-3.10-0.1.20140702gitdc9ac62.el7.x86_64.rpm to be installed
Examining kmod-libceph-3.10-0.1.20140702gitdc9ac62.el7.x86_64.rpm: kmod-libceph-3.10-0.1.20140702gitdc9ac62.el7.x86_64
Marking kmod-libceph-3.10-0.1.20140702gitdc9ac62.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package kmod-libceph.x86_64 0:3.10-0.1.20140702gitdc9ac62.el7 will be installed
---> Package kmod-rbd.x86_64 0:3.10-0.1.20140702gitdc9ac62.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================================================
 Package
   Arch   Version                         Repository                                        Size
=================================================================================================
Installing:
 kmod-libceph
   x86_64 3.10-0.1.20140702gitdc9ac62.el7 /kmod-libceph-3.10-0.1 ...
(more)
edit flag offensive delete link more

Comments

You have spent too much time to find simple solution. Unstable qemu binary built from source doesn't work with Nova, right ?

s.laskin gravatar images.laskin ( 2015-01-28 04:01:11 -0500 )edit

I'll check it today. I think this solution should work. QEMU should be rebuilt, but as I understand, there is no requirement to use latest version. There is requirement to build it with right params. And kmod - yes. I was looking for kmod in Ceph Giant repo, but it exists in Ceph Testing...

Alexey Shafranovich gravatar imageAlexey Shafranovich ( 2015-01-28 06:44:44 -0500 )edit
0

answered 2015-03-23 09:21:39 -0500

bishoy gravatar image

updated 2015-03-24 10:01:34 -0500

Enabling Ovirt repo and update using it worked for me as well! CHEERS!

[qemu-kvm-rhev]
name=oVirt rebuilds of qemu-kvm-rhev
baseurl=http://resources.ovirt.org/pub/ovirt-3.5/rpm/el7Server/
mirrorlist=http://resources.ovirt.org/pub/yum-repo/mirrorlist-ovirt-3.5-el7Server
enabled=1
skip_if_unavailable=1
gpgcheck=0
edit flag offensive delete link more
0

answered 2015-01-27 15:05:00 -0500

jharley gravatar image

I would recommend reading through the excellent document provided by Inktank:

http://ceph.com/docs/v0.80.5/rbd/rbd-...

edit flag offensive delete link more

Comments

Thank you. Yes it's a way it was set up. But after setup done that way, KVM unable to start virtual machine (already created by OpenStack), because it unable to connect to rbd (nova). And reason is qemu-kvm --drive format=? don't know about rbd at all.

Alexey Shafranovich gravatar imageAlexey Shafranovich ( 2015-01-28 06:49:39 -0500 )edit

If you had previously existing instances before making the changes to nova and cinder (I'm assuming you're booting from volume?), I think you may need to look into configuring multiple storage backends in cinder:

http://docs.openstack.org/admin-guide...

jharley gravatar imagejharley ( 2015-01-28 15:32:28 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2015-01-27 05:04:53 -0500

Seen: 2,020 times

Last updated: Mar 24 '15