Hi, I had a working openstack enveironment that used local lvm storage for vm's and volumes. Then i saw that ceph would be a better alternative to local storage and it supports live migration. So i went on an installed ceph.

From a storage pov i have 2 servers with 8x4TB hdd each and 3 compute nodes with 1x240GB ssd's each. Beside this i have 2 controllers aswell. So i added all storage devices to ceph and edited the crush map with 2 new rules, one for the osd's that reside on the hdd, so 16 osd on one rule and the other rule for the 3 ssd. Then i created 3 pools, volumes,images and ephemeral. Volumes and images were set to follow hdd rule and ephemeral ssd rule. I edited glance.conf, cinder.conf and nova.conf by the documentation on ceph site and made accounts for them. I copied the keys to the apropriete nodes, and followed the key integration for libvirt for the compute nodes.

So then i uploaded an image into glance, a ubuntu cloud image to be exact. When i tried to boot the image i got an error that no host was found. I checked nova-scheduler.log and found out that it was complaining about not having cinder.client key so it could not authentificate with cephx, although i integrated the key into libvirt. So i copied cinder.client key on the compute nodes into /etc/ceph. So now i could boot the ubuntu cloud image and from "ceph df" i can see it uses up space in ephemeral pool.

So then i went on and downloaded windows server 2012 triial image from openstack documentation. It's a 16GB image. I converted it to raw format, because as i read ceph doesnt support any other type of image to boot from except raw. I uploaded the newly raw image into glance and booted it up. To my surprise i got the same error that no valid host was found. So again i looked into nova-scheduler.log and this time is a different error. It says that there is not enough space left, althought i have more than enough. I might be doing something wrong, because my ceph.conf file is the same as it was when i installed it with ceph-deploy. I saw some samples on the net where some ceph.conf specified the osd's and other info.

I seriously don't know what to do next. Any help would be apreciated.

Update:

After doing more reading, i found somewhere that when i issue an instance creation from a glance image, the image is downloaded locally on a compute node and then uploaded into ceph again to the other pool.

The problem in my case is that i don't have enought local space on my compute node, only 9gb availabe and since windows 2012 image is about 16gb i get the not enought space error. Is there any ... (more)