Ask Your Question
1

not able to create volume from image with ceph backend after update from icehouse to juno

asked 2014-10-23 09:34:59 -0500

groen692 gravatar image

updated 2015-01-23 09:14:28 -0500

* this problem is solved by updating package python-glance_store from 0.1.8.2.1 to version 0.1.10-5.1 *

    Hi,

    I am currently running a openstack setup with a ceph backend for glance and cinder. 
    on icehouse it was working ok. I updated the setup to Juno and
    now I am not able to create volume from image anymore. The Volume status says error, However when I look at ceph with rbd there is a volume present.

    In the cinder volume log I can see ImageCopyFailure: Failed to copy image to volume: HTTPInternalServerError (HTTP 500)

update 2) at the glance api logs it says TypeError: 'ImageProxy' object is not callable

update 3)

A simple download from glance also does not seem to work, it fails with too many values to unpack

phoenix:/var/log/glance # glance image-download 1f5f7809-34a9-4035-bcf9-bb527a03720b 2014-11-04 06:39:20.216 5953 DEBUG routes.middleware [d84e93d9-9f5c-47f3-950c-ff3f0fe247f1 db3cb9cfeddd4cc294ede6cef97c2c89 03110aa32a35426bbc7d861dcb61ad94 - - -] Match dict: {'action': u'download', 'image_id': u'1f5f7809-34a9-4035-bcf9-bb527a03720b', 'controller': <glance.common.wsgi.resource object="" at="" 0x3343c90="">} __call__ /usr/lib/python2.7/site-packages/routes/middleware.py:103 2014-11-04 06:39:20.251 5953 DEBUG oslo.db.sqlalchemy.session [d84e93d9-9f5c-47f3-950c-ff3f0fe247f1 db3cb9cfeddd4cc294ede6cef97c2c89 03110aa32a35426bbc7d861dcb61ad94 - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _init_events /usr/lib/python2.7/site-packages/oslo/db/sqlalchemy/session.py:461 2014-11-04 06:39:20.391 5953 WARNING glance.location [d84e93d9-9f5c-47f3-950c-ff3f0fe247f1 db3cb9cfeddd4cc294ede6cef97c2c89 03110aa32a35426bbc7d861dcb61ad94 - - -] Get image 1f5f7809-34a9-4035-bcf9-bb527a03720b data failed: too many values to unpack. 2014-11-04 06:39:20.392 5953 ERROR glance.location [d84e93d9-9f5c-47f3-950c-ff3f0fe247f1 db3cb9cfeddd4cc294ede6cef97c2c89 03110aa32a35426bbc7d861dcb61ad94 - - -] Glance tried all active locations to get data for image 1f5f7809-34a9-4035-bcf9-bb527a03720b but all have failed.

It looks to me it makes an sql query and the values returned cannot be parsed properly

    BR,
    Jeroen



    phoenix:/etc/cinder # glance image-list
    +--------------------------------------+-------------------------+-------------+------------------+----------+--------+
    | ID                                   | Name                    | Disk Format | Container Format | Size     | Status |
    +--------------------------------------+-------------------------+-------------+------------------+----------+--------+
    | b7fff5f9-d63d-4729-ac1e-b8831a828e26 | cirros-0.3.3-x86_64     | qcow2       | bare             | 13200896 | active |
    | 7d9eef7f-15f1-40e3-9e75-593d94a8571d | raw cirros-0.3.3-x86_64 | qcow2       | bare             | 41126400 | active |
    +--------------------------------------+-------------------------+-------------+------------------+----------+--------+

    phoenix:/etc/cinder # rbd ls images
    7d9eef7f-15f1-40e3-9e75-593d94a8571d
    b7fff5f9-d63d-4729-ac1e-b8831a828e26


    phoenix:/etc/cinder # cinder create 1 --display-name vol_fail_test --image-id b7fff5f9-d63d-4729-ac1e-b8831a828e26 --availability-zone nova
    +---------------------+--------------------------------------+
    |       Property      |                Value                 |
    +---------------------+--------------------------------------+
    |     attachments     |                  []                  |
    |  availability_zone  |                 nova                 |
    |       bootable      |                false                 |
    |      created_at     |      2014-10-23T14:11:59.820165      |
    | display_description |                 None                 |
    |     display_name    |            vol_fail_test             |
    |      encrypted      |                False                 |
    |          id         | 52df23c7-b988-4f8c-9754-deab0a349887 |
    |       image_id      | b7fff5f9-d63d-4729-ac1e-b8831a828e26 |
    |       metadata      |                  {}                  |
    |         size        |                  1                   |
    |     snapshot_id     |                 None                 |
    |     source_volid    |                 None                 |
    |        status       |               creating               |
    |     volume_type     |                 None                 |
    +---------------------+--------------------------------------+
    phoenix:/etc/cinder # cinder list
    +--------------------------------------+--------+---------------------+------+-------------+----------+-------------+
    |                  ID                  | Status |     Display Name    | Size | Volume Type | Bootable | Attached to |
    +--------------------------------------+--------+---------------------+------+-------------+----------+-------------+
    | 52df23c7-b988-4f8c-9754-deab0a349887 | error  |    vol_fail_test    |  1   |     None    |  false   |             |
    | 628bf569-7654-4b49-aec5-b02ab49dd522 | error  | cirros-0.3.3-x86_64 |  1   |     None    |  false   |             |
    +--------------------------------------+--------+---------------------+------+-------------+----------+-------------+
    phoenix:/etc/cinder # rbd ls volumes
    volume-52df23c7-b988-4f8c-9754-deab0a349887
    volume-628bf569-7654-4b49-aec5-b02ab49dd522

    grep "^[a-zA-Z[]" cinder.conf

    [DEFAULT]
    verbose = True
    log_dir = /var/log/cinder
    auth_strategy = keystone
    rootwrap_config = /etc/cinder/rootwrap.conf
    state_path = /var/lib/cinder
    lock_path = /var/run/cinder
    rabbit_host=phoenix
    rabbit_userid=guest
    rabbit_password=passw
    rpc_backend=rabbit
    backup_driver=cinder.backup.drivers.ceph
    backup_ceph_conf=/etc/ceph/ceph.conf
    backup_ceph_user=cinder-backup
    backup_ceph_chunk_size=134217728
    backup_ceph_pool=backups
    backup_ceph_stripe_unit=0
    backup_ceph_stripe_count=0
    restore_discard_excess_bytes=true
    my_ip=172.16.98.3
    glance_host=phoenix
    debug=True
    volume_driver = cinder.volume.drivers.rbd.RBDDriver
    rbd_pool = volumes
    glance_api_version = 2
    rbd_ceph_conf = /etc/ceph/ceph.conf
    rbd_user = volumes
    rbd_secret_uuid=df556fc6-8137-4bc0-b54a-1cd9be3df459
    volume_tmp_dir = /tmp
    glance_host = phoenix
    [BRCD_FABRIC_EXAMPLE]
    [CISCO_FABRIC_EXAMPLE]
    [database]
    connection = mysql://cinder:passw@phoenix ...
(more)
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
1

answered 2014-10-29 17:31:32 -0500

Cristi Falcas gravatar image

updated 2014-10-31 08:53:23 -0500

Most probably it's a permission issue in ceph.

ceph user used by cinder need rx rights on the pool where the image is.

 ceph auth caps client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=backups, allow rx pool=images'

Or with your users:

ceph auth get-or-create client.volumes mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rx pool=images' > /etc/ceph/ceph.client.cinder.keyring

ceph auth caps client.volumes          mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rx pool=images'
edit flag offensive delete link more

Comments

client.images key: AQBk4UBU4OWtJxAAyGOE1KWyjuGj/EFlZg2p7A== caps: [mon] allow r caps: [osd] allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=images client.volumes key: AQAz4UBUMIG2CxAAiBRLJX9gHg9FLGxsliHARA=

groen692 gravatar imagegroen692 ( 2014-10-30 02:20:31 -0500 )edit

What rbd user you have in cider and the pool names for glance and cinder

Cristi Falcas gravatar imageCristi Falcas ( 2014-10-30 02:29:46 -0500 )edit

for glance user: images pool: images

for cinder user: volumes pool: volumes

groen692 gravatar imagegroen692 ( 2014-10-30 04:35:51 -0500 )edit

I think they are according to your suggestion.

client.volumes key: AQAz4UBUMIG2CxAAiBRLJX9gHg9FLGxsliHARA==

caps: [mon] allow r

caps: [osd] allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=images

groen692 gravatar imagegroen692 ( 2014-10-31 09:06:52 -0500 )edit
1

answered 2014-10-24 17:41:56 -0500

jtaguinerd gravatar image

Hi, I've come across a bug in Juno where glance-api is broken with ceph. You might want to check this https://bugs.launchpad.net/fuel/+bug/... and apply the patch.

Hope it works for you!

edit flag offensive delete link more

Comments

Thanks for the remark, but as you can see from my glance-api.conf I already added it. Without it the glance api service does not start at all.

I keep searching.

groen692 gravatar imagegroen692 ( 2014-10-25 03:47:39 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2014-10-23 09:34:59 -0500

Seen: 3,680 times

Last updated: Jan 23 '15