Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Icehouse: Cinder and Ceph Volume Creation error

Greetings,

OS: Ubuntu 14.04 OpenStack: Icehouse

I have setup Ceph with Cinder and for some reason, when I attempt to create a volume from a Cinder, it result in an error, while creating a volume from Nova, gets created successfully. For instance:

root@ccinder2:/etc/cinder# cinder create --display-name cinder-ceph-vol1 --display-description "Testing Vol Create" 100

+---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-22T22:01:41.454300 | | display_description | Testing Vol Create | | display_name | cinder-ceph-vol1 | | encrypted | False | | id | b3a5d66a-92e7-4f01-908f-64b3bd5b6592 | | metadata | {} | | size | 100 | | snapshot_id | None | | source_volid | None | | status | creating | | volume_type | None | +---------------------+--------------------------------------+

root@ccinder2:/etc/cinder# cinder list

+--------------------------------------+--------+------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+--------+------------------+------+-------------+----------+-------------+ | b3a5d66a-92e7-4f01-908f-64b3bd5b6592 | error | cinder-ceph-vol1 | 100 | None | false | | +--------------------------------------+--------+------------------+------+-------------+----------+-------------+

.. and now on Nova node, create a test volume:

root@ccomp1:~# nova volume-create --display_name test_rbd_vol 5

+---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-22T23:00:25.377476 | | display_description | - | | display_name | test_rbd_vol | | encrypted | False | | id | 723853f5-60b8-4ee2-806c-694597783cdb | | metadata | {} | | size | 5 | | snapshot_id | - | | source_volid | - | | status | creating | | volume_type | None | +---------------------+--------------------------------------+

List current volumes from nova-compute node:

root@ccomp1:~# nova volume-list

+--------------------------------------+-----------+-----------------+------+-------------+-------------+ | ID | Status | Display Name | Size | Volume Type | Attached to | +--------------------------------------+-----------+-----------------+------+-------------+-------------+ | 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol | 5 | None | | | 5962121d-e7bf-4bf7-81b5-a161a1e81775 | error | cinder-ceph-vol | 100 | None | | +--------------------------------------+-----------+-----------------+------+-------------+-------------+

As you can see, my volume "test_rbd_vol" has created successfully without a problem. Looking through cinder-volume.log I see the following:

2014-06-22 18:57:02.914 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 18:58:02.915 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 18:59:02.917 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 19:00:02.916 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 19:00:25.593 1420 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'tenant': u'0696d4e77c4c494ab653c91df7324ae4', 'user_identity': u'cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -'} 2014-06-22 19:00:25.775 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume 723853f5-60b8-4ee2-806c-694597783cdb: being created using CreateVolumeFromSpecTask._create_raw_volume with specification: {'status': u'creating', 'volume_size': 5, 'volume_name': u'volume-723853f5-60b8-4ee2-806c-694597783cdb'} 2014-06-22 19:00:25.929 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume volume-723853f5-60b8-4ee2-806c-694597783cdb (723853f5-60b8-4ee2-806c-694597783cdb): created successfully

2014-06-22 19:01:02.918 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 19:02:02.920 1420 INFO cinder.volume.manager [-] Updating volume status

There is nothing else thereafter. Here is my /etc/cinder/cinder.conf file:

[DEFAULT] rootwrap_config = /etc/cinder/rootwrap.conf api_paste_confg = /etc/cinder/api-paste.ini iscsi_helper = tgtadm volume_name_template = volume-%s volume_group = cinder-volumes verbose = True auth_strategy = keystone state_path = /var/lib/cinder lock_path = /var/lock/cinder volumes_dir = /var/lib/cinder/volumes # volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_pool=volumes glance_api_version=2 rbd_user=volumes rbd_secret_uuid=fe1776f8-041a-4976-a89c-9d0a611565cf rbd_ceph_conf=/etc/ceph/ceph.conf rbd_flatten_volume_from_snapshots=false rbd_max_clone_depth=5 # rpc_backend = cinder.openstack.common.rpc.impl_kombu rabbit_hosts = 10.10.10.79,10.10.10.80 rabbit_port = 5672 rabbit_use_ssl = false rabbit_userid = guest rabbit_password = guest rabbit_notification_exchange = glance rabbit_notification_topic = notifications rabbit_durable_queues = False glance_host = 10.10.10.91

[database] connection = mysql://cinder:XXXXXXXXXXX@10.10.10.91/cinder

[keystone_authtoken] auth_uri = http://10.10.10.91:5000 auth_host = 10.10.10.91 auth_port = 35357 auth_protocol = http admin_tenant_name = service admin_user = cinder admin_password = XXXXXXXXXXX

backup_driver=cinder.backup.drivers.ceph backup_ceph_conf=/etc/ceph/ceph.conf backup_ceph_user=cinder-backup backup_ceph_chunk_size=134217728 backup_ceph_pool=backups backup_ceph_stripe_unit=0 backup_ceph_stripe_count=0 restore_discard_excess_bytes=true

Here is my /etc/ceph/ceph.conf file on cinder host:

[global] fsid = e51ce43d-c764-449d-a823-38301f11a031 mon_initial_members = cceph1 mon_host = 10.10.10.85,10.10.10.86,10.10.10.87,10.10.10.88 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx filestore_xattr_use_omap = true

Anyone has any ideas?

Thanks a metric ton!

Alex

Icehouse: Cinder and Ceph Volume Creation error

Greetings,

OS: Ubuntu 14.04 OpenStack: Icehouse

I have setup Ceph with Cinder and for some reason, when I attempt to create a volume from a Cinder, it result in an error, while creating a volume from Nova, gets created successfully. For instance:

root@ccinder2:/etc/cinder# cinder **cinder create --display-name cinder-ceph-vol1 --display-description "Testing Vol Create" 100

100** +---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-22T22:01:41.454300 | | display_description | Testing Vol Create | | display_name | cinder-ceph-vol1 | | encrypted | False | | id | b3a5d66a-92e7-4f01-908f-64b3bd5b6592 | | metadata | {} | | size | 100 | | snapshot_id | None | | source_volid | None | | status | creating | | volume_type | None | +---------------------+--------------------------------------+

+---------------------+--------------------------------------+ root@ccinder2:/etc/cinder# cinder list

**cinder list** +--------------------------------------+--------+------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+--------+------------------+------+-------------+----------+-------------+ | b3a5d66a-92e7-4f01-908f-64b3bd5b6592 | error | cinder-ceph-vol1 | 100 | None | false | | +--------------------------------------+--------+------------------+------+-------------+----------+-------------+

+--------------------------------------+--------+------------------+------+-------------+----------+-------------+ .. and now on Nova node, create a test volume:

volume: root@ccomp1:~# nova **nova volume-create --display_name test_rbd_vol 5

5** +---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-22T23:00:25.377476 | | display_description | - | | display_name | test_rbd_vol | | encrypted | False | | id | 723853f5-60b8-4ee2-806c-694597783cdb | | metadata | {} | | size | 5 | | snapshot_id | - | | source_volid | - | | status | creating | | volume_type | None | +---------------------+--------------------------------------+

+---------------------+--------------------------------------+ List current volumes from nova-compute node:

node: root@ccomp1:~# nova volume-list

**nova volume-list** +--------------------------------------+-----------+-----------------+------+-------------+-------------+ | ID | Status | Display Name | Size | Volume Type | Attached to | +--------------------------------------+-----------+-----------------+------+-------------+-------------+ | 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol | 5 | None | | | 5962121d-e7bf-4bf7-81b5-a161a1e81775 | error | cinder-ceph-vol | 100 | None | | +--------------------------------------+-----------+-----------------+------+-------------+-------------+

+--------------------------------------+-----------+-----------------+------+-------------+-------------+ As you can see, my volume "test_rbd_vol" has created successfully without a problem. Looking through cinder-volume.log I see the following:

following: 2014-06-22 18:57:02.914 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 18:58:02.915 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 18:59:02.917 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 19:00:02.916 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 19:00:25.593 1420 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'tenant': u'0696d4e77c4c494ab653c91df7324ae4', 'user_identity': u'cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -'} 2014-06-22 19:00:25.775 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume 723853f5-60b8-4ee2-806c-694597783cdb: being created using CreateVolumeFromSpecTask._create_raw_volume with specification: {'status': u'creating', 'volume_size': 5, 'volume_name': u'volume-723853f5-60b8-4ee2-806c-694597783cdb'} 2014-06-22 19:00:25.929 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume volume-723853f5-60b8-4ee2-806c-694597783cdb (723853f5-60b8-4ee2-806c-694597783cdb): created successfully

(**723853f5-60b8-4ee2-806c-694597783cdb**): **created successfully** 2014-06-22 19:01:02.918 1420 INFO cinder.volume.manager [-] Updating volume status 2014-06-22 19:02:02.920 1420 INFO cinder.volume.manager [-] Updating volume status

status There is nothing else thereafter. Here is my /etc/cinder/cinder.conf file:

**/etc/cinder/cinder.conf** file: [DEFAULT] rootwrap_config = /etc/cinder/rootwrap.conf api_paste_confg = /etc/cinder/api-paste.ini iscsi_helper = tgtadm volume_name_template = volume-%s volume_group = cinder-volumes verbose = True auth_strategy = keystone state_path = /var/lib/cinder lock_path = /var/lock/cinder volumes_dir = /var/lib/cinder/volumes # volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_pool=volumes glance_api_version=2 rbd_user=volumes rbd_secret_uuid=fe1776f8-041a-4976-a89c-9d0a611565cf rbd_ceph_conf=/etc/ceph/ceph.conf rbd_flatten_volume_from_snapshots=false rbd_max_clone_depth=5 # rpc_backend = cinder.openstack.common.rpc.impl_kombu rabbit_hosts = 10.10.10.79,10.10.10.80 rabbit_port = 5672 rabbit_use_ssl = false rabbit_userid = guest rabbit_password = guest rabbit_notification_exchange = glance rabbit_notification_topic = notifications rabbit_durable_queues = False glance_host = 10.10.10.91

10.10.10.91 [database] connection = mysql://cinder:XXXXXXXXXXX@10.10.10.91/cinder

mysql://cinder:XXXXXXXXXXX@10.10.10.91/cinder [keystone_authtoken] auth_uri = http://10.10.10.91:5000 auth_host = 10.10.10.91 auth_port = 35357 auth_protocol = http admin_tenant_name = service admin_user = cinder admin_password = XXXXXXXXXXX

XXXXXXXXXXX backup_driver=cinder.backup.drivers.ceph backup_ceph_conf=/etc/ceph/ceph.conf backup_ceph_user=cinder-backup backup_ceph_chunk_size=134217728 backup_ceph_pool=backups backup_ceph_stripe_unit=0 backup_ceph_stripe_count=0 restore_discard_excess_bytes=true

restore_discard_excess_bytes=true Here is my /etc/ceph/ceph.conf **/etc/ceph/ceph.conf** file on cinder host:

host: [global] fsid = e51ce43d-c764-449d-a823-38301f11a031 mon_initial_members = cceph1 mon_host = 10.10.10.85,10.10.10.86,10.10.10.87,10.10.10.88 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx filestore_xattr_use_omap = true

true Anyone has any ideas?

Thanks a metric ton!

Alex

Alex

Icehouse: Cinder and Ceph Volume Creation error

Greetings,

OS: Ubuntu 14.04 OpenStack: Icehouse

I have setup Ceph with Cinder and for some reason, when I attempt to create a volume from a Cinder, it result in an error, while creating a volume from Nova, gets created successfully. For instance:

root@ccinder2:/etc/cinder# **cinder create --display-name cinder-ceph-vol1 --display-description "Testing Vol Create" 100**

+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2014-06-22T22:01:41.454300      |
| display_description |          Testing Vol Create          |
|     display_name    |           cinder-ceph-vol1           |
|      encrypted      |                False                 |
|          id         | b3a5d66a-92e7-4f01-908f-64b3bd5b6592 |
|       metadata      |                  {}                  |
|         size        |                 100                  |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+---------------------+--------------------------------------+

root@ccinder2:/etc/cinder# **cinder list**

+--------------------------------------+--------+------------------+------+-------------+----------+-------------+
|                  ID                  | Status |   Display Name   | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+------------------+------+-------------+----------+-------------+
| b3a5d66a-92e7-4f01-908f-64b3bd5b6592 | error  | cinder-ceph-vol1 | 100  |     None    |  false   |             |
+--------------------------------------+--------+------------------+------+-------------+----------+-------------+

.. and now on Nova node, create a test volume:

root@ccomp1:~# **nova volume-create --display_name test_rbd_vol 5**

+---------------------+--------------------------------------+
| Property            | Value                                |
+---------------------+--------------------------------------+
| attachments         | []                                   |
| availability_zone   | nova                                 |
| bootable            | false                                |
| created_at          | 2014-06-22T23:00:25.377476           |
| display_description | -                                    |
| display_name        | test_rbd_vol                         |
| encrypted           | False                                |
| id                  | 723853f5-60b8-4ee2-806c-694597783cdb |
| metadata            | {}                                   |
| size                | 5                                    |
| snapshot_id         | -                                    |
| source_volid        | -                                    |
| status              | creating                             |
| volume_type         | None                                 |
+---------------------+--------------------------------------+

List  current volumes from nova-compute node:

root@ccomp1:~# **nova volume-list**

+--------------------------------------+-----------+-----------------+------+-------------+-------------+
| ID                                   | Status    | Display Name    | Size | Volume Type | Attached to |
+--------------------------------------+-----------+-----------------+------+-------------+-------------+
| 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol    | 5    | None        |            |
| 5962121d-e7bf-4bf7-81b5-a161a1e81775 | error     | cinder-ceph-vol | 100  | None        |             |
+--------------------------------------+-----------+-----------------+------+-------------+-------------+

As you can see, my volume "test_rbd_vol" has created successfully without a problem. Looking through cinder-volume.log I see the following:

2014-06-22 18:57:02.914 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 18:58:02.915 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 18:59:02.917 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 19:00:02.916 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 19:00:25.593 1420 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'tenant': u'0696d4e77c4c494ab653c91df7324ae4', 'user_identity': u'cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -'}
2014-06-22 19:00:25.775 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume 723853f5-60b8-4ee2-806c-694597783cdb: being created using CreateVolumeFromSpecTask._create_raw_volume with specification: {'status': u'creating', 'volume_size': 5, 'volume_name': u'volume-723853f5-60b8-4ee2-806c-694597783cdb'}
2014-06-22 19:00:25.929 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume volume-723853f5-60b8-4ee2-806c-694597783cdb (**723853f5-60b8-4ee2-806c-694597783cdb**): **created successfully**

2014-06-22 19:01:02.918 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 19:02:02.920 1420 INFO cinder.volume.manager [-] Updating volume status

There is nothing else thereafter. Here is my **/etc/cinder/cinder.conf** file:

[DEFAULT]
rootwrap_config = /etc/cinder/rootwrap.conf
api_paste_confg = /etc/cinder/api-paste.ini
iscsi_helper = tgtadm
volume_name_template = volume-%s
volume_group = cinder-volumes
verbose = True
auth_strategy = keystone
state_path = /var/lib/cinder
lock_path = /var/lock/cinder
volumes_dir = /var/lib/cinder/volumes
#
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_pool=volumes
glance_api_version=2
rbd_user=volumes
rbd_secret_uuid=fe1776f8-041a-4976-a89c-9d0a611565cf
rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshots=false
rbd_max_clone_depth=5
#
rpc_backend = cinder.openstack.common.rpc.impl_kombu
rabbit_hosts = 10.10.10.79,10.10.10.80
rabbit_port = 5672
rabbit_use_ssl = false
rabbit_userid = guest
rabbit_password = guest
rabbit_notification_exchange = glance
rabbit_notification_topic = notifications
rabbit_durable_queues = False
glance_host = 10.10.10.91

[database]
connection = mysql://cinder:XXXXXXXXXXX@10.10.10.91/cinder

[keystone_authtoken]
auth_uri = http://10.10.10.91:5000
auth_host = 10.10.10.91
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = cinder
admin_password = XXXXXXXXXXX

backup_driver=cinder.backup.drivers.ceph
backup_ceph_conf=/etc/ceph/ceph.conf
backup_ceph_user=cinder-backup
backup_ceph_chunk_size=134217728
backup_ceph_pool=backups
backup_ceph_stripe_unit=0
backup_ceph_stripe_count=0
restore_discard_excess_bytes=true

Here is my **/etc/ceph/ceph.conf** file on cinder host:

[global]
fsid = e51ce43d-c764-449d-a823-38301f11a031
mon_initial_members = cceph1
mon_host = 10.10.10.85,10.10.10.86,10.10.10.87,10.10.10.88
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true

Anyone has any ideas? 

Thanks a metric ton! 

Alex
#

Here are the steps I took to re-create the error:

  1. Create volume under cinder:

root@ccinder2:/var/log/cinder# cinder create --display-name cinder-ceph-vol1 --display-description "Testing Test Vol1" 100 +---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-23T21:36:07.398955 | | display_description | Testing Test Vol1 | | display_name | cinder-ceph-vol1 | | encrypted | False | | id | 70084540-0314-4465-9706-6e59f871b3a8 | | metadata | {} | | size | 100 | | snapshot_id | None | | source_volid | None | | status | creating | | volume_type | None | +---------------------+--------------------------------------+

List cinder volumes:

root@ccinder2:/var/log/cinder# cinder list +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | 70084540-0314-4465-9706-6e59f871b3a8 | error | cinder-ceph-vol1 | 100 | None | false | | | 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol | 5 | None | false | | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+

Looking through errors on /var/log/cinder/cinder-volume.log and /var/log/cinder/cinder-api.log:

root@ccinder2:/var/log/cinder# grep b3a8 cinder-volume.log

Nothing under cinder-volume.log

Here are the some entries under cinder-api.log:

root@ccinder2:/var/log/cinder# grep b3a8 cinder-api.log 2014-06-23 17:36:07.585 1434 AUDIT cinder.api.v1.volumes [req-8b439d6c-8a32-4ed8-b54d-525b0f69718b cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] vol={'migration_status': None, 'availability_zone': 'nova', 'terminated_at': None, 'reservations': ['1c521c7c-ab40-4f25-a492-a84b8e43d615', '07084df8-80c6-4a9a-992b-c7796fb2cf35'], 'updated_at': None, 'provider_geometry': None, 'snapshot_id': None, 'ec2_id': None, 'mountpoint': None, 'deleted_at': None, 'id': '70084540-0314-4465-9706-6e59f871b3a8', 'size': 100, 'user_id': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'attach_time': None, 'attached_host': None, 'display_description': u'Testing Test Vol1', 'volume_admin_metadata': [], 'encryption_key_id': None, 'project_id': u'0696d4e77c4c494ab653c91df7324ae4', 'launched_at': None, 'scheduled_at': None, 'status': 'creating', 'volume_type_id': None, 'deleted': False, 'provider_location': None, 'host': None, 'source_volid': None, 'provider_auth': None, 'display_name': u'cinder-ceph-vol1', 'instance_uuid': None, 'bootable': False, 'created_at': datetime.datetime(2014, 6, 23, 21, 36, 7, 398955), 'attach_status': 'detached', 'volume_type': None, '_name_id': None, 'volume_metadata': [], 'metadata': {}} 2014-06-23 17:36:15.035 1434 AUDIT cinder.api.v1.volumes [req-ad1e2bba-8bdc-4597-a9d5-97b94a65af0e cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] vol={'migration_status': None, 'availability_zone': u'nova', 'terminated_at': None, 'updated_at': datetime.datetime(2014, 6, 23, 21, 36, 7), 'provider_geometry': None, 'snapshot_id': None, 'ec2_id': None, 'mountpoint': None, 'deleted_at': None, 'id': u'70084540-0314-4465-9706-6e59f871b3a8', 'size': 100L, 'user_id': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'attach_time': None, 'attached_host': None, 'display_description': u'Testing Test Vol1', 'volume_admin_metadata': [], 'encryption_key_id': None, 'project_id': u'0696d4e77c4c494ab653c91df7324ae4', 'launched_at': None, 'scheduled_at': None, 'status': u'error', 'volume_type_id': None, 'deleted': False, 'provider_location': None, 'host': None, 'source_volid': None, 'provider_auth': None, 'display_name': u'cinder-ceph-vol1', 'instance_uuid': None, 'bootable': False, 'created_at': datetime.datetime(2014, 6, 23, 21, 36, 7), 'attach_status': u'detached', 'volume_type': None, '_name_id': None, 'volume_metadata': []}

Thanks again!!!

Icehouse: Cinder and Ceph Volume Creation error

Greetings,

OS: Ubuntu 14.04 OpenStack: Icehouse

I have setup Ceph with Cinder and for some reason, when I attempt to create a volume from a Cinder, it result in an error, while creating a volume from Nova, gets created successfully. For instance:

root@ccinder2:/etc/cinder# **cinder create --display-name cinder-ceph-vol1 --display-description "Testing Vol Create" 100**

+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2014-06-22T22:01:41.454300      |
| display_description |          Testing Vol Create          |
|     display_name    |           cinder-ceph-vol1           |
|      encrypted      |                False                 |
|          id         | b3a5d66a-92e7-4f01-908f-64b3bd5b6592 |
|       metadata      |                  {}                  |
|         size        |                 100                  |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+---------------------+--------------------------------------+

root@ccinder2:/etc/cinder# **cinder list**

+--------------------------------------+--------+------------------+------+-------------+----------+-------------+
|                  ID                  | Status |   Display Name   | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+--------+------------------+------+-------------+----------+-------------+
| b3a5d66a-92e7-4f01-908f-64b3bd5b6592 | error  | cinder-ceph-vol1 | 100  |     None    |  false   |             |
+--------------------------------------+--------+------------------+------+-------------+----------+-------------+

.. and now on Nova node, create a test volume:

root@ccomp1:~# **nova volume-create --display_name test_rbd_vol 5**

+---------------------+--------------------------------------+
| Property            | Value                                |
+---------------------+--------------------------------------+
| attachments         | []                                   |
| availability_zone   | nova                                 |
| bootable            | false                                |
| created_at          | 2014-06-22T23:00:25.377476           |
| display_description | -                                    |
| display_name        | test_rbd_vol                         |
| encrypted           | False                                |
| id                  | 723853f5-60b8-4ee2-806c-694597783cdb |
| metadata            | {}                                   |
| size                | 5                                    |
| snapshot_id         | -                                    |
| source_volid        | -                                    |
| status              | creating                             |
| volume_type         | None                                 |
+---------------------+--------------------------------------+

List  current volumes from nova-compute node:

root@ccomp1:~# **nova volume-list**

+--------------------------------------+-----------+-----------------+------+-------------+-------------+
| ID                                   | Status    | Display Name    | Size | Volume Type | Attached to |
+--------------------------------------+-----------+-----------------+------+-------------+-------------+
| 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol    | 5    | None        |            |
| 5962121d-e7bf-4bf7-81b5-a161a1e81775 | error     | cinder-ceph-vol | 100  | None        |             |
+--------------------------------------+-----------+-----------------+------+-------------+-------------+

As you can see, my volume "test_rbd_vol" has created successfully without a problem. Looking through cinder-volume.log I see the following:

2014-06-22 18:57:02.914 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 18:58:02.915 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 18:59:02.917 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 19:00:02.916 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 19:00:25.593 1420 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'tenant': u'0696d4e77c4c494ab653c91df7324ae4', 'user_identity': u'cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -'}
2014-06-22 19:00:25.775 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume 723853f5-60b8-4ee2-806c-694597783cdb: being created using CreateVolumeFromSpecTask._create_raw_volume with specification: {'status': u'creating', 'volume_size': 5, 'volume_name': u'volume-723853f5-60b8-4ee2-806c-694597783cdb'}
2014-06-22 19:00:25.929 1420 INFO cinder.volume.flows.manager.create_volume [req-44841245-b46e-4e72-8dd3-066bf35f4ee8 cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] Volume volume-723853f5-60b8-4ee2-806c-694597783cdb (**723853f5-60b8-4ee2-806c-694597783cdb**): **created successfully**

2014-06-22 19:01:02.918 1420 INFO cinder.volume.manager [-] Updating volume status
2014-06-22 19:02:02.920 1420 INFO cinder.volume.manager [-] Updating volume status

There is nothing else thereafter. Here is my **/etc/cinder/cinder.conf** file:

[DEFAULT]
rootwrap_config = /etc/cinder/rootwrap.conf
api_paste_confg = /etc/cinder/api-paste.ini
iscsi_helper = tgtadm
volume_name_template = volume-%s
volume_group = cinder-volumes
verbose = True
auth_strategy = keystone
state_path = /var/lib/cinder
lock_path = /var/lock/cinder
volumes_dir = /var/lib/cinder/volumes
#
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_pool=volumes
glance_api_version=2
rbd_user=volumes
rbd_secret_uuid=fe1776f8-041a-4976-a89c-9d0a611565cf
rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshots=false
rbd_max_clone_depth=5
#
rpc_backend = cinder.openstack.common.rpc.impl_kombu
rabbit_hosts = 10.10.10.79,10.10.10.80
rabbit_port = 5672
rabbit_use_ssl = false
rabbit_userid = guest
rabbit_password = guest
rabbit_notification_exchange = glance
rabbit_notification_topic = notifications
rabbit_durable_queues = False
glance_host = 10.10.10.91

[database]
connection = mysql://cinder:XXXXXXXXXXX@10.10.10.91/cinder

[keystone_authtoken]
auth_uri = http://10.10.10.91:5000
auth_host = 10.10.10.91
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = cinder
admin_password = XXXXXXXXXXX

backup_driver=cinder.backup.drivers.ceph
backup_ceph_conf=/etc/ceph/ceph.conf
backup_ceph_user=cinder-backup
backup_ceph_chunk_size=134217728
backup_ceph_pool=backups
backup_ceph_stripe_unit=0
backup_ceph_stripe_count=0
restore_discard_excess_bytes=true

Here is my **/etc/ceph/ceph.conf** file on cinder host:

[global]
fsid = e51ce43d-c764-449d-a823-38301f11a031
mon_initial_members = cceph1
mon_host = 10.10.10.85,10.10.10.86,10.10.10.87,10.10.10.88
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true

Anyone has any ideas? 

Thanks a metric ton! 

Alex
#

Here are the steps I took to re-create the error:

  1. Create volume under cinder:

root@ccinder2:/var/log/cinder# cinder create --display-name cinder-ceph-vol1 --display-description "Testing Test Vol1" 100 +---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-23T21:36:07.398955 | | display_description | Testing Test Vol1 | | display_name | cinder-ceph-vol1 | | encrypted | False | | id | 70084540-0314-4465-9706-6e59f871b3a8 | | metadata | {} | | size | 100 | | snapshot_id | None | | source_volid | None | | status | creating | | volume_type | None | +---------------------+--------------------------------------+

List cinder volumes:

root@ccinder2:/var/log/cinder# cinder list +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | 70084540-0314-4465-9706-6e59f871b3a8 | error | cinder-ceph-vol1 | 100 | None | false | | | 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol | 5 | None | false | | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+

Looking through errors on /var/log/cinder/cinder-volume.log and /var/log/cinder/cinder-api.log:

root@ccinder2:/var/log/cinder# grep b3a8 cinder-volume.log

Nothing under cinder-volume.log

Here are the some entries under cinder-api.log:

root@ccinder2:/var/log/cinder# grep b3a8 cinder-api.log 2014-06-23 17:36:07.585 1434 AUDIT cinder.api.v1.volumes [req-8b439d6c-8a32-4ed8-b54d-525b0f69718b cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] vol={'migration_status': None, 'availability_zone': 'nova', 'terminated_at': None, 'reservations': ['1c521c7c-ab40-4f25-a492-a84b8e43d615', '07084df8-80c6-4a9a-992b-c7796fb2cf35'], 'updated_at': None, 'provider_geometry': None, 'snapshot_id': None, 'ec2_id': None, 'mountpoint': None, 'deleted_at': None, 'id': '70084540-0314-4465-9706-6e59f871b3a8', 'size': 100, 'user_id': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'attach_time': None, 'attached_host': None, 'display_description': u'Testing Test Vol1', 'volume_admin_metadata': [], 'encryption_key_id': None, 'project_id': u'0696d4e77c4c494ab653c91df7324ae4', 'launched_at': None, 'scheduled_at': None, 'status': 'creating', 'volume_type_id': None, 'deleted': False, 'provider_location': None, 'host': None, 'source_volid': None, 'provider_auth': None, 'display_name': u'cinder-ceph-vol1', 'instance_uuid': None, 'bootable': False, 'created_at': datetime.datetime(2014, 6, 23, 21, 36, 7, 398955), 'attach_status': 'detached', 'volume_type': None, '_name_id': None, 'volume_metadata': [], 'metadata': {}} 2014-06-23 17:36:15.035 1434 AUDIT cinder.api.v1.volumes [req-ad1e2bba-8bdc-4597-a9d5-97b94a65af0e cd72fa7e6af44bc5a5b7c3bc7473c367 0696d4e77c4c494ab653c91df7324ae4 - - -] vol={'migration_status': None, 'availability_zone': u'nova', 'terminated_at': None, 'updated_at': datetime.datetime(2014, 6, 23, 21, 36, 7), 'provider_geometry': None, 'snapshot_id': None, 'ec2_id': None, 'mountpoint': None, 'deleted_at': None, 'id': u'70084540-0314-4465-9706-6e59f871b3a8', 'size': 100L, 'user_id': u'cd72fa7e6af44bc5a5b7c3bc7473c367', 'attach_time': None, 'attached_host': None, 'display_description': u'Testing Test Vol1', 'volume_admin_metadata': [], 'encryption_key_id': None, 'project_id': u'0696d4e77c4c494ab653c91df7324ae4', 'launched_at': None, 'scheduled_at': None, 'status': u'error', 'volume_type_id': None, 'deleted': False, 'provider_location': None, 'host': None, 'source_volid': None, 'provider_auth': None, 'display_name': u'cinder-ceph-vol1', 'instance_uuid': None, 'bootable': False, 'created_at': datetime.datetime(2014, 6, 23, 21, 36, 7), 'attach_status': u'detached', 'volume_type': None, '_name_id': None, 'volume_metadata': []}

#

Here I am trying to re-create cinder volume create on Ceph with a debug on

#

root@ccinder2:~# cinder --debug create --display-name cinder-ceph-vol3 --display-description "Test2" 100

REQ: curl -i http://10.10.10.91:35357/v2.0/tokens -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-cinderclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "XXXXXXXX "}}}'

DEBUG:cinderclient.client: REQ: curl -i http://10.10.10.91:35357/v2.0/tokens -X POST -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-cinderclient" -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "XXXXXXXX "}}}'

RESP: [200] CaseInsensitiveDict({'date': 'Mon, 23 Jun 2014 22:19:48 GMT', 'vary': 'X-Auth-Token', 'content-length': '6223', 'content-type': 'application/json', 'x-distribution': 'Ubuntu'}) RESP BODY: {"access": {"token": {"issued_at": "2014-06-23T22:19:48.036848", "expires": "2014-06-23T23:19:48Z", "id": "MIILLwYJKoZIhvcNAQcCoIILIDCCCxwCAQExDTALBglghkgBZQMEAgEwggl9BgkqhkiG9w0BBwGgggluBIIJansiYWNjZXNzIjogeyJ0b2tlbiI6IHsiaXNzdWVkX2F0IjogIjIwMTQtMDYtMjNUMjI6MTk6NDguMDM2ODQ4IiwgImV4cGlyZXMiOiAiMjAxNC0wNi0yM1QyMzoxOTo0OFoiLCAiaWQiOiAicGxhY2Vob2xkZXIiLCAidGVuYW50IjogeyJkZXNjcmlwdGlvbiI6IG51bGwsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogIjA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgInJlZ2lvbiI6ICJyZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjg3NzQvdjIvMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiM2NjMzg0ZmUwNjJkNDk0Y2E5MWVjNjExZTkzZmNkZGMiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0In1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImNvbXB1dGUiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6OTI5MiIsICJpZCI6ICIwMTRmYjMzYmFiNWE0MjQyODJmNGFjYWM3OTBiMjJmOCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaW1hZ2UiLCAibmFtZSI6ICJnbGFuY2UifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc2L3YxLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgImlkIjogIjViNjUzNTMwNGVkMjQxMjdhZDcwNDg4MWQ3NmJkZGUxIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWUiLCAibmFtZSI6ICJjaW5kZXIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4NzczL3NlcnZpY2VzL0Nsb3VkIiwgImlkIjogIjExYjA2ZmE3ZDg3YjQ1MTBiNGE1ODM1MmYxODI5NGQ3IiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9DbG91ZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJlYzIiLCAibmFtZSI6ICJlYzIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODA4MCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiODIwZDg5Mjc2YzMzNDY3MzhhYTkzZjFkNWJmZjk1MTIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAib2JqZWN0LXN0b3JlIiwgIm5hbWUiOiAic3dpZnQifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAiLCAiaWQiOiAiNGY4YzI0YjRkZGYwNGVhYTg4MWE5OTE1Y2IzZjllNWQiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaWRlbnRpdHkiLCAibmFtZSI6ICJrZXlzdG9uZSJ9XSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICJjZDcyZmE3ZTZhZjQ0YmM1YTViN2MzYmM3NDczYzM2NyIsICJyb2xlcyI6IFt7Im5hbWUiOiAiYWRtaW4ifV0sICJuYW1lIjogImFkbWluIn0sICJtZXRhZGF0YSI6IHsiaXNfYWRtaW4iOiAwLCAicm9sZXMiOiBbImI4MTQ1Y2UxYzFhNTRkMzA5N2NjNzA1YWU4Y2RlZGIzIl19fX0xggGFMIIBgQIBATBcMFcxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVVbnNldDEOMAwGA1UEBwwFVW5zZXQxDjAMBgNVBAoMBVVuc2V0MRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20CAQEwCwYJYIZIAWUDBAIBMA0GCSqGSIb3DQEBAQUABIIBAGd9w5+HzIH4F5fwaLbFgoT+IYuDdc-ra1pIiYd+37-cC6hAM+PrEEh6Z+0MhtV2o3h-Uw4H1jz3t918YdRioxZqDpeJlBBaD8oEK2AF0QKhwmljRomFYmZsArhSG5jiBOIRUF4JVH48AuNVxJq1ZS1gQvU6b9TQdMe1Z2T1EmePCnztcLQYfrXUv2wWICr8v50y2MbDX5RkgHfEF1tE0wieVu9hJY32s7OlSb20UfT0T8R-gvw9nvZIrQ5K+Qxgb8BwD81tphhDoGIgjuVhMHi4PgNOTQkhXV4oK285Sw9zS3HVL5+BFuBf9oHHQcrEgWH9yuF8N9XKu4aiMyJwZnw=", "tenant": {"description": null, "enabled": true, "id": "0696d4e77c4c494ab653c91df7324ae4", "name": "admin"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.10.10.91:8774/v2/0696d4e77c4c494ab653c91df7324ae4", "region": "regionOne", "internalURL": "http://10.10.10.91:8774/v2/0696d4e77c4c494ab653c91df7324ae4", "id": "3cc384fe062d494ca91ec611e93fcddc", "publicURL": "http://10.10.10.91:8774/v2/0696d4e77c4c494ab653c91df7324ae4"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.10.10.91:9292", "region": "regionOne", "internalURL": "http://10.10.10.91:9292", "id": "014fb33bab5a424282f4acac790b22f8", "publicURL": "http://10.10.10.91:9292"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4", "region": "regionOne", "internalURL": "http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4", "id": "5b6535304ed24127ad704881d76bdde1", "publicURL": "http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4"}], "endpoints_links": [], "type": "volume", "name": "cinder"}, {"endpoints": [{"adminURL": "http://10.10.10.91:8773/services/Admin", "region": "regionOne", "internalURL": "http://10.10.10.91:8773/services/Cloud", "id": "11b06fa7d87b4510b4a58352f18294d7", "publicURL": "http://10.10.10.91:8773/services/Cloud"}], "endpoints_links": [], "type": "ec2", "name": "ec2"}, {"endpoints": [{"adminURL": "http://10.10.10.91:8080", "region": "regionOne", "internalURL": "http://10.10.10.91:8080/v1/AUTH_0696d4e77c4c494ab653c91df7324ae4", "id": "820d89276c3346738aa93f1d5bff9512", "publicURL": "http://10.10.10.91:8080/v1/AUTH_0696d4e77c4c494ab653c91df7324ae4"}], "endpoints_links": [], "type": "object-store", "name": "swift"}, {"endpoints": [{"adminURL": "http://10.10.10.91:35357/v2.0", "region": "regionOne", "internalURL": "http://10.10.10.91:5000/v2.0", "id": "4f8c24b4ddf04eaa881a9915cb3f9e5d", "publicURL": "http://10.10.10.91:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "admin", "roles_links": [], "id": "cd72fa7e6af44bc5a5b7c3bc7473c367", "roles": [{"name": "admin"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles": ["b8145ce1c1a54d3097cc705ae8cdedb3"]}}}

DEBUG:cinderclient.client:RESP: [200] CaseInsensitiveDict({'date': 'Mon, 23 Jun 2014 22:19:48 GMT', 'vary': 'X-Auth-Token', 'content-length': '6223', 'content-type': 'application/json', 'x-distribution': 'Ubuntu'}) RESP BODY: {"access": {"token": {"issued_at": "2014-06-23T22:19:48.036848", "expires": "2014-06-23T23:19:48Z", "id": "MIILLwYJKoZIhvcNAQcCoIILIDCCCxwCAQExDTALBglghkgBZQMEAgEwggl9BgkqhkiG9w0BBwGgggluBIIJansiYWNjZXNzIjogeyJ0b2tlbiI6IHsiaXNzdWVkX2F0IjogIjIwMTQtMDYtMjNUMjI6MTk6NDguMDM2ODQ4IiwgImV4cGlyZXMiOiAiMjAxNC0wNi0yM1QyMzoxOTo0OFoiLCAiaWQiOiAicGxhY2Vob2xkZXIiLCAidGVuYW50IjogeyJkZXNjcmlwdGlvbiI6IG51bGwsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogIjA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgInJlZ2lvbiI6ICJyZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjg3NzQvdjIvMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiM2NjMzg0ZmUwNjJkNDk0Y2E5MWVjNjExZTkzZmNkZGMiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0In1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImNvbXB1dGUiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6OTI5MiIsICJpZCI6ICIwMTRmYjMzYmFiNWE0MjQyODJmNGFjYWM3OTBiMjJmOCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaW1hZ2UiLCAibmFtZSI6ICJnbGFuY2UifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc2L3YxLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgImlkIjogIjViNjUzNTMwNGVkMjQxMjdhZDcwNDg4MWQ3NmJkZGUxIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWUiLCAibmFtZSI6ICJjaW5kZXIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4NzczL3NlcnZpY2VzL0Nsb3VkIiwgImlkIjogIjExYjA2ZmE3ZDg3YjQ1MTBiNGE1ODM1MmYxODI5NGQ3IiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9DbG91ZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJlYzIiLCAibmFtZSI6ICJlYzIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODA4MCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiODIwZDg5Mjc2YzMzNDY3MzhhYTkzZjFkNWJmZjk1MTIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAib2JqZWN0LXN0b3JlIiwgIm5hbWUiOiAic3dpZnQifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAiLCAiaWQiOiAiNGY4YzI0YjRkZGYwNGVhYTg4MWE5OTE1Y2IzZjllNWQiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaWRlbnRpdHkiLCAibmFtZSI6ICJrZXlzdG9uZSJ9XSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICJjZDcyZmE3ZTZhZjQ0YmM1YTViN2MzYmM3NDczYzM2NyIsICJyb2xlcyI6IFt7Im5hbWUiOiAiYWRtaW4ifV0sICJuYW1lIjogImFkbWluIn0sICJtZXRhZGF0YSI6IHsiaXNfYWRtaW4iOiAwLCAicm9sZXMiOiBbImI4MTQ1Y2UxYzFhNTRkMzA5N2NjNzA1YWU4Y2RlZGIzIl19fX0xggGFMIIBgQIBATBcMFcxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVVbnNldDEOMAwGA1UEBwwFVW5zZXQxDjAMBgNVBAoMBVVuc2V0MRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20CAQEwCwYJYIZIAWUDBAIBMA0GCSqGSIb3DQEBAQUABIIBAGd9w5+HzIH4F5fwaLbFgoT+IYuDdc-ra1pIiYd+37-cC6hAM+PrEEh6Z+0MhtV2o3h-Uw4H1jz3t918YdRioxZqDpeJlBBaD8oEK2AF0QKhwmljRomFYmZsArhSG5jiBOIRUF4JVH48AuNVxJq1ZS1gQvU6b9TQdMe1Z2T1EmePCnztcLQYfrXUv2wWICr8v50y2MbDX5RkgHfEF1tE0wieVu9hJY32s7OlSb20UfT0T8R-gvw9nvZIrQ5K+Qxgb8BwD81tphhDoGIgjuVhMHi4PgNOTQkhXV4oK285Sw9zS3HVL5+BFuBf9oHHQcrEgWH9yuF8N9XKu4aiMyJwZnw=", "tenant": {"description": null, "enabled": true, "id": "0696d4e77c4c494ab653c91df7324ae4", "name": "admin"}}, "serviceCatalog": [{"endpoints": [{"adminURL": "http://10.10.10.91:8774/v2/0696d4e77c4c494ab653c91df7324ae4", "region": "regionOne", "internalURL": "http://10.10.10.91:8774/v2/0696d4e77c4c494ab653c91df7324ae4", "id": "3cc384fe062d494ca91ec611e93fcddc", "publicURL": "http://10.10.10.91:8774/v2/0696d4e77c4c494ab653c91df7324ae4"}], "endpoints_links": [], "type": "compute", "name": "nova"}, {"endpoints": [{"adminURL": "http://10.10.10.91:9292", "region": "regionOne", "internalURL": "http://10.10.10.91:9292", "id": "014fb33bab5a424282f4acac790b22f8", "publicURL": "http://10.10.10.91:9292"}], "endpoints_links": [], "type": "image", "name": "glance"}, {"endpoints": [{"adminURL": "http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4", "region": "regionOne", "internalURL": "http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4", "id": "5b6535304ed24127ad704881d76bdde1", "publicURL": "http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4"}], "endpoints_links": [], "type": "volume", "name": "cinder"}, {"endpoints": [{"adminURL": "http://10.10.10.91:8773/services/Admin", "region": "regionOne", "internalURL": "http://10.10.10.91:8773/services/Cloud", "id": "11b06fa7d87b4510b4a58352f18294d7", "publicURL": "http://10.10.10.91:8773/services/Cloud"}], "endpoints_links": [], "type": "ec2", "name": "ec2"}, {"endpoints": [{"adminURL": "http://10.10.10.91:8080", "region": "regionOne", "internalURL": "http://10.10.10.91:8080/v1/AUTH_0696d4e77c4c494ab653c91df7324ae4", "id": "820d89276c3346738aa93f1d5bff9512", "publicURL": "http://10.10.10.91:8080/v1/AUTH_0696d4e77c4c494ab653c91df7324ae4"}], "endpoints_links": [], "type": "object-store", "name": "swift"}, {"endpoints": [{"adminURL": "http://10.10.10.91:35357/v2.0", "region": "regionOne", "internalURL": "http://10.10.10.91:5000/v2.0", "id": "4f8c24b4ddf04eaa881a9915cb3f9e5d", "publicURL": "http://10.10.10.91:5000/v2.0"}], "endpoints_links": [], "type": "identity", "name": "keystone"}], "user": {"username": "admin", "roles_links": [], "id": "cd72fa7e6af44bc5a5b7c3bc7473c367", "roles": [{"name": "admin"}], "name": "admin"}, "metadata": {"is_admin": 0, "roles": ["b8145ce1c1a54d3097cc705ae8cdedb3"]}}}

REQ: curl -i http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4/volumes -X POST -H "X-Auth-Project-Id: admin" -H "User-Agent: python-cinderclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: MIILLwYJKoZIhvcNAQcCoIILIDCCCxwCAQExDTALBglghkgBZQMEAgEwggl9BgkqhkiG9w0BBwGgggluBIIJansiYWNjZXNzIjogeyJ0b2tlbiI6IHsiaXNzdWVkX2F0IjogIjIwMTQtMDYtMjNUMjI6MTk6NDguMDM2ODQ4IiwgImV4cGlyZXMiOiAiMjAxNC0wNi0yM1QyMzoxOTo0OFoiLCAiaWQiOiAicGxhY2Vob2xkZXIiLCAidGVuYW50IjogeyJkZXNjcmlwdGlvbiI6IG51bGwsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogIjA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgInJlZ2lvbiI6ICJyZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjg3NzQvdjIvMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiM2NjMzg0ZmUwNjJkNDk0Y2E5MWVjNjExZTkzZmNkZGMiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0In1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImNvbXB1dGUiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6OTI5MiIsICJpZCI6ICIwMTRmYjMzYmFiNWE0MjQyODJmNGFjYWM3OTBiMjJmOCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaW1hZ2UiLCAibmFtZSI6ICJnbGFuY2UifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc2L3YxLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgImlkIjogIjViNjUzNTMwNGVkMjQxMjdhZDcwNDg4MWQ3NmJkZGUxIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWUiLCAibmFtZSI6ICJjaW5kZXIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4NzczL3NlcnZpY2VzL0Nsb3VkIiwgImlkIjogIjExYjA2ZmE3ZDg3YjQ1MTBiNGE1ODM1MmYxODI5NGQ3IiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9DbG91ZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJlYzIiLCAibmFtZSI6ICJlYzIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODA4MCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiODIwZDg5Mjc2YzMzNDY3MzhhYTkzZjFkNWJmZjk1MTIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAib2JqZWN0LXN0b3JlIiwgIm5hbWUiOiAic3dpZnQifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAiLCAiaWQiOiAiNGY4YzI0YjRkZGYwNGVhYTg4MWE5OTE1Y2IzZjllNWQiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaWRlbnRpdHkiLCAibmFtZSI6ICJrZXlzdG9uZSJ9XSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICJjZDcyZmE3ZTZhZjQ0YmM1YTViN2MzYmM3NDczYzM2NyIsICJyb2xlcyI6IFt7Im5hbWUiOiAiYWRtaW4ifV0sICJuYW1lIjogImFkbWluIn0sICJtZXRhZGF0YSI6IHsiaXNfYWRtaW4iOiAwLCAicm9sZXMiOiBbImI4MTQ1Y2UxYzFhNTRkMzA5N2NjNzA1YWU4Y2RlZGIzIl19fX0xggGFMIIBgQIBATBcMFcxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVVbnNldDEOMAwGA1UEBwwFVW5zZXQxDjAMBgNVBAoMBVVuc2V0MRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20CAQEwCwYJYIZIAWUDBAIBMA0GCSqGSIb3DQEBAQUABIIBAGd9w5+HzIH4F5fwaLbFgoT+IYuDdc-ra1pIiYd+37-cC6hAM+PrEEh6Z+0MhtV2o3h-Uw4H1jz3t918YdRioxZqDpeJlBBaD8oEK2AF0QKhwmljRomFYmZsArhSG5jiBOIRUF4JVH48AuNVxJq1ZS1gQvU6b9TQdMe1Z2T1EmePCnztcLQYfrXUv2wWICr8v50y2MbDX5RkgHfEF1tE0wieVu9hJY32s7OlSb20UfT0T8R-gvw9nvZIrQ5K+Qxgb8BwD81tphhDoGIgjuVhMHi4PgNOTQkhXV4oK285Sw9zS3HVL5+BFuBf9oHHQcrEgWH9yuF8N9XKu4aiMyJwZnw=" -d '{"volume": {"status": "creating", "availability_zone": null, "source_volid": null, "display_description": "Test2", "snapshot_id": null, "user_id": null, "size": 100, "display_name": "cinder-ceph-vol3", "imageRef": null, "attach_status": "detached", "volume_type": null, "project_id": null, "metadata": {}}}'

DEBUG:cinderclient.client: REQ: curl -i http://10.10.10.91:8776/v1/0696d4e77c4c494ab653c91df7324ae4/volumes -X POST -H "X-Auth-Project-Id: admin" -H "User-Agent: python-cinderclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: MIILLwYJKoZIhvcNAQcCoIILIDCCCxwCAQExDTALBglghkgBZQMEAgEwggl9BgkqhkiG9w0BBwGgggluBIIJansiYWNjZXNzIjogeyJ0b2tlbiI6IHsiaXNzdWVkX2F0IjogIjIwMTQtMDYtMjNUMjI6MTk6NDguMDM2ODQ4IiwgImV4cGlyZXMiOiAiMjAxNC0wNi0yM1QyMzoxOTo0OFoiLCAiaWQiOiAicGxhY2Vob2xkZXIiLCAidGVuYW50IjogeyJkZXNjcmlwdGlvbiI6IG51bGwsICJlbmFibGVkIjogdHJ1ZSwgImlkIjogIjA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgIm5hbWUiOiAiYWRtaW4ifX0sICJzZXJ2aWNlQ2F0YWxvZyI6IFt7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgInJlZ2lvbiI6ICJyZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjg3NzQvdjIvMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiM2NjMzg0ZmUwNjJkNDk0Y2E5MWVjNjExZTkzZmNkZGMiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc0L3YyLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0In1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImNvbXB1dGUiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIiLCAicmVnaW9uIjogInJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6OTI5MiIsICJpZCI6ICIwMTRmYjMzYmFiNWE0MjQyODJmNGFjYWM3OTBiMjJmOCIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEwLjEwLjEwLjkxOjkyOTIifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaW1hZ2UiLCAibmFtZSI6ICJnbGFuY2UifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4Nzc2L3YxLzA2OTZkNGU3N2M0YzQ5NGFiNjUzYzkxZGY3MzI0YWU0IiwgImlkIjogIjViNjUzNTMwNGVkMjQxMjdhZDcwNDg4MWQ3NmJkZGUxIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3Ni92MS8wNjk2ZDRlNzdjNGM0OTRhYjY1M2M5MWRmNzMyNGFlNCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJ2b2x1bWUiLCAibmFtZSI6ICJjaW5kZXIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4NzczL3NlcnZpY2VzL0Nsb3VkIiwgImlkIjogIjExYjA2ZmE3ZDg3YjQ1MTBiNGE1ODM1MmYxODI5NGQ3IiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODc3My9zZXJ2aWNlcy9DbG91ZCJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJlYzIiLCAibmFtZSI6ICJlYzIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6ODA4MCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQiLCAiaWQiOiAiODIwZDg5Mjc2YzMzNDY3MzhhYTkzZjFkNWJmZjk1MTIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo4MDgwL3YxL0FVVEhfMDY5NmQ0ZTc3YzRjNDk0YWI2NTNjOTFkZjczMjRhZTQifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAib2JqZWN0LXN0b3JlIiwgIm5hbWUiOiAic3dpZnQifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTAuMTAuMTAuOTE6MzUzNTcvdjIuMCIsICJyZWdpb24iOiAicmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAiLCAiaWQiOiAiNGY4YzI0YjRkZGYwNGVhYTg4MWE5OTE1Y2IzZjllNWQiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMC4xMC4xMC45MTo1MDAwL3YyLjAifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiaWRlbnRpdHkiLCAibmFtZSI6ICJrZXlzdG9uZSJ9XSwgInVzZXIiOiB7InVzZXJuYW1lIjogImFkbWluIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICJjZDcyZmE3ZTZhZjQ0YmM1YTViN2MzYmM3NDczYzM2NyIsICJyb2xlcyI6IFt7Im5hbWUiOiAiYWRtaW4ifV0sICJuYW1lIjogImFkbWluIn0sICJtZXRhZGF0YSI6IHsiaXNfYWRtaW4iOiAwLCAicm9sZXMiOiBbImI4MTQ1Y2UxYzFhNTRkMzA5N2NjNzA1YWU4Y2RlZGIzIl19fX0xggGFMIIBgQIBATBcMFcxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVVbnNldDEOMAwGA1UEBwwFVW5zZXQxDjAMBgNVBAoMBVVuc2V0MRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20CAQEwCwYJYIZIAWUDBAIBMA0GCSqGSIb3DQEBAQUABIIBAGd9w5+HzIH4F5fwaLbFgoT+IYuDdc-ra1pIiYd+37-cC6hAM+PrEEh6Z+0MhtV2o3h-Uw4H1jz3t918YdRioxZqDpeJlBBaD8oEK2AF0QKhwmljRomFYmZsArhSG5jiBOIRUF4JVH48AuNVxJq1ZS1gQvU6b9TQdMe1Z2T1EmePCnztcLQYfrXUv2wWICr8v50y2MbDX5RkgHfEF1tE0wieVu9hJY32s7OlSb20UfT0T8R-gvw9nvZIrQ5K+Qxgb8BwD81tphhDoGIgjuVhMHi4PgNOTQkhXV4oK285Sw9zS3HVL5+BFuBf9oHHQcrEgWH9yuF8N9XKu4aiMyJwZnw=" -d '{"volume": {"status": "creating", "availability_zone": null, "source_volid": null, "display_description": "Test2", "snapshot_id": null, "user_id": null, "size": 100, "display_name": "cinder-ceph-vol3", "imageRef": null, "attach_status": "detached", "volume_type": null, "project_id": null, "metadata": {}}}'

RESP: [200] CaseInsensitiveDict({'date': 'Mon, 23 Jun 2014 22:19:48 GMT', 'x-compute-request-id': 'req-e4b7aab0-281b-419c-8790-2f82d1de06fa', 'content-type': 'application/json', 'content-length': '376', 'x-openstack-request-id': 'req-e4b7aab0-281b-419c-8790-2f82d1de06fa'}) RESP BODY: {"volume": {"status": "creating", "display_name": "cinder-ceph-vol3", "attachments": [], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2014-06-23T22:19:48.314028", "display_description": "Test2", "volume_type": "None", "snapshot_id": null, "source_volid": null, "metadata": {}, "id": "a5f04645-603c-451c-8d01-802af9ddfcb2", "size": 100}}

DEBUG:cinderclient.client:RESP: [200] CaseInsensitiveDict({'date': 'Mon, 23 Jun 2014 22:19:48 GMT', 'x-compute-request-id': 'req-e4b7aab0-281b-419c-8790-2f82d1de06fa', 'content-type': 'application/json', 'content-length': '376', 'x-openstack-request-id': 'req-e4b7aab0-281b-419c-8790-2f82d1de06fa'}) RESP BODY: {"volume": {"status": "creating", "display_name": "cinder-ceph-vol3", "attachments": [], "availability_zone": "nova", "bootable": "false", "encrypted": false, "created_at": "2014-06-23T22:19:48.314028", "display_description": "Test2", "volume_type": "None", "snapshot_id": null, "source_volid": null, "metadata": {}, "id": "a5f04645-603c-451c-8d01-802af9ddfcb2", "size": 100}}

+---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | created_at | 2014-06-23T22:19:48.314028 | | display_description | Test2 | | display_name | cinder-ceph-vol3 | | encrypted | False | | id | a5f04645-603c-451c-8d01-802af9ddfcb2 | | metadata | {} | | size | 100 | | snapshot_id | None | | source_volid | None | | status | creating | | volume_type | None | +---------------------+--------------------------------------+ root@ccinder2:~#

root@ccinder2:~# cinder list +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+ | 70084540-0314-4465-9706-6e59f871b3a8 | error | cinder-ceph-vol1 | 100 | None | false | | | 723853f5-60b8-4ee2-806c-694597783cdb | available | test_rbd_vol | 5 | None | false | | | 826043a3-8586-47f9-ac02-f93aa5b4f913 | error | cinder_ceph_vol2 | 100 | None | false | | | a5f04645-603c-451c-8d01-802af9ddfcb2 | error | cinder-ceph-vol3 | 100 | None | false | | +--------------------------------------+-----------+------------------+------+-------------+----------+-------------+

Thanks again!!!