Ask Your Question
0

glance and ceph snapshot not working

asked 2017-06-30 17:45:57 -0600

sauloaugustosilva gravatar image

Hi,

I have setup a configuration with Openstack and CEPH as backend . Everything is working except the instance SNAPSHOT . When we use the Horizon the image get on queued and instances at Image uploading . When we run cli command

nova image-create --show --poll 251ebb41-d57e-43e9-b16a-ea395e5922f2 CLI-SNAP-01

System stuck in this output

Server snapshotting... 25% complete

But I can see that the image was created in ceph

# nova image-list --limit 5
+--------------------------------------+------------------+--------+--------------------------------------+
| ID                                   | Name             | Status | Server                               |
+--------------------------------------+------------------+--------+--------------------------------------+
| 7b2bd798-6237-4c2d-8de6-5c7e3b3348de | CLI-SNAP-01      | SAVING | 251ebb41-d57e-43e9-b16a-ea395e5922f2 |
| 0ef8b698-96be-44a1-b41e-abf973f05a55 | CRM                | ACTIVE | 9b96dd70-38d5-4762-9d18-679db13a4333 |
| d3f18d34-bb00-44e1-85cd-71bfae7ca24a | Centos-7         | ACTIVE |                                      |
| c62b12a0-7d9e-4bea-b9e8-4eb27113cbee | SNAP01           | SAVING | a7ecfc9a-8f60-4940-9f8e-443b71e9499d |
| 476221a7-f331-423e-82af-0b68e8a78347 | SNAP02           | SAVING | a36dee50-8c87-478e-9264-f2bc5e909876 |
+--------------------------------------+------------------+--------+--------------------------------------+


# rbd -p images ls | grep 7b2bd798-6237-4c2d-8de6-5c7e3b3348de
7b2bd798-6237-4c2d-8de6-5c7e3b3348de
# rbd info images/7b2bd798-6237-4c2d-8de6-5c7e3b3348de 
  rbd image '7b2bd798-6237-4c2d-8de6-5c7e3b3348de':
        size 20480 MB in 5120 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.e6fe6576d1f0
        format: 2
        features: layering, striping
        flags:
        stripe unit: 4096 kB
        stripe count: 1

I can create and upload image with a really good speed with no problems . I also tested it with QCOW2 and RAW image with same result . Some snapshots finished after 5 days .

glance-api.conf file

[DEFAULT]
debug = True
verbose = True
log_dir = /var/log/glance
notification_driver = messaging
lock_path = /var/run/glance
show_multiple_locations = True
show_image_direct_url = True
enable_v2_api=True
enable_v1_api=True
enable_v2_registry = True
[database]
connection = mysql+pymysql://glance:XXXXXXX@host/glance
[image_format]
[keystone_authtoken]
auth_uri = http://host:5000
auth_url = http://host:35357
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = XXXXXXX
memcached_servers = host:11211
[matchmaker_redis]
[matchmaker_ring]
[oslo_concurrency]
[oslo_messaging_amqp]
[oslo_messaging_qpid]
[oslo_messaging_rabbit]
rabbit_host = host
rabbit_userid = openstack
rabbit_password = XXXXXX
[oslo_messaging_notifications]
driver = messagingv2
[oslo_policy]
[paste_deploy]
flavor = keystone
[store_type_location_strategy]
[task]
[taskflow_executor]
[glance_store]
stores = rbd
default_store = rbd
rbd_store_pool = images
rbd_store_user = images
rbd_store_ceph_conf = /etc/ceph/ceph.conf
rbd_store_chunk_size = 8

and this is my nova.conf file

[DEFAULT]
instance_usage_audit = True
instance_usage_audit_period = hour
notify_on_state_change = vm_and_task_state
notification_driver = messagingv2
enabled_apis = osapi_compute,metadata
rpc_backend = rabbit
auth_strategy = keystone
my_ip = Y.Y.Y.Y
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
notification_topics = notifications,notifications_designate
compute_driver = libvirt.LibvirtDriver
notification_driver=ceilometer.compute.nova_notifier
allow_resize_to_same_host = True
scheduler_default_filters = AllHostsFilter
debug = True
verbose = True
[api_database]
connection = mysql+pymysql://nova:0p3nM4ST3R@host/nova_api
[barbican]
[cache]
[cells]
[cinder]
os_region_name = RegionOne
[conductor]
[cors]
[cors.subdomain]
[database]
connection = mysql+pymysql://nova:0p3nM4ST3R@host/nova
[ephemeral_storage_encryption]
[glance]
api_servers = http://host:9292
[guestfs]
[hyperv]
[image_file_url]
[ironic]
[keymgr]
[keystone_authtoken]
auth_uri = http://host:5000
auth_url = http://host:35357
memcached_servers = host:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = nova
password = XxXXXXXXX
[libvirt]
virt_type = kvm
images_rbd_pool = vms
images_type = rbd
rbd_secret_uuid = c791e4ea-59fe-4f6e-9571-84b38fa800b2
rbd_user = cinder
disk_cachemodes = "network=writeback"
snapshots_directory = /var/lib/nova/snapshots
[matchmaker_redis]
[metrics]
[neutron]
url = http://host:9696
auth_url = http://host:35357
auth_type = password
project_domain_name = default
user_domain_name = default
region_name = RegionOne
project_name = service
username = neutron
password = XXXXXXX
service_metadata_proxy = True
metadata_proxy_shared_secret = XXXXXXX
[osapi_v21]
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
rabbit_host = openstack
rabbit_userid = openstack
rabbit_password = XXXXXXXX
[oslo_middleware]
[oslo_policy]
[rdp]
[serial_console]
[spice]
[ssl]
[trusted_computing]
[upgrade_levels]
[vmware]
[vnc]
enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = Y.Y.Y.Y
novncproxy_base_url = https://host.com:6080/vnc_auto.html
[workarounds]
[xenserver]
edit retag flag offensive close merge delete

Comments

i have same problem what should i do thanks

novainfinite gravatar imagenovainfinite ( 2018-11-26 05:58:59 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2017-07-05 20:44:27 -0600

sauloaugustosilva gravatar image

updated 2017-07-05 21:48:46 -0600

Some times snapshots works really fast . Although after 3 or 4 snapshot it got really slow again . So I started to investigating all components and what I did was : 1- Include the following configuration ( what I don't think that was the cause of the fix )

hw_disk_discard = unmap 
live_migration_flag = "VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST,VIR_MIGRATE_TUNNELLED" 
images_rbd_ceph_conf = /etc/ceph/ceph.conf

2- Include at [client] section of /etc/ceph/ceph.conf file with the follow statement in all compute node :

 admin socket = /var/run/ceph/guests/$cluster-$type.$id.$pid.$cctid.asok
 log file = /var/log/qemu/qemu-guest-$pid.log
 rbd concurrent management ops = 20

There was also another important fix that I supposed that helped in the solution as fix some unclean pg in my ceph cluster . I do not know why it interfere at the fix because the unclean pgs was in another pool that was not related to the vms or images pools .

After some test with 80 GB instance the image stuck at saving state . Investigating .

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2017-06-30 17:45:57 -0600

Seen: 494 times

Last updated: Jul 05 '17