Ask Your Question
1

Failure to create instance via `nova boot` & creating cinder volume simalteniously on F20 RDO Havana

asked 2014-03-26 09:55:08 -0500

dbaxps gravatar image 
[root@dfw02 ~(keystone_boris)]$ nova boot --flavor 2 --key_name oskey25 --block-device source=image,\
> id=de93ee44-4085-4111-b022-a7437da8feac,dest=volume,size=5,shutdown=preserve,bootindex=0 VF20RS025
+--------------------------------------+-------------------------------------------------+
| Property                             | Value                                           |
+--------------------------------------+-------------------------------------------------+
| status                               | BUILD                                           |
| updated                              | 2014-03-26T14:41:33Z                            |
| OS-EXT-STS:task_state                | scheduling                                      |
| key_name                             | oskey25                                         |
| image                                | Attempt to boot from volume - no image supplied |
| hostId                               |                                                 |
| OS-EXT-STS:vm_state                  | building                                        |
| OS-SRV-USG:launched_at               | None                                            |
| flavor                               | m1.small                                        |
| id                                   | 0ff31180-3cb5-42e7-b346-124d6c3a35b1            |
| security_groups                      | [{u'name': u'default'}]                         |
| OS-SRV-USG:terminated_at             | None                                            |
| user_id                              | 162021e787c54cac906ab3296a386006                |
| name                                 | VF20RS025                                       |
| adminPass                            | LXdfLNyu8JdC                                    |
| tenant_id                            | 4dacfff9e72c4245a48d648ee23468d5                |
| created                              | 2014-03-26T14:41:33Z                            |
| OS-DCF:diskConfig                    | MANUAL                                          |
| metadata                             | {}                                              |
| os-extended-volumes:volumes_attached | []                                              |
| accessIPv4                           |                                                 |
| accessIPv6                           |                                                 |
| progress                             | 0                                               |
| OS-EXT-STS:power_state               | 0                                               |
| OS-EXT-AZ:availability_zone          | nova                                            |
| config_drive                         |                                                 |
+--------------------------------------+-------------------------------------------------+
[root@dfw02 ~(keystone_boris)]$ nova list
+--------------------------------------+--------------+-----------+----------------------+-------------+------------------------------+
| ID                                   | Name         | Status    | Task State           | Power State | Networks                     |
+--------------------------------------+--------------+-----------+----------------------+-------------+------------------------------+
| e0c24a31-d6cf-4255-a3c7-46357859ba07 | UbuntuTRS025 | SUSPENDED | None                 | Shutdown    | int1=40.0.0.7, 192.168.1.111 |
| d8e650e1-f46b-4af3-afa9-2a7d7d90c27c | UbuntuTRS026 | SUSPENDED | None                 | Shutdown    | int1=40.0.0.2, 192.168.1.101 |
| edba5bfe-eb77-49e0-a15c-2f3355ff3a3b | VF20RS012    | SUSPENDED | None                 | Shutdown    | int1=40.0.0.5, 192.168.1.110 |
| 95b5f88b-fb02-4269-a3af-6f0e36686888 | VF20RS017    | SUSPENDED | None                 | Shutdown    | int1=40.0.0.4, 192.168.1.112 |
| 0ff31180-3cb5-42e7-b346-124d6c3a35b1 | VF20RS025    | BUILD     | block_device_mapping | NOSTATE     |                              |
+--------------------------------------+--------------+-----------+----------------------+-------------+------------------------------+

/var/log/nova/compute.log :

2014-03-26 18:42:42.408 2450 TRACE nova.compute.manager [instance: 0ff31180-3cb5-42e7-b346-124d6c3a35b1]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 907, in _await_block_device_map_created
2014-03-26 18:42:42.408 2450 TRACE nova.compute.manager [instance: 0ff31180-3cb5-42e7-b346-124d6c3a35b1]     attempts=attempts)
2014-03-26 18:42:42.408 2450 TRACE nova.compute.manager [instance: 0ff31180-3cb5-42e7-b346-124d6c3a35b1] VolumeNotCreated: Volume 5a363cb9-98f3-4ccd-bdf0-5bd8229be670 did not finish being created even after we waited 67 seconds or 60 attempts.
2014-03-26 18:42:42.408 2450 TRACE nova.compute.manager [instance: 0ff31180-3cb5-42e7-b346-124d6c3a35b1] 
2014-03-26 18:42:43.178 2450 ERROR nova.openstack.common.rpc.amqp [req-54d8d5cf-dfbf-4079-a762-85901277f75a 162021e787c54cac906ab3296a386006 4dacfff9e72c4245a48d648ee23468d5] Exception during message handling
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last):
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp     **args)
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp     result = getattr(proxyobj, method)(ctxt, **kwargs)
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 90, in wrapped
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp     payload)
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 73, in wrapped
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp     return f(self, context, *args, **kw)
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 243, in decorated_function
2014-03-26 18:42:43.178 2450 TRACE nova.openstack.common.rpc.amqp     pass
2014-03-26 18:42 ...
(more)
edit retag flag offensive close merge delete
add a comment

2 answers

Sort by ยป oldest newest most voted
0

answered 2014-03-26 16:56:12 -0500

dbaxps gravatar image

updated 2014-03-29 04:41:30 -0500

  1. parameters max_tries and wait_between of method ComputeManager._await_block_device_map_created should be configurable

  2. Admin event callback API

    Provide an API for external services to send defined events to the compute service for synchronization. This includes immediate needs for nova-neutron interaction around boot timing and network info updates.

  3. Change I2f771d8b: Make libvirt wait for neutron to confirm plugging before boot

Actually, volume with system number like 9e810025-bfee-49d5-a109-b1d6a53afcba been created ,before failure to load, is OK, like it has been done with cinder create --image-id ... . I can create Ubuntu 14.04 instance via this volume in same dashboard env. Created instance will be functional at least via my experience. It really looks like nova-compute tries to load instance too early. 

[root@dfw02 ~(keystone_boris)]$ nova list
+--------------------------------------+--------------+-----------+------------+-------------+------------------------------+
| ID                                   | Name         | Status    | Task State | Power State | Networks                     |
+--------------------------------------+--------------+-----------+------------+-------------+------------------------------+
| 609b7d67-8e1f-4c65-ae82-4d21254a2392 | UbuntuTRS001 | ACTIVE    | None       | Running     | int1=40.0.0.5, 192.168.1.106 |
| 2ebd744e-52a4-469b-84be-421a564da3e5 | UbuntuTRS029 | ACTIVE    | None       | Running     | int1=40.0.0.7, 192.168.1.102 |
| 115bae2d-3458-4a2f-bd65-c895a8a7a8a4 | VF20RS012    | SUSPENDED | None       | Shutdown    | int1=40.0.0.4, 192.168.1.105 |
| f264ce45-0b5a-448c-84e7-c92e0a350053 | VF20RS027    | ACTIVE    | None       | Running     | int1=40.0.0.2, 192.168.1.101 |
| 97825e96-b4d1-4638-9212-480a37c61dd5 | VF20RS047    | SUSPENDED | None       | Shutdown    | int1=40.0.0.6, 192.168.1.107 |
+--------------------------------------+--------------+-----------+------------+-------------+------------------------------+
[root@dfw02 ~(keystone_boris)]$ nova show UbuntuTRS001
+--------------------------------------+----------------------------------------------------------+
| Property                             | Value                                                    |
+--------------------------------------+----------------------------------------------------------+
| status                               | ACTIVE                                                   |
| updated                              | 2014-03-29T09:14:03Z                                     |
| OS-EXT-STS:task_state                | None                                                     |
| key_name                             | oskey25                                                  |
| image                                | Attempt to boot from volume - no image supplied          |
| hostId                               | 73ee4f5bd4da8ad7b39d768d0b167a03ac0471ea50d9ded6c6190fb1 |
| OS-EXT-STS:vm_state                  | active                                                   |
| OS-SRV-USG:launched_at               | 2014-03-29T09:14:03.000000                               |
| flavor                               | m1.small (2)                                             |
| id                                   | 609b7d67-8e1f-4c65-ae82-4d21254a2392                     |
| security_groups                      | [{u'name': u'default'}]                                  |
| OS-SRV-USG:terminated_at             | None                                                     |
| user_id                              | 162021e787c54cac906ab3296a386006                         |
| name                                 | UbuntuTRS001                                             |
| created                              | 2014-03-29T09:13:56Z                                     |
| tenant_id                            | 4dacfff9e72c4245a48d648ee23468d5                         |
| OS-DCF:diskConfig                    | MANUAL                                                   |
| metadata                             | {}                                                       |
| os-extended-volumes:volumes_attached | [{u'id': u'9e810025-bfee-49d5-a109-b1d6a53afcba'}]       |
| accessIPv4                           |                                                          |
| accessIPv6                           |                                                          |
| progress                             | 0                                                        |
| OS-EXT-STS:power_state               | 1                                                        |
| OS-EXT-AZ:availability_zone          | nova                                                     |
| int1 network                         | 40.0.0.5, 192.168.1.106                                  |
| config_drive                         |                                                          |
+--------------------------------------+----------------------------------------------------------+
[root@dfw02 ~(keystone_boris)]$ cinder list | grep 9e810025-bfee-49d5-a109-b1d6a53afcba
| 9e810025-bfee-49d5-a109-b1d6a53afcba | in-use |               |  5   |     None    |   true   | 609b7d67-8e1f-4c65-ae82-4d21254a2392 |

Hack with adding row to MySQL table with cinder's password for FQDN by some reasons works, but only for F20 images.

edit flag offensive delete link more

Comments

I also tested recompiling /usr/lib/python2.7/site-packages/nova/compute/manager.py , method _await_block_device_map_created as suggested in parameters max_tries and wait_between of method ComputeManager._await_block_device_map_created should be configurable . It doesn't work for me.

dbaxps gravatar imagedbaxps ( 2014-03-29 09:33:21 -0500 )edit
add a comment
0

answered 2014-03-26 11:09:40 -0500

Trump.Zhang gravatar image

From the compute log info , it seems that cinder had some problem during creating volume, could you show the corresponding cinder log ?

edit flag offensive delete link more

Comments

2014-03-26 08:14:19.884 2401 WARNING cinder.openstack.common.db.sqlalchemy.session [req-5d96bcbe-531c-4202-9e92-bc2d929b956e None None] SQL connection failed. 10 attempts left. 2014-03-26 08:14:31.998 2401 WARNING cinder.openstack.common.db.sqlalchemy.session [req-5d96bcbe-531c-4202-9e92-bc2d929b956e None None] SQL connection failed. 9 attempts left. 2014-03-26 08:14:36.076 3110 WARNING cinder.volume.manager [req-364f2a1a-dd0a-443d-94a3-8ecc5b758944 None None] Unable to update stats, driver is uninitialized 2014-03-26 18:10:39.719 2403 WARNING cinder.openstack.common.db.sqlalchemy.session [req-3beb9012-9d81-45e9-8472-0ac7d89db452 None None] SQL connection failed. 10 attempts left. 2014-03-26 18:10:43.024 2723 WARNING cinder.openstack.common.db.sqlalchemy.session [req-3f056360-

    dedf-41c8-bddd-522e469db414 None None] SQL connection failed. 10 attempts left.
    2014-03-26 18:10:58.089 2723 WARNING cinder.volume.manager [req-0d6f2a99-d7ef-41fa-b999-df9b673e0ef0 None None] Unable to update stats, driver is uninitialized
dbaxps gravatar imagedbaxps ( 2014-03-26 16:12:50 -0500 )edit

So, from the cinder log, it seems your database cannot be connected by cinder.

Trump.zhang gravatar imageTrump.zhang ( 2014-03-26 20:22:14 -0500 )edit
add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

subscribe to rss feed

Stats

Asked: 2014-03-26 09:55:08 -0500

Seen: 950 times

Last updated: Mar 29 '14

Related questions

Failed to scheduler_run_instance: No valid host was found [closed]

Why i can't get my cinder-volume service started ?

Can not launch instances: Build of instance was re-scheduled [closed]

RDO install newton in the dashboard can't create images.

missing public network

How to prevent cloud-init from disable ssh PasswordAuthentication? [closed]

Cinder-volume not registered?

Cinder-volume backup

which version of QEMU supports Per Node memory Binding (NUMA Pinning on Kilo)

Trove Rabbit Config in RDO Packstack install