Revision history [back]

click to hide/show revision 1
initial version

unable retrieve instance list after upgrade from newton to pike .

Hi,

I have perform a migration from mitaka to newton without implement nova cells and that worked fine . Now I tried to perform the migration from newton to pike and I got this strange error on nova-api.log

That error only occurs when I run the instance list from any tenant except admin .

openstack server list 
+--------------------------------------+------+--------+-------------------------------+------------+-------------+
| ID                                   | Name | Status | Networks                      | Image      | Flavor      |
+--------------------------------------+------+--------+-------------------------------+------------+-------------+
| 6a60dc73-6c93-4c8e-b06e-472c0aa9bedd | 03   | ACTIVE | NET-10.10.12.0-22=10.10.15.0  | Centos-7   | C1.M4.HDD20 |
| 6bff0a14-8633-4c72-890f-c420051e704e | 04   | ACTIVE | NET-10.10.12.0-22=10.10.15.12 | Centos-7   | C1.M4.HDD20 |
| 17c8a11a-7498-433a-8501-ae95d66f2922 | 01   | ERROR  |                               | Debian - 8 | m1.large    |
+--------------------------------------+------+--------+-------------------------------+------------+-------------+

openstack server list  --all-projects
Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-32428f8a-7d0e-47bb-b765-5865a0f01d68)

the /var/log/nova/nova-api.log is :

2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 336, in wrapped
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 181, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 168, in detail
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     servers = self._get_servers(req, is_detail=True)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 311, in _get_servers
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     sort_keys=sort_keys, sort_dirs=sort_dirs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2415, in get_all
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     sort_dirs=sort_dirs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2512, in _get_instances_by_filters_all_cells
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2543, in _get_instances_by_filters
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     expected_attrs=fields, sort_keys=sort_keys, sort_dirs=sort_dirs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 184, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     result = fn(cls, context, *args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1245, in get_by_filters
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     use_slave=use_slave, sort_keys=sort_keys, sort_dirs=sort_dirs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 235, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1235, in _get_by_filters_impl
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     expected_attrs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1198, in _make_instance_list
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     expected_attrs=expected_attrs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 448, in _from_db_object
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     objects.Service, db_inst['services'])
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 1121, in obj_make_list
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     **extra_args)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/objects/service.py", line 246, in _from_db_object
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     service.save()
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py", line 226, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return fn(self, *args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/objects/service.py", line 363, in save
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     db_service = db.service_update(self._context, self.id, updates)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 189, in service_update
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return IMPL.service_update(context, service_id, values)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 150, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     ectxt.value = e.inner_exc
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     self.force_reraise()
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     six.reraise(self.type_, self.value, self.tb)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_db/api.py", line 138, in wrapper
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 249, in wrapped
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     with ctxt_mgr.writer.using(context):
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     return self.gen.next()
2
018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1028, in _transaction_scope
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     context=context) as resource:
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 684, in _produce_block
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     self._writer()
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 699, in _writer
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions     "Can't upgrade a READER transaction "
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions TypeError: Can't upgrade a READER transaction to a WRITER mid-transaction
2018-01-29 11:46:32.494 19540 ERROR nova.api.openstack.extensions

Any help on this ?