Revision history [back]

click to hide/show revision 1
initial version

[Ocata] nova.api.openstack Caught error: Could not determine a suitable URL for the plugin

I ran yum update and now any request to the nova api fails with nova.api.openstack [-] Caught error: Could not determine a suitable URL for the plugin. I can get the token from keystone but when I use it to lookup hypervisors or a server it results in error.

List of updated packages and versions:

---> Package openstack-cinder.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-cinder.noarch 1:10.0.5-1.el7 will be an update
---> Package openstack-dashboard.noarch 1:11.0.1-2.el7 will be updated
---> Package openstack-dashboard.noarch 1:11.0.3-1.el7 will be an update
---> Package openstack-heat-api.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-api.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-api-cfn.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-api-cfn.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-common.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-common.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-engine.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-engine.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-keystone.noarch 1:11.0.0-1.el7 will be updated
---> Package openstack-keystone.noarch 1:11.0.3-1.el7 will be an update
---> Package openstack-magnum-api.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-api.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-magnum-common.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-common.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-magnum-conductor.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-conductor.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-neutron.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-common.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-common.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-fwaas.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-fwaas.noarch 1:10.1.0-1.el7 will be an update
---> Package openstack-neutron-ml2.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-ml2.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-openvswitch.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-openvswitch.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-nova-api.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-api.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-common.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-common.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-conductor.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-conductor.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-console.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-console.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-novncproxy.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-novncproxy.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-placement-api.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-placement-api.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-scheduler.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-scheduler.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-selinux.noarch 0:0.7.13-2.el7 will be updated
---> Package openstack-selinux.noarch 0:0.8.7-1.el7 will be an update

Nova config file:

[DEFAULT]
my_ip = 172.31.252.10
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
enabled_apis = osapi_compute,metadata
transport_url = rabbit://openstack:password@hostname.domain.com
state_path = /iscsi_lun/nova
ram_allocation_ratio = 3
[api]
auth_strategy = keystone
[api_database]
connection = postgresql://osnova_user:"password"@$my_ip/osnova_api_db
[barbican]
[cache]
[cells]
[cinder]
os_region_name = RegionOne
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[cloudpipe]
[conductor]
[console]
[consoleauth]
[cors]
[cors.subdomain]
[crypto]
key_file = /etc/pki/tls/private/hostname.domain.com.key
ca_file = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[database]
connection = postgresql://osnova_user:"password"@$my_ip/osnova_db
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers = http://hostname.domain.com:9292
[guestfs]
[healthcheck]
[hyperv]
[image_file_url]
[ironic]
[key_manager]
[keystone_authtoken]
auth_uri = https://hostname.domain.com:5000
auth_url = https://hostname.domain.com:35357
memcached_servers = hostname.domain.com:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = password
[libvirt]
virt_type = kvm
[matchmaker_redis]
[metrics]
[mks]
[neutron]
url = http://hostname.domain.com:9696
auth_url = https://hostname.domain.com:35357
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = password
service_metadata_proxy = true
metadata_proxy_shared_secret = password
ovs_bridge = br-ovs
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[notifications]
[osapi_v21]
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[pci]
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = https://hostname.domain.com:35357/v3
username = placement
password = password
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[ssl]
[trusted_computing]
[upgrade_levels]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled = true
vncserver_listen = $my_ip
vncserver_proxyclient_address = $my_ip
[workarounds]
[wsgi]
[xenserver]
[xvp]

HTTP POST #1:

GET /v2.1/servers/detail HTTP/1.1
Host: controller00.critical-sec.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0
X-Auth-Token: XXXXXXXXXXXXXXXXXXX
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Content-Type: application/json
Content-Length: 0
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1

HTTP POST #2:

GET /v2.1/os-hypervisors/detail HTTP/1.1
Host: controller00.critical-sec.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0
X-Auth-Token: XXXXXXXXXXXXXX
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Content-Type: application/json
Content-Length: 0
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1

Result in this Error:

WARNING keystoneauth.identity.generic.base [-] Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
ERROR nova.api.openstack [-] Caught error: Could not determine a suitable URL for the plugin
ERROR nova.api.openstack Traceback (most recent call last):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 88, in __call__
ERROR nova.api.openstack     return req.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 108, in __call__
ERROR nova.api.openstack     return request.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 332, in __call__
ERROR nova.api.openstack     response = self.process_request(req)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 623, in process_request
ERROR nova.api.openstack     resp = super(AuthProtocol, self).process_request(request)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 405, in process_request
ERROR nova.api.openstack     allow_expired=allow_expired)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 435, in _do_fetch_token
ERROR nova.api.openstack     data = self.fetch_token(token, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 762, in fetch_token
ERROR nova.api.openstack     allow_expired=allow_expired)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 217, in verify_token
ERROR nova.api.openstack     auth_ref = self._request_strategy.verify_token(
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 168, in _request_strategy
ERROR nova.api.openstack     strategy_class = self._get_strategy_class()
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 190, in _get_strategy_class
ERROR nova.api.openstack     if self._adapter.get_endpoint(version=klass.AUTH_VERSION):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 176, in get_endpoint
ERROR nova.api.openstack     return self.session.get_endpoint(auth or self.auth, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 856, in get_endpoint
ERROR nova.api.openstack     return auth.get_endpoint(self, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 212, in get_endpoint
ERROR nova.api.openstack     service_catalog = self.get_access(session).service_catalog
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 136, in get_access
ERROR nova.api.openstack     self.auth_ref = self.get_auth_ref(session)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 196, in get_auth_ref
ERROR nova.api.openstack     self._plugin = self._do_create_plugin(session)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 191, in _do_create_plugin
ERROR nova.api.openstack     raise exceptions.DiscoveryFailure('Could not determine a suitable URL '
ERROR nova.api.openstack DiscoveryFailure: Could not determine a suitable URL for the plugin
ERROR nova.api.openstack

[Ocata] nova.api.openstack Caught error: Could not determine a suitable URL for the plugin

I ran yum update and now any request to the nova api fails with nova.api.openstack [-] Caught error: Could not determine a suitable URL for the plugin. I can get the token from keystone but when I use it to lookup hypervisors or a server it results in error.

List of updated packages and versions:

---> Package openstack-cinder.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-cinder.noarch 1:10.0.5-1.el7 will be an update
---> Package openstack-dashboard.noarch 1:11.0.1-2.el7 will be updated
---> Package openstack-dashboard.noarch 1:11.0.3-1.el7 will be an update
---> Package openstack-heat-api.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-api.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-api-cfn.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-api-cfn.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-common.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-common.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-engine.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-engine.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-keystone.noarch 1:11.0.0-1.el7 will be updated
---> Package openstack-keystone.noarch 1:11.0.3-1.el7 will be an update
---> Package openstack-magnum-api.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-api.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-magnum-common.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-common.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-magnum-conductor.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-conductor.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-neutron.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-common.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-common.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-fwaas.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-fwaas.noarch 1:10.1.0-1.el7 will be an update
---> Package openstack-neutron-ml2.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-ml2.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-openvswitch.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-openvswitch.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-nova-api.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-api.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-common.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-common.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-conductor.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-conductor.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-console.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-console.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-novncproxy.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-novncproxy.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-placement-api.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-placement-api.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-scheduler.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-scheduler.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-selinux.noarch 0:0.7.13-2.el7 will be updated
---> Package openstack-selinux.noarch 0:0.8.7-1.el7 will be an update

Nova config file:

[DEFAULT]
my_ip = 172.31.252.10
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
enabled_apis = osapi_compute,metadata
transport_url = rabbit://openstack:password@hostname.domain.com
state_path = /iscsi_lun/nova
ram_allocation_ratio = 3
[api]
auth_strategy = keystone
[api_database]
connection = postgresql://osnova_user:"password"@$my_ip/osnova_api_db
[barbican]
[cache]
[cells]
[cinder]
os_region_name = RegionOne
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[cloudpipe]
[conductor]
[console]
[consoleauth]
[cors]
[cors.subdomain]
[crypto]
key_file = /etc/pki/tls/private/hostname.domain.com.key
ca_file = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[database]
connection = postgresql://osnova_user:"password"@$my_ip/osnova_db
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers = http://hostname.domain.com:9292
[guestfs]
[healthcheck]
[hyperv]
[image_file_url]
[ironic]
[key_manager]
[keystone_authtoken]
auth_uri = https://hostname.domain.com:5000
auth_url = https://hostname.domain.com:35357
memcached_servers = hostname.domain.com:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = password
[libvirt]
virt_type = kvm
[matchmaker_redis]
[metrics]
[mks]
[neutron]
url = http://hostname.domain.com:9696
auth_url = https://hostname.domain.com:35357
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = password
service_metadata_proxy = true
metadata_proxy_shared_secret = password
ovs_bridge = br-ovs
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[notifications]
[osapi_v21]
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[pci]
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = https://hostname.domain.com:35357/v3
username = placement
password = password
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[ssl]
[trusted_computing]
[upgrade_levels]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled = true
vncserver_listen = $my_ip
vncserver_proxyclient_address = $my_ip
[workarounds]
[wsgi]
[xenserver]
[xvp]

HTTP POST #1:

GET /v2.1/servers/detail HTTP/1.1
Host: controller00.critical-sec.com
hostname.domain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0
X-Auth-Token: XXXXXXXXXXXXXXXXXXX
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Content-Type: application/json
Content-Length: 0
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1

HTTP POST #2:

GET /v2.1/os-hypervisors/detail HTTP/1.1
Host: controller00.critical-sec.com
hostname.domain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0
X-Auth-Token: XXXXXXXXXXXXXX
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Content-Type: application/json
Content-Length: 0
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1

Result in this Error:

WARNING keystoneauth.identity.generic.base [-] Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
ERROR nova.api.openstack [-] Caught error: Could not determine a suitable URL for the plugin
ERROR nova.api.openstack Traceback (most recent call last):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 88, in __call__
ERROR nova.api.openstack     return req.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 108, in __call__
ERROR nova.api.openstack     return request.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 332, in __call__
ERROR nova.api.openstack     response = self.process_request(req)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 623, in process_request
ERROR nova.api.openstack     resp = super(AuthProtocol, self).process_request(request)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 405, in process_request
ERROR nova.api.openstack     allow_expired=allow_expired)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 435, in _do_fetch_token
ERROR nova.api.openstack     data = self.fetch_token(token, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 762, in fetch_token
ERROR nova.api.openstack     allow_expired=allow_expired)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 217, in verify_token
ERROR nova.api.openstack     auth_ref = self._request_strategy.verify_token(
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 168, in _request_strategy
ERROR nova.api.openstack     strategy_class = self._get_strategy_class()
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 190, in _get_strategy_class
ERROR nova.api.openstack     if self._adapter.get_endpoint(version=klass.AUTH_VERSION):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 176, in get_endpoint
ERROR nova.api.openstack     return self.session.get_endpoint(auth or self.auth, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 856, in get_endpoint
ERROR nova.api.openstack     return auth.get_endpoint(self, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 212, in get_endpoint
ERROR nova.api.openstack     service_catalog = self.get_access(session).service_catalog
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 136, in get_access
ERROR nova.api.openstack     self.auth_ref = self.get_auth_ref(session)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 196, in get_auth_ref
ERROR nova.api.openstack     self._plugin = self._do_create_plugin(session)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 191, in _do_create_plugin
ERROR nova.api.openstack     raise exceptions.DiscoveryFailure('Could not determine a suitable URL '
ERROR nova.api.openstack DiscoveryFailure: Could not determine a suitable URL for the plugin
ERROR nova.api.openstack

[Ocata] nova.api.openstack Caught error: Could not determine a suitable URL for the plugin

I ran yum update and now any request to the nova api fails with nova.api.openstack [-] Caught error: Could not determine a suitable URL for the plugin. I can get the token from keystone but when I use it to lookup hypervisors or a server it results in error.

List of updated packages and versions:

---> Package openstack-cinder.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-cinder.noarch 1:10.0.5-1.el7 will be an update
---> Package openstack-dashboard.noarch 1:11.0.1-2.el7 will be updated
---> Package openstack-dashboard.noarch 1:11.0.3-1.el7 will be an update
---> Package openstack-heat-api.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-api.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-api-cfn.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-api-cfn.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-common.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-common.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-heat-engine.noarch 1:8.0.1-1.el7 will be updated
---> Package openstack-heat-engine.noarch 1:8.0.4-1.el7 will be an update
---> Package openstack-keystone.noarch 1:11.0.0-1.el7 will be updated
---> Package openstack-keystone.noarch 1:11.0.3-1.el7 will be an update
---> Package openstack-magnum-api.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-api.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-magnum-common.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-common.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-magnum-conductor.noarch 0:4.1.0-2.el7 will be updated
---> Package openstack-magnum-conductor.noarch 0:4.1.3-1.el7 will be an update
---> Package openstack-neutron.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-common.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-common.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-fwaas.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-fwaas.noarch 1:10.1.0-1.el7 will be an update
---> Package openstack-neutron-ml2.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-ml2.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-neutron-openvswitch.noarch 1:10.0.1-1.el7 will be updated
---> Package openstack-neutron-openvswitch.noarch 1:10.0.3-1.el7 will be an update
---> Package openstack-nova-api.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-api.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-common.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-common.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-conductor.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-conductor.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-console.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-console.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-novncproxy.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-novncproxy.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-placement-api.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-placement-api.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-nova-scheduler.noarch 1:15.0.3-2.el7 will be updated
---> Package openstack-nova-scheduler.noarch 1:15.0.7-1.el7 will be an update
---> Package openstack-selinux.noarch 0:0.7.13-2.el7 will be updated
---> Package openstack-selinux.noarch 0:0.8.7-1.el7 will be an update

Nova config file:

[DEFAULT]
my_ip = 172.31.252.10
use_neutron = True
firewall_driver = nova.virt.firewall.NoopFirewallDriver
enabled_apis = osapi_compute,metadata
transport_url = rabbit://openstack:password@hostname.domain.com
state_path = /iscsi_lun/nova
ram_allocation_ratio = 3
[api]
auth_strategy = keystone
[api_database]
connection = postgresql://osnova_user:"password"@$my_ip/osnova_api_db
[barbican]
[cache]
[cells]
[cinder]
os_region_name = RegionOne
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[cloudpipe]
[conductor]
[console]
[consoleauth]
[cors]
[cors.subdomain]
[crypto]
key_file = /etc/pki/tls/private/hostname.domain.com.key
ca_file = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[database]
connection = postgresql://osnova_user:"password"@$my_ip/osnova_db
[ephemeral_storage_encryption]
[filter_scheduler]
[glance]
api_servers = http://hostname.domain.com:9292
[guestfs]
[healthcheck]
[hyperv]
[image_file_url]
[ironic]
[key_manager]
[keystone_authtoken]
auth_uri = https://hostname.domain.com:5000
auth_url = https://hostname.domain.com:35357
memcached_servers = hostname.domain.com:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = password
[libvirt]
virt_type = kvm
[matchmaker_redis]
[metrics]
[mks]
[neutron]
url = http://hostname.domain.com:9696
auth_url = https://hostname.domain.com:35357
auth_type = password
project_domain_name = Default
user_domain_name = Default
region_name = RegionOne
project_name = service
username = neutron
password = password
service_metadata_proxy = true
metadata_proxy_shared_secret = password
ovs_bridge = br-ovs
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[notifications]
[osapi_v21]
[oslo_concurrency]
lock_path = /var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_kafka]
[oslo_messaging_notifications]
[oslo_messaging_rabbit]
[oslo_messaging_zmq]
[oslo_middleware]
[oslo_policy]
[pci]
[placement]
os_region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = https://hostname.domain.com:35357/v3
username = placement
password = password
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt
[quota]
[rdp]
[remote_debug]
[scheduler]
[serial_console]
[service_user]
[spice]
[ssl]
[trusted_computing]
[upgrade_levels]
[vendordata_dynamic_auth]
[vmware]
[vnc]
enabled = true
vncserver_listen = $my_ip
vncserver_proxyclient_address = $my_ip
[workarounds]
[wsgi]
[xenserver]
[xvp]

HTTP POST #1:

GET /v2.1/servers/detail HTTP/1.1
Host: hostname.domain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0
X-Auth-Token: XXXXXXXXXXXXXXXXXXX
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Content-Type: application/json
Content-Length: 0
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1

HTTP POST #2:

GET /v2.1/os-hypervisors/detail HTTP/1.1
Host: hostname.domain.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) Gecko/20100101 Firefox/54.0
X-Auth-Token: XXXXXXXXXXXXXX
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Content-Type: application/json
Content-Length: 0
DNT: 1
Connection: close
Upgrade-Insecure-Requests: 1

Result in this Error:

WARNING keystoneauth.identity.generic.base [-] Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL.
ERROR nova.api.openstack [-] Caught error: Could not determine a suitable URL for the plugin
ERROR nova.api.openstack Traceback (most recent call last):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 88, in __call__
ERROR nova.api.openstack     return req.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 108, in __call__
ERROR nova.api.openstack     return request.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 332, in __call__
ERROR nova.api.openstack     response = self.process_request(req)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 623, in process_request
ERROR nova.api.openstack     resp = super(AuthProtocol, self).process_request(request)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 405, in process_request
ERROR nova.api.openstack     allow_expired=allow_expired)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 435, in _do_fetch_token
ERROR nova.api.openstack     data = self.fetch_token(token, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 762, in fetch_token
ERROR nova.api.openstack     allow_expired=allow_expired)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 217, in verify_token
ERROR nova.api.openstack     auth_ref = self._request_strategy.verify_token(
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 168, in _request_strategy
ERROR nova.api.openstack     strategy_class = self._get_strategy_class()
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/_identity.py", line 190, in _get_strategy_class
ERROR nova.api.openstack     if self._adapter.get_endpoint(version=klass.AUTH_VERSION):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 176, in get_endpoint
ERROR nova.api.openstack     return self.session.get_endpoint(auth or self.auth, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 856, in get_endpoint
ERROR nova.api.openstack     return auth.get_endpoint(self, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 212, in get_endpoint
ERROR nova.api.openstack     service_catalog = self.get_access(session).service_catalog
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 136, in get_access
ERROR nova.api.openstack     self.auth_ref = self.get_auth_ref(session)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 196, in get_auth_ref
ERROR nova.api.openstack     self._plugin = self._do_create_plugin(session)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 191, in _do_create_plugin
ERROR nova.api.openstack     raise exceptions.DiscoveryFailure('Could not determine a suitable URL '
ERROR nova.api.openstack DiscoveryFailure: Could not determine a suitable URL for the plugin
ERROR nova.api.openstack

UPDATE:

Modified the [keystone_authtoken] config a bit..

[keystone_authtoken]
cafile = /etc/pki/tls/certs/gd_bundle-g2-g1.crt # <= added
auth_port = 35357 # <= added
auth_protocol = https # <= added
auth_host = controller00.critical-sec.com # <= added
auth_uri = https://host.domain.com:5000/v3 # <= added /v3
auth_url = https://host.domain.com:35357/v3 # <= added /v3
memcached_servers = host.domain.com:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = password

NEW ERROR:

ERROR nova.api.openstack [req-958fcfa3-c88a-44a6-99f0-376fc50c1d93 47ab0bb23880493b83a581c7fb8c1759 4a67db713b4242deb829a08f2a7bfd4b - default default] Caught error: The server has either erred or is incapable of performing the requested operation.<br /><br />



Neutron server returns request_ids: ['req-a76d49bb-5acb-4f71-b463-b3ea5c042dc5']
ERROR nova.api.openstack Traceback (most recent call last):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 88, in __call__
ERROR nova.api.openstack     return req.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/osprofiler/web.py", line 108, in __call__
ERROR nova.api.openstack     return request.get_response(self.application)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 335, in __call__
ERROR nova.api.openstack     response = req.get_response(self._app)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1299, in send
ERROR nova.api.openstack     application, catch_exc_info=False)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1263, in call_application
ERROR nova.api.openstack     app_iter = application(self.environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__
ERROR nova.api.openstack     response = self.app(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
ERROR nova.api.openstack     return resp(environ, start_response)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
ERROR nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
ERROR nova.api.openstack     return self.func(req, *args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 599, in __call__
ERROR nova.api.openstack     content_type, body, accept)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 679, in _process_stack
ERROR nova.api.openstack     request, action_args)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 545, in process_extensions
ERROR nova.api.openstack     **action_args)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/security_groups.py", line 496, in detail
ERROR nova.api.openstack     self._extend_servers(req, list(resp_obj.obj['servers']))
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/security_groups.py", line 466, in _extend_servers
ERROR nova.api.openstack     servers))
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/security_group/neutron_driver.py", line 379, in get_instances_security_groups_bindings
2017-09-27 08:28:52.546 17435 ERROR nova.api.openstack     ports = self._get_ports_from_server_list(servers, neutron)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/security_group/neutron_driver.py", line 338, in _get_ports_from_server_list
ERROR nova.api.openstack     ports.extend(neutron.list_ports(**search_opts).get('ports'))
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 99, in wrapper
ERROR nova.api.openstack     ret = obj(*args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 768, in list_ports
ERROR nova.api.openstack     **_params)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 99, in wrapper
ERROR nova.api.openstack     ret = obj(*args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 376, in list
ERROR nova.api.openstack     for r in self._pagination(collection, path, **params):
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 391, in _pagination
ERROR nova.api.openstack     res = self.get(path, params=params)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 99, in wrapper
ERROR nova.api.openstack     ret = obj(*args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 361, in get
ERROR nova.api.openstack     headers=headers, params=params)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 99, in wrapper
ERROR nova.api.openstack     ret = obj(*args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 338, in retry_request
ERROR nova.api.openstack     headers=headers, params=params)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 99, in wrapper
ERROR nova.api.openstack     ret = obj(*args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 301, in do_request
ERROR nova.api.openstack     self._handle_fault_response(status_code, replybody, resp)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 99, in wrapper
ERROR nova.api.openstack     ret = obj(*args, **kwargs)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 276, in _handle_fault_response
ERROR nova.api.openstack     exception_handler_v20(status_code, error_body)
ERROR nova.api.openstack   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 92, in exception_handler_v20
ERROR nova.api.openstack     request_ids=request_ids)
ERROR nova.api.openstack InternalServerError: The server has either erred or is incapable of performing the requested operation.<br /><br />
ERROR nova.api.openstack 
ERROR nova.api.openstack 
ERROR nova.api.openstack 
ERROR nova.api.openstack Neutron server returns request_ids: ['req-a76d49bb-5acb-4f71-b463-b3ea5c042dc5']
ERROR nova.api.openstack 
INFO nova.api.openstack [req-958fcfa3-c88a-44a6-99f0-376fc50c1d93 47ab0bb23880493b83a581c7fb8c1759 4a67db713b4242deb829a08f2a7bfd4b - default default] http://hostname.domain.com:8774/v2.1/servers/detail returned with HTTP 500
2017-09-27 08:28:52.549 17435 INFO nova.osapi_compute.wsgi.server [req-958fcfa3-c88a-44a6-99f0-376fc50c1d93 47ab0bb23880493b83a581c7fb8c1759 4a67db713b4242deb829a08f2a7bfd4b - default default] 172.31.252.10 "GET /v2.1/servers/detail HTTP/1.1" status: 500 len: 495 time: 1.0688879