Revision history [back]

click to hide/show revision 1
initial version

trove NotFound: The resource could not be found. (HTTP 404)

i can make trove to work and i see this error in trove-taskmanager.log

2016-11-14 11:25:17.331 14817 DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: 49d5bbcd8c9f48a6a9639ed07ecd71e9 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:196 2016-11-14 11:25:17.344 14817 DEBUG trove.instance.models [-] Instance eec6047c-a7d2-4c73-a654-ea134503b2cd service status is new. load_instance /usr/lib/python2.7/site-packages/trove/instance/models.py:526 2016-11-14 11:25:17.352 14817 INFO trove.taskmanager.models [-] Creating instance eec6047c-a7d2-4c73-a654-ea134503b2cd. 2016-11-14 11:25:17.353 14817 DEBUG neutronclient.client [-] REQ: curl -i http://controller01:9696//v2.0/security-groups.json -X POST -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA1}73920586ff2340e6e4ba85264ee4848e1bde3cb7" -d '{"security_group": {"name": "SecGroup_eec6047c-a7d2-4c73-a654-ea134503b2cd", "description": "Security Group for eec6047c-a7d2-4c73-a654-ea134503b2cd"}}' http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:185 2016-11-14 11:25:17.579 14817 DEBUG neutronclient.client [-] RESP: 201 {'Date': 'Mon, 14 Nov 2016 09:25:17 GMT', 'Content-Length': '1396', 'Connection': 'keep-alive', 'Content-Type': 'application/json; charset=UTF-8', 'X-Openstack-Request-Id': 'req-e4984d4f-ec0f-4b3d-a320-ecff65ac7c7a'} {"security_group": {"description": "Security Group for eec6047c-a7d2-4c73-a654-ea134503b2cd", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "created_at": "2016-11-14T09:25:17Z", "updated_at": "2016-11-14T09:25:17Z", "security_group_rules": [{"direction": "egress", "protocol": null, "description": null, "port_range_max": null, "updated_at": "2016-11-14T09:25:17Z", "revision_number": 1, "id": "9f96c6c5-423b-4a18-b13b-21b37eb2b00d", "remote_group_id": null, "remote_ip_prefix": null, "created_at": "2016-11-14T09:25:17Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": null, "ethertype": "IPv4", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}, {"direction": "egress", "protocol": null, "description": null, "port_range_max": null, "updated_at": "2016-11-14T09:25:17Z", "revision_number": 1, "id": "f51520a1-8fb5-4c2c-a7f7-f297989d7e5c", "remote_group_id": null, "remote_ip_prefix": null, "created_at": "2016-11-14T09:25:17Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": null, "ethertype": "IPv6", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}], "revision_number": 1, "project_id": "1c7b790d9fd148ef97008ba3c153c1ed", "id": "b9e12174-4175-4040-accd-53c65ef780d0", "name": "SecGroup_eec6047c-a7d2-4c73-a654-ea134503b2cd"}} http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:194 2016-11-14 11:25:17.579 14817 DEBUG neutronclient.v2_0.client [-] POST call to neutron for http://controller01:9696//v2.0/security-groups.json used request id req-e4984d4f-ec0f-4b3d-a320-ecff65ac7c7a _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127 2016-11-14 11:25:17.580 14817 DEBUG trove.db.models [-] Saving SecurityGroup: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 17, 580723), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5612650="">, u'description': u'Security Group for eec6047c-a7d2-4c73-a654-ea134503b2cd', u'created': datetime.datetime(2016, 11, 14, 9, 25, 17, 580554), u'deleted': False, u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', 'errors': {}, u'user': u'8f8e3296fec7438fad4dec5806b6e489', u'id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'name': u'SecGroup_eec6047c-a7d2-4c73-a654-ea134503b2cd'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:17.614 14817 DEBUG trove.db.models [-] Saving SecurityGroupInstanceAssociation: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 17, 614188), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x561a810="">, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17, 614077), u'deleted': False, u'security_group_id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'instance_id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', 'errors': {}, u'id': 'ba102e5d-e733-4181-8db5-9f64f6a8ae8c'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:17.647 14817 DEBUG neutronclient.client [-] REQ: curl -i http://controller01:9696//v2.0/security-group-rules.json -X POST -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA1}73920586ff2340e6e4ba85264ee4848e1bde3cb7" -d '{"security_group_rule": {"direction": "ingress", "protocol": "tcp", "remote_ip_prefix": "0.0.0.0/0", "port_range_max": 3306, "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "port_range_min": 3306, "ethertype": "IPv4"}}' http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:185 2016-11-14 11:25:17.869 14817 DEBUG neutronclient.client [-] RESP: 201 {'Date': 'Mon, 14 Nov 2016 09:25:17 GMT', 'Content-Length': '518', 'Connection': 'keep-alive', 'Content-Type': 'application/json; charset=UTF-8', 'X-Openstack-Request-Id': 'req-0574ab50-1d90-46db-a396-6c9e96b82e4a'} {"security_group_rule": {"direction": "ingress", "protocol": "tcp", "description": "", "port_range_max": 3306, "updated_at": "2016-11-14T09:25:17Z", "revision_number": 1, "id": "52943d65-bb10-447f-82d0-00723c37246a", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "created_at": "2016-11-14T09:25:17Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": 3306, "ethertype": "IPv4", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}} http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:194 2016-11-14 11:25:17.870 14817 DEBUG neutronclient.v2_0.client [-] POST call to neutron for http://controller01:9696//v2.0/security-group-rules.json used request id req-0574ab50-1d90-46db-a396-6c9e96b82e4a _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127 2016-11-14 11:25:17.871 14817 DEBUG trove.db.models [-] Saving SecurityGroupRule: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 17, 871003), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5604c90="">, u'protocol': 'tcp', 'errors': {}, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17, 870729), u'deleted': False, u'to_port': 3306, u'from_port': 3306, u'cidr': '0.0.0.0/0', u'group_id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'id': u'52943d65-bb10-447f-82d0-00723c37246a'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:17.911 14817 DEBUG neutronclient.client [-] REQ: curl -i http://controller01:9696//v2.0/security-group-rules.json -X POST -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA1}73920586ff2340e6e4ba85264ee4848e1bde3cb7" -d '{"security_group_rule": {"direction": "ingress", "protocol": "icmp", "remote_ip_prefix": "0.0.0.0/0", "port_range_max": null, "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "port_range_min": null, "ethertype": "IPv4"}}' http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:185 2016-11-14 11:25:18.100 14817 DEBUG neutronclient.client [-] RESP: 201 {'Date': 'Mon, 14 Nov 2016 09:25:18 GMT', 'Content-Length': '519', 'Connection': 'keep-alive', 'Content-Type': 'application/json; charset=UTF-8', 'X-Openstack-Request-Id': 'req-ef02499d-f453-4181-80a3-911651d3adda'} {"security_group_rule": {"direction": "ingress", "protocol": "icmp", "description": "", "port_range_max": null, "updated_at": "2016-11-14T09:25:18Z", "revision_number": 1, "id": "0cec7f35-1cb6-4744-be3f-8a11ddb31f8a", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "created_at": "2016-11-14T09:25:18Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": null, "ethertype": "IPv4", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}} http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:194 2016-11-14 11:25:18.101 14817 DEBUG neutronclient.v2_0.client [-] POST call to neutron for http://controller01:9696//v2.0/security-group-rules.json used request id req-ef02499d-f453-4181-80a3-911651d3adda _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127 2016-11-14 11:25:18.101 14817 DEBUG trove.db.models [-] Saving SecurityGroupRule: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 18, 101930), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5604c50="">, u'protocol': 'icmp', 'errors': {}, u'created': datetime.datetime(2016, 11, 14, 9, 25, 18, 101806), u'deleted': False, u'to_port': None, u'from_port': None, u'cidr': '0.0.0.0/0', u'group_id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'id': u'0cec7f35-1cb6-4744-be3f-8a11ddb31f8a'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:18.150 14817 DEBUG trove.taskmanager.models [-] Successfully created security group for instance: eec6047c-a7d2-4c73-a654-ea134503b2cd create_instance /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:396 2016-11-14 11:25:18.151 14817 DEBUG trove.taskmanager.models [-] Begin _create_server_volume_individually for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_server_volume_individually /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:800 2016-11-14 11:25:18.152 14817 DEBUG trove.taskmanager.models [-] trove volume support = True _build_volume_info /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:829 2016-11-14 11:25:18.152 14817 DEBUG trove.taskmanager.models [-] Begin _create_volume for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:864 2016-11-14 11:25:18.518 14817 DEBUG trove.db.models [-] Saving DBInstance: {u'cluster_id': None, u'shard_id': None, u'deleted_at': None, u'id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', u'datastore_version_id': u'0c7c0b64-7930-44e8-be2f-2c80c2e585af', 'errors': {}, u'hostname': None, u'server_status': None, u'task_description': u'The instance is building.', u'volume_size': 5, u'type': None, u'updated': datetime.datetime(2016, 11, 14, 9, 25, 18, 518209), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5637f10="">, u'deleted': 0, u'configuration_id': None, u'volume_id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', u'slave_of_id': None, u'task_start_time': None, u'name': u'mysql_instance_4', u'task_id': 5, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17), u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', u'compute_instance_id': None, u'flavor_id': u'751fac89-5d4c-44da-ad31-32e46575fa88'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:20.750 14817 DEBUG trove.taskmanager.models [-] End _create_volume for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:884 2016-11-14 11:25:20.750 14817 DEBUG trove.taskmanager.models [-] Created volume <volume: 050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a=""> _build_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:888 2016-11-14 11:25:20.751 14817 DEBUG trove.taskmanager.models [-] block_device = {'vdb': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a::5:1'} volume = [{'id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', 'size': 5}] device_path = /dev/vdb mount_point = /var/lib/mysql _build_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:908 2016-11-14 11:25:21.598 14817 DEBUG trove.taskmanager.models [-] Created new compute instance 2ba8dd5a-acdf-4f47-a8ba-12b7abfc8882 for instance eec6047c-a7d2-4c73-a654-ea134503b2cd _create_server /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:941 2016-11-14 11:25:21.604 14817 DEBUG trove.db.models [-] Saving DBInstance: {u'cluster_id': None, u'shard_id': None, u'deleted_at': None, u'id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', u'datastore_version_id': u'0c7c0b64-7930-44e8-be2f-2c80c2e585af', 'errors': {}, u'hostname': None, u'server_status': None, u'task_description': u'The instance is building.', u'volume_size': 5, u'type': None, u'updated': datetime.datetime(2016, 11, 14, 9, 25, 21, 604496), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x563eb10="">, u'deleted': 0, u'configuration_id': None, u'volume_id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', u'slave_of_id': None, u'task_start_time': None, u'name': u'mysql_instance_4', u'task_id': 5, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17), u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', u'compute_instance_id': u'2ba8dd5a-acdf-4f47-a8ba-12b7abfc8882', u'flavor_id': u'751fac89-5d4c-44da-ad31-32e46575fa88'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:21.644 14817 DEBUG trove.taskmanager.models [-] End _create_server_volume_individually for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_server_volume_individually /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:820 2016-11-14 11:25:21.669 14817 DEBUG trove.taskmanager.models [-] Entering guest_prepare _guest_prepare /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:949 2016-11-14 11:25:21.671 14817 DEBUG trove.guestagent.api [-] Sending the call to prepare the Guest. prepare /usr/lib/python2.7/site-packages/trove/guestagent/api.py:235 2016-11-14 11:25:21.712 14817 DEBUG trove.guestagent.api [-] Casting prepare _cast /usr/lib/python2.7/site-packages/trove/guestagent/api.py:77 2016-11-14 11:25:21.731 14817 DEBUG oslo_messaging._drivers.amqpdriver [-] CAST unique_id: 614f700f4e9747b789c1cc6d68932cd5 exchange 'trove' topic 'guestagent.eec6047c-a7d2-4c73-a654-ea134503b2cd' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:448 2016-11-14 11:25:21.735 14817 INFO trove.instance.models [-] Resetting task status to NONE on instance eec6047c-a7d2-4c73-a654-ea134503b2cd. 2016-11-14 11:25:21.742 14817 DEBUG trove.db.models [-] Saving DBInstance: {u'cluster_id': None, u'shard_id': None, u'deleted_at': None, u'id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', u'datastore_version_id': u'0c7c0b64-7930-44e8-be2f-2c80c2e585af', 'errors': {}, u'hostname': None, u'server_status': None, u'task_description': 'No tasks for the instance.', u'volume_size': 5, u'type': None, u'updated': datetime.datetime(2016, 11, 14, 9, 25, 21, 742320), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x56ae4d0="">, u'deleted': 0, u'configuration_id': None, u'volume_id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', u'slave_of_id': None, u'task_start_time': None, u'name': u'mysql_instance_4', u'task_id': 1, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17), u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', u'compute_instance_id': u'2ba8dd5a-acdf-4f47-a8ba-12b7abfc8882', u'flavor_id': u'751fac89-5d4c-44da-ad31-32e46575fa88'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:21.776 14817 DEBUG trove.taskmanager.models [-] trove dns support = False _create_dns_entry /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:963 2016-11-14 11:25:21.777 14817 DEBUG trove.taskmanager.models [-] <greenlet.greenlet object="" at="" 0x53dd190="">: DNS not enabled for instance: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_dns_entry /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:1004 2016-11-14 11:26:00.394 14817 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.publish_exists_event run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task [-] Error during Manager.publish_exists_event 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task Traceback (most recent call last): 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task task(self, context) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/taskmanager/manager.py", line 429, in publish_exists_event 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task self.admin_context) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 178, in publish_exist_events 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task notifications = transformer() 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 271, in __call__ 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task client=self.nova_client) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 40, in load_mgmt_instances 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task mgmt_servers = client.servers.list(search_opts={'all_tenants': 1}) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 835, in list 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task "servers") 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 249, in _list 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task resp, body = self.api.client.get(url) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 480, in get 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task return self._cs_request(url, 'GET', *kwargs) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 436, in _cs_request 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task self.authenticate() 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 612, in authenticate 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task auth_url = self._v1_auth(auth_url) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 648, in _v1_auth 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task resp, body = self._time_request(url, 'GET', headers=headers) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 431, in _time_request 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task resp, body = self.request(url, method, *kwargs) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 425, in request 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task raise exceptions.from_response(resp, body, url, method) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task NotFound: The resource could not be found. (HTTP 404) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task 2016-11-14 11:27:00.396 14817 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.publish_exists_event run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task [-] Error during Manager.publish_exists_event 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task Traceback (most recent call last): 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task task(self, context) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/taskmanager/manager.py", line 429, in publish_exists_event 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task self.admin_context) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 178, in publish_exist_events 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task notifications = transformer() 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 271, in __call__ 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task client=self.nova_client) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 40, in load_mgmt_instances 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task mgmt_servers = client.servers.list(search_opts={'all_tenants': 1}) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 835, in list 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task "servers") 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 249, in _list 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task resp, body = self.api.client.get(url) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 480, in get 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task return self._cs_request(url, 'GET', *kwargs) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 436, in _cs_request 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task self.authenticate() 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 612, in authenticate 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task auth_url = self._v1_auth(auth_url) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 648, in _v1_auth 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task resp, body = self._time_request(url, 'GET', headers=headers) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 431, in _time_request 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task resp, body = self.request(url, method, *kwargs) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 425, in request 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task raise exceptions.from_response(resp, body, url, method) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task NotFound: The resource could not be found. (HTTP 404) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task

trove NotFound: The resource could not be found. (HTTP 404)

i can cant make trove to work working and i see this error in trove-taskmanager.logtrove-taskmanager.log when i check rabbit i see a message waiting in the queue guestagent.eec6047c-a7d2-4c73-a654-ea134503b2cd but no consumer

2016-11-14 11:25:17.331 14817 DEBUG oslo_messaging._drivers.amqpdriver [-] received message with unique_id: 49d5bbcd8c9f48a6a9639ed07ecd71e9 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:196 2016-11-14 11:25:17.344 14817 DEBUG trove.instance.models [-] Instance eec6047c-a7d2-4c73-a654-ea134503b2cd service status is new. load_instance /usr/lib/python2.7/site-packages/trove/instance/models.py:526 2016-11-14 11:25:17.352 14817 INFO trove.taskmanager.models [-] Creating instance eec6047c-a7d2-4c73-a654-ea134503b2cd. 2016-11-14 11:25:17.353 14817 DEBUG neutronclient.client [-] REQ: curl -i http://controller01:9696//v2.0/security-groups.json -X POST -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA1}73920586ff2340e6e4ba85264ee4848e1bde3cb7" -d '{"security_group": {"name": "SecGroup_eec6047c-a7d2-4c73-a654-ea134503b2cd", "description": "Security Group for eec6047c-a7d2-4c73-a654-ea134503b2cd"}}' http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:185 2016-11-14 11:25:17.579 14817 DEBUG neutronclient.client [-] RESP: 201 {'Date': 'Mon, 14 Nov 2016 09:25:17 GMT', 'Content-Length': '1396', 'Connection': 'keep-alive', 'Content-Type': 'application/json; charset=UTF-8', 'X-Openstack-Request-Id': 'req-e4984d4f-ec0f-4b3d-a320-ecff65ac7c7a'} {"security_group": {"description": "Security Group for eec6047c-a7d2-4c73-a654-ea134503b2cd", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "created_at": "2016-11-14T09:25:17Z", "updated_at": "2016-11-14T09:25:17Z", "security_group_rules": [{"direction": "egress", "protocol": null, "description": null, "port_range_max": null, "updated_at": "2016-11-14T09:25:17Z", "revision_number": 1, "id": "9f96c6c5-423b-4a18-b13b-21b37eb2b00d", "remote_group_id": null, "remote_ip_prefix": null, "created_at": "2016-11-14T09:25:17Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": null, "ethertype": "IPv4", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}, {"direction": "egress", "protocol": null, "description": null, "port_range_max": null, "updated_at": "2016-11-14T09:25:17Z", "revision_number": 1, "id": "f51520a1-8fb5-4c2c-a7f7-f297989d7e5c", "remote_group_id": null, "remote_ip_prefix": null, "created_at": "2016-11-14T09:25:17Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": null, "ethertype": "IPv6", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}], "revision_number": 1, "project_id": "1c7b790d9fd148ef97008ba3c153c1ed", "id": "b9e12174-4175-4040-accd-53c65ef780d0", "name": "SecGroup_eec6047c-a7d2-4c73-a654-ea134503b2cd"}} http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:194 2016-11-14 11:25:17.579 14817 DEBUG neutronclient.v2_0.client [-] POST call to neutron for http://controller01:9696//v2.0/security-groups.json used request id req-e4984d4f-ec0f-4b3d-a320-ecff65ac7c7a _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127 2016-11-14 11:25:17.580 14817 DEBUG trove.db.models [-] Saving SecurityGroup: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 17, 580723), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5612650="">, u'description': u'Security Group for eec6047c-a7d2-4c73-a654-ea134503b2cd', u'created': datetime.datetime(2016, 11, 14, 9, 25, 17, 580554), u'deleted': False, u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', 'errors': {}, u'user': u'8f8e3296fec7438fad4dec5806b6e489', u'id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'name': u'SecGroup_eec6047c-a7d2-4c73-a654-ea134503b2cd'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:17.614 14817 DEBUG trove.db.models [-] Saving SecurityGroupInstanceAssociation: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 17, 614188), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x561a810="">, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17, 614077), u'deleted': False, u'security_group_id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'instance_id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', 'errors': {}, u'id': 'ba102e5d-e733-4181-8db5-9f64f6a8ae8c'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:17.647 14817 DEBUG neutronclient.client [-] REQ: curl -i http://controller01:9696//v2.0/security-group-rules.json -X POST -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA1}73920586ff2340e6e4ba85264ee4848e1bde3cb7" -d '{"security_group_rule": {"direction": "ingress", "protocol": "tcp", "remote_ip_prefix": "0.0.0.0/0", "port_range_max": 3306, "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "port_range_min": 3306, "ethertype": "IPv4"}}' http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:185 2016-11-14 11:25:17.869 14817 DEBUG neutronclient.client [-] RESP: 201 {'Date': 'Mon, 14 Nov 2016 09:25:17 GMT', 'Content-Length': '518', 'Connection': 'keep-alive', 'Content-Type': 'application/json; charset=UTF-8', 'X-Openstack-Request-Id': 'req-0574ab50-1d90-46db-a396-6c9e96b82e4a'} {"security_group_rule": {"direction": "ingress", "protocol": "tcp", "description": "", "port_range_max": 3306, "updated_at": "2016-11-14T09:25:17Z", "revision_number": 1, "id": "52943d65-bb10-447f-82d0-00723c37246a", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "created_at": "2016-11-14T09:25:17Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": 3306, "ethertype": "IPv4", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}} http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:194 2016-11-14 11:25:17.870 14817 DEBUG neutronclient.v2_0.client [-] POST call to neutron for http://controller01:9696//v2.0/security-group-rules.json used request id req-0574ab50-1d90-46db-a396-6c9e96b82e4a _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127 2016-11-14 11:25:17.871 14817 DEBUG trove.db.models [-] Saving SecurityGroupRule: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 17, 871003), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5604c90="">, u'protocol': 'tcp', 'errors': {}, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17, 870729), u'deleted': False, u'to_port': 3306, u'from_port': 3306, u'cidr': '0.0.0.0/0', u'group_id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'id': u'52943d65-bb10-447f-82d0-00723c37246a'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:17.911 14817 DEBUG neutronclient.client [-] REQ: curl -i http://controller01:9696//v2.0/security-group-rules.json -X POST -H "User-Agent: python-neutronclient" -H "X-Auth-Token: {SHA1}73920586ff2340e6e4ba85264ee4848e1bde3cb7" -d '{"security_group_rule": {"direction": "ingress", "protocol": "icmp", "remote_ip_prefix": "0.0.0.0/0", "port_range_max": null, "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "port_range_min": null, "ethertype": "IPv4"}}' http_log_req /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:185 2016-11-14 11:25:18.100 14817 DEBUG neutronclient.client [-] RESP: 201 {'Date': 'Mon, 14 Nov 2016 09:25:18 GMT', 'Content-Length': '519', 'Connection': 'keep-alive', 'Content-Type': 'application/json; charset=UTF-8', 'X-Openstack-Request-Id': 'req-ef02499d-f453-4181-80a3-911651d3adda'} {"security_group_rule": {"direction": "ingress", "protocol": "icmp", "description": "", "port_range_max": null, "updated_at": "2016-11-14T09:25:18Z", "revision_number": 1, "id": "0cec7f35-1cb6-4744-be3f-8a11ddb31f8a", "remote_group_id": null, "remote_ip_prefix": "0.0.0.0/0", "created_at": "2016-11-14T09:25:18Z", "security_group_id": "b9e12174-4175-4040-accd-53c65ef780d0", "tenant_id": "1c7b790d9fd148ef97008ba3c153c1ed", "port_range_min": null, "ethertype": "IPv4", "project_id": "1c7b790d9fd148ef97008ba3c153c1ed"}} http_log_resp /usr/lib/python2.7/site-packages/neutronclient/common/utils.py:194 2016-11-14 11:25:18.101 14817 DEBUG neutronclient.v2_0.client [-] POST call to neutron for http://controller01:9696//v2.0/security-group-rules.json used request id req-ef02499d-f453-4181-80a3-911651d3adda _append_request_id /usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py:127 2016-11-14 11:25:18.101 14817 DEBUG trove.db.models [-] Saving SecurityGroupRule: {u'updated': datetime.datetime(2016, 11, 14, 9, 25, 18, 101930), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5604c50="">, u'protocol': 'icmp', 'errors': {}, u'created': datetime.datetime(2016, 11, 14, 9, 25, 18, 101806), u'deleted': False, u'to_port': None, u'from_port': None, u'cidr': '0.0.0.0/0', u'group_id': u'b9e12174-4175-4040-accd-53c65ef780d0', u'id': u'0cec7f35-1cb6-4744-be3f-8a11ddb31f8a'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:18.150 14817 DEBUG trove.taskmanager.models [-] Successfully created security group for instance: eec6047c-a7d2-4c73-a654-ea134503b2cd create_instance /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:396 2016-11-14 11:25:18.151 14817 DEBUG trove.taskmanager.models [-] Begin _create_server_volume_individually for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_server_volume_individually /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:800 2016-11-14 11:25:18.152 14817 DEBUG trove.taskmanager.models [-] trove volume support = True _build_volume_info /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:829 2016-11-14 11:25:18.152 14817 DEBUG trove.taskmanager.models [-] Begin _create_volume for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:864 2016-11-14 11:25:18.518 14817 DEBUG trove.db.models [-] Saving DBInstance: {u'cluster_id': None, u'shard_id': None, u'deleted_at': None, u'id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', u'datastore_version_id': u'0c7c0b64-7930-44e8-be2f-2c80c2e585af', 'errors': {}, u'hostname': None, u'server_status': None, u'task_description': u'The instance is building.', u'volume_size': 5, u'type': None, u'updated': datetime.datetime(2016, 11, 14, 9, 25, 18, 518209), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x5637f10="">, u'deleted': 0, u'configuration_id': None, u'volume_id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', u'slave_of_id': None, u'task_start_time': None, u'name': u'mysql_instance_4', u'task_id': 5, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17), u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', u'compute_instance_id': None, u'flavor_id': u'751fac89-5d4c-44da-ad31-32e46575fa88'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:20.750 14817 DEBUG trove.taskmanager.models [-] End _create_volume for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:884 2016-11-14 11:25:20.750 14817 DEBUG trove.taskmanager.models [-] Created volume <volume: 050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a=""> _build_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:888 2016-11-14 11:25:20.751 14817 DEBUG trove.taskmanager.models [-] block_device = {'vdb': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a::5:1'} volume = [{'id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', 'size': 5}] device_path = /dev/vdb mount_point = /var/lib/mysql _build_volume /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:908 2016-11-14 11:25:21.598 14817 DEBUG trove.taskmanager.models [-] Created new compute instance 2ba8dd5a-acdf-4f47-a8ba-12b7abfc8882 for instance eec6047c-a7d2-4c73-a654-ea134503b2cd _create_server /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:941 2016-11-14 11:25:21.604 14817 DEBUG trove.db.models [-] Saving DBInstance: {u'cluster_id': None, u'shard_id': None, u'deleted_at': None, u'id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', u'datastore_version_id': u'0c7c0b64-7930-44e8-be2f-2c80c2e585af', 'errors': {}, u'hostname': None, u'server_status': None, u'task_description': u'The instance is building.', u'volume_size': 5, u'type': None, u'updated': datetime.datetime(2016, 11, 14, 9, 25, 21, 604496), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x563eb10="">, u'deleted': 0, u'configuration_id': None, u'volume_id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', u'slave_of_id': None, u'task_start_time': None, u'name': u'mysql_instance_4', u'task_id': 5, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17), u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', u'compute_instance_id': u'2ba8dd5a-acdf-4f47-a8ba-12b7abfc8882', u'flavor_id': u'751fac89-5d4c-44da-ad31-32e46575fa88'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:21.644 14817 DEBUG trove.taskmanager.models [-] End _create_server_volume_individually for id: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_server_volume_individually /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:820 2016-11-14 11:25:21.669 14817 DEBUG trove.taskmanager.models [-] Entering guest_prepare _guest_prepare /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:949 2016-11-14 11:25:21.671 14817 DEBUG trove.guestagent.api [-] Sending the call to prepare the Guest. prepare /usr/lib/python2.7/site-packages/trove/guestagent/api.py:235 2016-11-14 11:25:21.712 14817 DEBUG trove.guestagent.api [-] Casting prepare _cast /usr/lib/python2.7/site-packages/trove/guestagent/api.py:77 2016-11-14 11:25:21.731 14817 DEBUG oslo_messaging._drivers.amqpdriver [-] CAST unique_id: 614f700f4e9747b789c1cc6d68932cd5 exchange 'trove' topic 'guestagent.eec6047c-a7d2-4c73-a654-ea134503b2cd' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:448 2016-11-14 11:25:21.735 14817 INFO trove.instance.models [-] Resetting task status to NONE on instance eec6047c-a7d2-4c73-a654-ea134503b2cd. 2016-11-14 11:25:21.742 14817 DEBUG trove.db.models [-] Saving DBInstance: {u'cluster_id': None, u'shard_id': None, u'deleted_at': None, u'id': u'eec6047c-a7d2-4c73-a654-ea134503b2cd', u'datastore_version_id': u'0c7c0b64-7930-44e8-be2f-2c80c2e585af', 'errors': {}, u'hostname': None, u'server_status': None, u'task_description': 'No tasks for the instance.', u'volume_size': 5, u'type': None, u'updated': datetime.datetime(2016, 11, 14, 9, 25, 21, 742320), '_sa_instance_state': <sqlalchemy.orm.state.instancestate object="" at="" 0x56ae4d0="">, u'deleted': 0, u'configuration_id': None, u'volume_id': u'050b21c7-55d7-4cc3-b0ea-ecf5d39b8e0a', u'slave_of_id': None, u'task_start_time': None, u'name': u'mysql_instance_4', u'task_id': 1, u'created': datetime.datetime(2016, 11, 14, 9, 25, 17), u'tenant_id': u'1c7b790d9fd148ef97008ba3c153c1ed', u'compute_instance_id': u'2ba8dd5a-acdf-4f47-a8ba-12b7abfc8882', u'flavor_id': u'751fac89-5d4c-44da-ad31-32e46575fa88'} save /usr/lib/python2.7/site-packages/trove/db/models.py:62 2016-11-14 11:25:21.776 14817 DEBUG trove.taskmanager.models [-] trove dns support = False _create_dns_entry /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:963 2016-11-14 11:25:21.777 14817 DEBUG trove.taskmanager.models [-] <greenlet.greenlet object="" at="" 0x53dd190="">: DNS not enabled for instance: eec6047c-a7d2-4c73-a654-ea134503b2cd _create_dns_entry /usr/lib/python2.7/site-packages/trove/taskmanager/models.py:1004 2016-11-14 11:26:00.394 14817 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.publish_exists_event run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task [-] Error during Manager.publish_exists_event 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task Traceback (most recent call last): 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task task(self, context) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/taskmanager/manager.py", line 429, in publish_exists_event 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task self.admin_context) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 178, in publish_exist_events 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task notifications = transformer() 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 271, in __call__ 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task client=self.nova_client) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 40, in load_mgmt_instances 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task mgmt_servers = client.servers.list(search_opts={'all_tenants': 1}) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 835, in list 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task "servers") 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 249, in _list 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task resp, body = self.api.client.get(url) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 480, in get 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task return self._cs_request(url, 'GET', *kwargs) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 436, in _cs_request 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task self.authenticate() 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 612, in authenticate 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task auth_url = self._v1_auth(auth_url) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 648, in _v1_auth 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task resp, body = self._time_request(url, 'GET', headers=headers) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 431, in _time_request 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task resp, body = self.request(url, method, *kwargs) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 425, in request 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task raise exceptions.from_response(resp, body, url, method) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task NotFound: The resource could not be found. (HTTP 404) 2016-11-14 11:26:00.400 14817 ERROR oslo_service.periodic_task 2016-11-14 11:27:00.396 14817 DEBUG oslo_service.periodic_task [-] Running periodic task Manager.publish_exists_event run_periodic_tasks /usr/lib/python2.7/site-packages/oslo_service/periodic_task.py:215 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task [-] Error during Manager.publish_exists_event 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task Traceback (most recent call last): 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/oslo_service/periodic_task.py", line 220, in run_periodic_tasks 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task task(self, context) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/taskmanager/manager.py", line 429, in publish_exists_event 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task self.admin_context) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 178, in publish_exist_events 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task notifications = transformer() 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 271, in __call__ 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task client=self.nova_client) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/trove/extensions/mgmt/instances/models.py", line 40, in load_mgmt_instances 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task mgmt_servers = client.servers.list(search_opts={'all_tenants': 1}) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 835, in list 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task "servers") 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 249, in _list 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task resp, body = self.api.client.get(url) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 480, in get 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task return self._cs_request(url, 'GET', *kwargs) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 436, in _cs_request 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task self.authenticate() 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 612, in authenticate 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task auth_url = self._v1_auth(auth_url) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 648, in _v1_auth 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task resp, body = self._time_request(url, 'GET', headers=headers) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 431, in _time_request 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task resp, body = self.request(url, method, *kwargs) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 425, in request 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task raise exceptions.from_response(resp, body, url, method) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task NotFound: The resource could not be found. (HTTP 404) 2016-11-14 11:27:00.404 14817 ERROR oslo_service.periodic_task