ceilometer dispatcher always get KeyError: 'resource'

asked 2016-08-28 22:54:12 -0600

zero gravatar image

updated 2016-08-29 03:06:37 -0600

yprokule gravatar image

Dear friends

I use devstack to deploy ceilometer with gnocchi env. I met two questions.

Question one: I creates two nova VMs in env. and I found a lot of KeyError: 'resource' in ceilometer-collector service, which occurs at ceilometer dispatchs gnocchi_data.

gnocchi_data looks like this :

2016-07-21 14:59:54.303 23070 DEBUG ceilometer.dispatcher.gnocchi [-] {'318844c8-c8aa-4746-b8ba-dfc8c0210712': {'resource_extra': {'display_name': u't1', 'host': u'520ca6f60edddddc5bf8e758f15ed9fb84e6ccadff36101c215828ed', 'image_ref': u'708dd071-88e0-4178-ad6b-9f88b6cdba30', 'flavor_id': u'1'}, 'resource': {'user_id': u'e8414c6050a2461ba8ca8b4bd303a935', 'metrics': {'disk.write.requests.rate': {'archive_policy_name': 'low'}, 'disk.latency': {'archive_policy_name': 'low'}, 'disk.ephemeral.size': {'archive_policy_name': 'low', 'unit': u'GB'}, 'disk.read.bytes': {'archive_policy_name': 'low'}, 'disk.write.bytes': {'archive_policy_name': 'low'}, 'memory.resident': {'archive_policy_name': 'low', 'unit': u'MB'}, 'instance': {'archive_policy_name': 'low', 'unit': u'instance'}, 'disk.capacity': {'archive_policy_name': 'low'}, 'vcpus': {'archive_policy_name': 'low', 'unit': u'vcpu'}, 'disk.usage': {'archive_policy_name': 'low'}, 'disk.read.bytes.rate': {'archive_policy_name': 'low'}, 'cpu_util': {'archive_policy_name': 'low'}, 'cpu.delta': {'archive_policy_name': 'low'}, 'disk.write.bytes.rate': {'archive_policy_name': 'low'}, 'memory.usage': {'archive_policy_name': 'low'}, 'memory': {'archive_policy_name': 'low', 'unit': u'MB'}, 'disk.read.requests.rate': {'archive_policy_name': 'low'}, 'disk.iops': {'archive_policy_name': 'low'}, 'disk.allocation': {'archive_policy_name': 'low', 'unit': u'B'}, 'disk.write.requests': {'archive_policy_name': 'low', 'unit': u'request'}, 'disk.root.size': {'archive_policy_name': 'low', 'unit': u'GB'}, 'disk.read.requests': {'archive_policy_name': 'low', 'unit': u'request'}, 'cpu': {'archive_policy_name': 'low'}}, 'host': u'520ca6f60edddddc5bf8e758f15ed9fb84e6ccadff36101c215828ed', 'image_ref': u'708dd071-88e0-4178-ad6b-9f88b6cdba30', 'flavor_id': u'1', 'display_name': u't1', 'project_id': u'075ad551ddb44f18b0576dbc9882f7c9', 'id': u'318844c8-c8aa-4746-b8ba-dfc8c0210712'}, 'resource_type': 'instance'}} record_metering_data /opt/stack/ceilometer/ceilometer/dispatcher/gnocchi.py:358

but after this method, self.batch_measures(measures, gnocchi_data, stats)

the gnocchi_data likes this:

2016-07-21 14:59:54.442 23071 DEBUG ceilometer.dispatcher.gnocchi [-] {'318844c8-c8aa-4746-b8ba-dfc8c0210712': {}} record_metering_data /opt/stack/ceilometer/ceilometer/dispatcher/gnocchi.py:369

which cause the KeyError: 'resource'

In fact, I do not know why it behaves like this. then I try pdb to debug, then I got second question here.

Question two: then I try to use pdb to debug ceilometer-collector gnocchi dispatcher (record_metering_data method). however pdb can not stop the process. I check the ceilometer-collector worker in ceilometer.conf, actually it is default 1. it seems a lot messages from messaging-queue are waited.

Any one can give me some hint or helps?

Best Regards

edit retag flag offensive close merge delete