How to fetch metric values with the Python API?

asked 2014-09-29 17:53:37 -0600

Dan Zerkle gravatar image

I've been assigned a project to develop a system to monitor an OpenStack environment with standard real-time metrics such as CPU usage, memory usage, and disk usage. I need separate metrics for the hypervisors and virtual machines. I'm trying to us the Python API to do this. Unfortunately, the documentation on how to do this appears to be mostly non-existent.

Can anyone explain how to do this? I'm currently learning both OpenStack and Python, so please avoid or explain any confusing jargon.

I've read about nova and ceilometer. I have some scripts that create nova and ceilometer clients then fetch data with code like this:

samples = ceilometer_client.samples.list()
for s in sample:
    print s;

This outputs some objects in JSON format, which at least lets me see their content.

I've been able to get a list of meters. I haven't, however, found the actual real-time data. Where is it hiding? Should I be using Nova or Ceilometer or something else? Also, I'll need a list of the actual virtual machines.

Also, it appears that the objects that come back have different sets of attributes. Is there a way in Python to determine what's there?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2014-09-30 09:20:47 -0600

larsks gravatar image

updated 2014-09-30 11:44:29 -0600

The client.samples.list() method is how you get the actual data. Consider this:

for sample in cc.samples.list(meter_name='cpu_util', limit=10):
    print sample

The first output from this code -- on my system -- is:

<Sample {u'counter_name': u'cpu_util', u'user_id': u'564bafe6f37f4d88bd028c12d47d6dde', u'resource_id': u'dd8395e8-f351-449f-a684-75e4cc26b957', u'timestamp': u'2014-09-30T13:58:05', u'counter_volume': 100.016666666667, u'resource_metadata': {u'ephemeral_gb': u'0', u'flavor.vcpus': u'1', u'OS-EXT-AZ.availability_zone': u'nova', u'display_name': u'f21orig', u'': u'2', u'ramdisk_id': u'None', u'': u'm1.small', u'disk_gb': u'20', u'kernel_id': u'None', u'': u'69d074b1-40b4-4834-aebe-92ae3b295d7f', u'flavor.ram': u'2048', u'host': u'ead1d5de73d7cffd0ed28d757dc8e612fbd642f37a99b9e2abba61c3', u'flavor.ephemeral': u'0', u'': u'fedora-21-cloud-orig', u'image_ref_url': u'', u'image.links': u'["{u\'href\': u\'\', u\'rel\': u\'bookmark\'}"]', u'cpu_number': u'1', u'flavor.disk': u'20', u'root_gb': u'20', u'name': u'instance-0000024f', u'memory_mb': u'2048', u'instance_type': u'2', u'vcpus': u'1', u'image_ref': u'69d074b1-40b4-4834-aebe-92ae3b295d7f', u'flavor.links': u'["{u\'href\': u\'\', u\'rel\': u\'bookmark\'}"]'}, u'source': u'openstack', u'counter_unit': u'%', u'recorded_at': u'2014-09-30T13:58:05.646141', u'project_id': u'95d9bbd9b446438a89a353d8adb60704', u'message_id': u'cd23df2a-48a9-11e4-be67-a2f475909d4a', u'counter_type': u'gauge'}>

This is not JSON data. This is the Python repr() of a Sample object, which is for the most part the string representation of a Python dictionary. We can use the to_dict() method of the Sample class to get an actual dictionary back, and then use the pprint module to format it nicely:

{u'counter_name': u'cpu_util',
 u'counter_type': u'gauge',
 u'counter_unit': u'%',
 u'counter_volume': 100.016666666667,
 u'message_id': u'cd23df2a-48a9-11e4-be67-a2f475909d4a',
 u'project_id': u'95d9bbd9b446438a89a353d8adb60704',
 u'recorded_at': u'2014-09-30T13:58:05.646141',
 u'resource_id': u'dd8395e8-f351-449f-a684-75e4cc26b957',
 u'resource_metadata': {u'OS-EXT-AZ.availability_zone': u'nova',
                        u'cpu_number': u'1',
                        u'disk_gb': u'20',
                        u'display_name': u'f21orig',
                        u'ephemeral_gb': u'0',
                        u'flavor.disk': u'20',
                        u'flavor.ephemeral': u'0',
                        u'': u'2',
                        u'flavor.links': u'["{u\'href\': u\'\', u\'rel\': u\'bookmark\'}"]',
                        u'': u'm1.small',
                        u'flavor.ram': u'2048',
                        u'flavor.vcpus': u'1',
                        u'host': u'ead1d5de73d7cffd0ed28d757dc8e612fbd642f37a99b9e2abba61c3',
                        u'': u'69d074b1-40b4-4834-aebe-92ae3b295d7f',
                        u'image.links': u'["{u\'href\': u\'\', u\'rel\': u\'bookmark\'}"]',
                        u'': u'fedora-21-cloud-orig',
                        u'image_ref': u'69d074b1-40b4-4834-aebe-92ae3b295d7f',
                        u'image_ref_url': u'',
                        u'instance_type': u'2',
                        u'kernel_id': u'None',
                        u'memory_mb': u'2048',
                        u'name': u'instance-0000024f',
                        u'ramdisk_id': u'None',
                        u'root_gb': u'20',
                        u'vcpus': u'1'},
 u'source': u'openstack',
 u'timestamp': u'2014-09-30T13:58:05',
 u'user_id': u'564bafe6f37f4d88bd028c12d47d6dde'}

You can see here the meter name (cpu_util) and the ... (more)

edit flag offensive delete link more


This is tremendously helpful. Thank you. However, I do have more questions.

I need the current value of the meters. Do I use limit=1 to get that?

The metrics need to be tied to the virtual machine host name. Is this what the resource_id field is for? How do I get a list of VM servers?

Dan Zerkle gravatar imageDan Zerkle ( 2014-09-30 10:41:50 -0600 )edit

The data for cpu_util seems to be in the counter_volume field. Is it the same for other meters, or is it in a field with a different name? Is there a way to find the correct field?

Dan Zerkle gravatar imageDan Zerkle ( 2014-09-30 10:50:06 -0600 )edit

I'm experimenting.

Setting limit=1 won't work directly because the list() call comes back with separate data for each CPU and each resource_id. Maybe it will work in combination with a query somehow.

Dan Zerkle gravatar imageDan Zerkle ( 2014-09-30 11:34:58 -0600 )edit

I've updated the answer in repsonse to your questions. If you've found this answer helpful, consider marking it as "accepted". Thanks!

larsks gravatar imagelarsks ( 2014-09-30 11:43:53 -0600 )edit

+1 for a nice explanation..

Syed Awais Ali gravatar imageSyed Awais Ali ( 2014-09-30 13:17:26 -0600 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools



Asked: 2014-09-29 17:52:34 -0600

Seen: 1,276 times

Last updated: Sep 30 '14