Ceilometer - Keystone authentication status code 401

asked 2016-04-29 19:05:53 -0500

mitakauser
21 ●2 ●3 ●6

updated 2016-05-02 18:11:34 -0500

Hi everyone,

At work we've been trying to figure out how to get Ceilometer on the compute node to authenticate agasint Keystone but to no avail.

Here's the output of the keystone-wsgi-public log (controller node)

2016-04-29 18:54:19.074 26606 INFO keystone.common.wsgi [req-3e3fab30-5b37-4a22-8ad1-9b42bc46780d - - - - -] POST http://controller:5000/v2.0/tokens
2016-04-29 18:54:19.081 26606 WARNING keystone.common.wsgi [req-3e3fab30-5b37-4a22-8ad1-9b42bc46780d - - - - -] Authorization failed. The request you have made requires authentication. from xx.xx.xx.xx
2016-04-29 18:54:19.090 26604 DEBUG keystone.middleware.auth [req-08ae6a8a-4584-4033-9bd0-904d20cb6f1f - - - - -] There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set. _build_auth_context /usr/lib/python2.7/dist-packages/keystone/middleware/auth.py:71

And here's the output of the ceilometer-compute.log (compute node)

2016-04-29 18:35:26.708 24510 DEBUG ceilometer.pipeline [req-34e15155-ebb3-4a29-88ab-c6869c6693c8 admin - - - -] Polling config file: /etc/ceilometer/pipeline.yaml _setup_polling_manager /usr/lib/python2.7/dist-packages/ceilometer/pipeline.py:804
2016-04-29 18:35:26.723 24510 INFO ceilometer.pipeline [req-34e15155-ebb3-4a29-88ab-c6869c6693c8 admin - - - -] Pipeline config: {'sources': [{'interval': 600, 'meters': ['*'], 'name': 'meter_source', 'sinks': ['meter_sink']}, {'interval': 600, 'meters': ['cpu'], 'name': 'cpu_source', 'sinks': ['cpu_sink', 'cpu_delta_sink']}, {'interval': 600, 'meters': ['disk.read.bytes', 'disk.read.requests', 'disk.write.bytes', 'disk.write.requests', 'disk.device.read.bytes', 'disk.device.read.requests', 'disk.device.write.bytes', 'disk.device.write.requests'], 'name': 'disk_source', 'sinks': ['disk_sink']}, {'interval': 600, 'meters': ['network.incoming.bytes', 'network.incoming.packets', 'network.outgoing.bytes', 'network.outgoing.packets'], 'name': 'network_source', 'sinks': ['network_sink']}], 'sinks': [{'publishers': ['notifier://'], 'transformers': None, 'name': 'meter_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'rate_of_change', 'parameters': {'target': {'scale': '100.0 / (10**9 * (resource_metadata.cpu_number or 1))', 'type': 'gauge', 'name': 'cpu_util', 'unit': '%'}}}], 'name': 'cpu_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'delta', 'parameters': {'target': {'name': 'cpu.delta'}, 'growth_only': True}}], 'name': 'cpu_delta_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'rate_of_change', 'parameters': {'source': {'map_from': {'name': '(disk\\.device|disk)\\.(read|write)\\.(bytes|requests)', 'unit': '(B|request)'}}, 'target': {'type': 'gauge', 'map_to': {'name': '\\1.\\2.\\3.rate', 'unit': '\\1/s'}}}}], 'name': 'disk_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'rate_of_change', 'parameters': {'source': {'map_from': {'name': 'network\\.(incoming|outgoing)\\.(bytes|packets)', 'unit': '(B|packet)'}}, 'target': {'type': 'gauge', 'map_to': {'name': 'network.\\1.\\2.rate', 'unit': '\\1/s'}}}}], 'name': 'network_sink'}]}
2016-04-29 18:35:26.724 24510 INFO ceilometer.pipeline [req-34e15155-ebb3-4a29-88ab-c6869c6693c8 admin - - - -] detected decoupled pipeline config format
2016-04-29 18:35:26.728 24510 DEBUG keystoneauth.identity.v2 [req-34e15155-ebb3-4a29-88ab-c6869c6693c8 admin - - - -] Making authentication request to http://controller:35357/v2.0/tokens get_auth_ref /usr/lib/python2.7/dist-packages/keystoneauth1/identity/v2.py:63
2016-04-29 18:35:29.135 24510 DEBUG keystoneauth.session [req-34e15155-ebb3-4a29-88ab-c6869c6693c8 admin - - - -] Request returned failure status: 401 request /usr/lib/python2.7/dist-packages/keystoneauth1/session.py:466
2016-04-29 18:35:29.136 24510 ERROR ceilometer.nova_client [req-34e15155-ebb3-4a29-88ab-c6869c6693c8 admin - - - -] The request you have made requires authentication. (HTTP 401) (Request-ID: req-128a4b95-3d5d-486c-bab1-1a9be6f676d1)
2016-04-29 18:35:29.136 24510 ERROR ceilometer.nova_client Traceback (most recent call last):
2016-04-29 18:35:29.136 24510 ERROR ceilometer.nova_client   File "/usr/lib/python2.7/dist-packages/ceilometer/nova_client.py", line 52, in with_logging
2016-04-29 18:35:29.136 24510 ERROR ceilometer.nova_client     return func(*args ...

(more)

edit retag flag offensive close merge delete

add a comment

4 answers

Sort by » oldest newest most voted

answered 2016-05-07 07:38:48 -0500

todotani

31 ●2 ●4 ●8

updated 2016-05-10 09:02:53 -0500

After I changed service_credentials as @mitakuser mentioned, keystone authentication error was gone, but instead I got a following error. Don't you have the same error?

modified config
auth_url = http://controller:5000/v3

=== edit ===
My configuration change was not complete as @mitakuser was instructed. With the following configuration everything working fine. I needed to define project_domain_name and user_domain_name not project_domain_id and user_domain_id. Anyway it is documentation error.

[service_credentials]
#os_auth_url = http://nuc1:5000/v2.0
#os_username = ceilometer
#os_tenant_name = service
#os_password = xxxxxxx
auth_url = http://nuc1:5000/v3
username = ceilometer
tenant_name = service
password = xxxxxx
project_name = service
project_domain_name = default
user_domain_name = default
auth_type = password

======= error in /var/logceilometer/compute.log ===========

2016-05-07 21:03:20.723 959 DEBUG ceilometer.pipeline [req-724c4a89-264c-482e-8f73-60eff7190848 admin - - - -] Polling config file: /etc/ceilometer/pipeline.yaml _setup_polling_manager /usr/lib/python2.7/site-packages/ceilometer/pipeline.py:804
2016-05-07 21:03:20.768 959 INFO ceilometer.pipeline [req-724c4a89-264c-482e-8f73-60eff7190848 admin - - - -] Pipeline config: {'sources': [{'interval': 600, 'meters': ['*'], 'name': 'meter_source', 'sinks': ['meter_sink']}, {'interval': 600, 'meters': ['cpu'], 'name': 'cpu_source', 'sinks': ['cpu_sink', 'cpu_delta_sink']}, {'interval': 600, 'meters': ['disk.read.bytes', 'disk.read.requests', 'disk.write.bytes', 'disk.write.requests', 'disk.device.read.bytes', 'disk.device.read.requests', 'disk.device.write.bytes', 'disk.device.write.requests'], 'name': 'disk_source', 'sinks': ['disk_sink']}, {'interval': 600, 'meters': ['network.incoming.bytes', 'network.incoming.packets', 'network.outgoing.bytes', 'network.outgoing.packets'], 'name': 'network_source', 'sinks': ['network_sink']}], 'sinks': [{'publishers': ['notifier://'], 'transformers': None, 'name': 'meter_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'rate_of_change', 'parameters': {'target': {'scale': '100.0 / (10**9 * (resource_metadata.cpu_number or 1))', 'type': 'gauge', 'name': 'cpu_util', 'unit': '%'}}}], 'name': 'cpu_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'delta', 'parameters': {'target': {'name': 'cpu.delta'}, 'growth_only': True}}], 'name': 'cpu_delta_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'rate_of_change', 'parameters': {'source': {'map_from': {'name': '(disk\\.device|disk)\\.(read|write)\\.(bytes|requests)', 'unit': '(B|request)'}}, 'target': {'type': 'gauge', 'map_to': {'name': '\\1.\\2.\\3.rate', 'unit': '\\1/s'}}}}], 'name': 'disk_sink'}, {'publishers': ['notifier://'], 'transformers': [{'name': 'rate_of_change', 'parameters': {'source': {'map_from': {'name': 'network\\.(incoming|outgoing)\\.(bytes|packets)', 'unit': '(B|packet)'}}, 'target': {'type': 'gauge', 'map_to': {'name': 'network.\\1.\\2.rate', 'unit': '\\1/s'}}}}], 'name': 'network_sink'}]}
2016-05-07 21:03:20.770 959 INFO ceilometer.pipeline [req-724c4a89-264c-482e-8f73-60eff7190848 admin - - - -] detected decoupled pipeline config format
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client [req-724c4a89-264c-482e-8f73-60eff7190848 admin - - - -] The resource could not be found. (HTTP 404) (Request-ID: req-9e8da5aa-5426-4e22-afee-0ab8c1746139)
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client Traceback (most recent call last):
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client   File "/usr/lib/python2.7/site-packages/ceilometer/nova_client.py", line 52, in with_logging
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client     return func(*args, **kwargs)
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client   File "/usr/lib/python2.7/site-packages/ceilometer/nova_client.py", line 157, in instance_get_all_by_host
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client     search_opts=search_opts))
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client   File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 749, in list
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client     "servers")
2016-05-07 21:03:20.803 959 ERROR ceilometer.nova_client   File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 242, in _list
2016-05-07 ...

(more)

edit flag offensive delete link

Comments

@todotani - have U deployed KeystoneV3 ? can U check OS_AUTH_URL form your RC file?

yprokule ( 2016-05-09 01:17:14 -0500 )edit

Hello @yprokule - I'v installed Mitaka from centos-release-openstack-mitaka repository (clean installed).
keystone version is keystone-9.0.0-1.el7
My adminrc contain export OS_AUTH_URL=http://nuc1:35357/v3
Keystone admin endpoint is also configured as http://nuc1:35357/v3

todotani ( 2016-05-09 06:02:32 -0500 )edit

@todotani have you created the keystone API V3 endpoints?. I followed the Ubuntu installation guide but CentOS should be the same. http://docs.openstack.org/mitaka/inst...

mitakauser ( 2016-05-09 11:23:43 -0500 )edit

@mitakauser - I found that my configuration change was not complete as U instructed.
I've changed just v2.0 to v3 but left os_auth_url, os_username and so on.
I've modified such as auth_url, user_name as U instucted and now went well. Thank you for your comment.

todotani ( 2016-05-09 16:25:33 -0500 )edit

add a comment

answered 2016-05-04 13:40:46 -0500

mitakauser
21 ●2 ●3 ●6

I was able to resolve my issue by migrating the ceilometer auth to keystone to v3. This is how the ceilometer service_credentials section looks now

[service_credentials]
auth_url = http://controller:5000/v3
username = ceilometer
tenant_name = service
password = xxxxxx
interface = internalURL
region_name = RegionOne
project_name = service
project_domain_id = xxxxxxxx
user_domain_id = xxxxxxxx
auth_type = password

With this configuration I was able to authenticate to keystone successfully.

edit flag offensive delete link

Comments

@mitakuser - have U installed OpenStack manually or with some installer(e.g. packstack)?

If with some installer - than this is a bug :(

yprokule ( 2016-05-05 01:33:26 -0500 )edit

@mitakauser - I installed mitaka/ceilometer on nova according to Ubuntu installation guide. The parameters you are using are totally different from those on the guide. I tried as you reported and ceilometer started properly without any 401 messages- @mitakauser thank you for posting your solution!

marduk ( 2016-05-05 11:02:03 -0500 )edit

@marduk - interesting. In the example they are referencing to keystone_v2. (http://docs.openstack.org/mitaka/inst...)

os_auth_url = http://controller:5000/v2.0

If U followed the guide this explains misconfiguration

yprokule ( 2016-05-06 05:54:02 -0500 )edit

I can't get nova compute meters, any way to solve this issus?

Hu Fu ( 2016-06-19 21:12:38 -0500 )edit

add a comment

answered 2016-05-02 12:12:24 -0500

marduk
1 ●1 ●1 ●1

Hey @mitakauser, I've a similar issue when configuring ceilometer on a Ubuntu deployment of Mitaka. I see this issue on ceilometer's logs when starting ceilometer-agent-compute service in a nova node.

2016-05-02 12:09:14.864 16871 ERROR ceilometer.nova_client [req-9e6faf45-5f04-4905-be51-68456f700d2d admin - - - -] The request you have made requires authentication. (HTTP 401) (Request-ID: req-d6f8de81-7ae2-4388-8e56-752fbc585e7a)

Do you have any progress?

Thanks In Advance Marduk

edit flag offensive delete link

Comments

@marduk - please check your configuration according - 'http://docs.openstack.org/mitaka/install-guide-rdo/ceilometer-nova.html'

yprokule ( 2016-05-03 02:33:43 -0500 )edit

Yep, I've already configured ceilometer according to this guide - http://docs.openstack.org/mitaka/install-guide-ubuntu/ceilometer-nova.html (http://docs.openstack.org/mitaka/inst...), but not luck yet when restarting the service. Even I've used directly ceilometer-polling with all the required env variables exported.

marduk ( 2016-05-03 12:43:16 -0500 )edit

what are roles for ceilometer and nova users in service tenant? can U do a basic operations with ceilomeer's credentials sourced, e.g: 'ceilometer meter-list, swift auth, neutron net-list'

yprokule ( 2016-05-04 04:48:07 -0500 )edit

Yep, the weird thing is that I was getting the 401 issue even when using the python API directly, with all the env variables exported.

marduk ( 2016-05-05 11:03:48 -0500 )edit

add a comment

answered 2016-05-02 03:17:54 -0500

yprokule
611 ●4 ●8

@mitakuser, how the environment was deployed - manually or by some installer? Do U have a dedicated compute node or this is allinone installation?

Why do U have 2 definitions of auth_uri in ceilometer.conf ?

I'd recommend creating an 'keystonerc' file with ceilometer's credentials and run some nova commands, e.g: nova list, since traceback indicates error with nova_client.

edit flag offensive delete link

Comments

Thanks @yprokule . In the ceilometer.conf there's the auth_uri and then the auth_url definitions. At first I was confused as well. The installation was deployed manually and indeed I have a dedicated compute node for my openstack setup.

mitakauser ( 2016-05-02 11:58:14 -0500 )edit

I have followed your suggestion by creating a keystonerc file with the values in the service credentials section ceilometer.conf and got the same error ERROR (Unauthorized): The request you have made requires authentication. (HTTP 401) it seems parameters are missing...similar to adminopenrc?

mitakauser ( 2016-05-02 12:02:11 -0500 )edit

@mitakauser - copy your adminopenrc and change values for user/tenant/password to match one from the ceilometer.conf.

Also check roles ceilometer user has in service tenant

yprokule ( 2016-05-02 16:22:48 -0500 )edit

add a comment

Ceilometer - Keystone authentication status code 401

4 answers

Comments

Comments

Comments

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

Ceilometer - Keystone authentication status code 401 edit

4 answers

Comments

Comments

Comments

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

Ceilometer - Keystone authentication status code 401