Ask Your Question
1

IceHouse: dashboard problems using multi-domain support

asked 2014-08-25 13:42:52 -0500

echapin gravatar image

updated 2014-08-29 17:43:36 -0500

I am prototyping multi-domain support using an "out-of-the-box" RDO installation of IceHouse on a single machine. Using the default single-domain settings, I have tested and verified that admin and regular users can both log in to the dashboard, view images, launch instances etc. When multi-domain dashboard support is activated, users cannot access any of the services (nova, glance etc.)

In the intial version of this question, I followed http://www.florentflament.com/blog/setting-keystone-v3-domains.html (this guide) to create the cloud_admin user, two domains (dom1, and dom2), as well as a domain administrator (adm1) and user (usr1) on dom1. This is all done using curl rather than python clients. I now realize that the purpose of this work isn't necessarily to enable basic multi-domain support, rather the ability to delegate administration of domains to special domain administrators. While this feature also doesn't work for me, it is off-topic.

Since then I have reverted to a more basic setup to simplify things. Leaving everything else in the default state (including the single-domain keystone/policy.json), I only enable multi-domain support in the dashboard, e.g., in /etc/openstack-dashboard:

OPENSTACK_API_VERSIONS = {
     "identity": 3
}
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
OPENSTACK_KEYSTONE_URL = "http://x.x.x.x:5000/v3"

With this, I can now log in as the admin user on the "default" domain, create new domains, users and projects in those domains etc.

The problem is that when one of these domain users logs in, they cannot do anything (e.g., list images, launch a VM). The initial indication of this problem is when nova is queried to generate the summary page.

I enter "dom1", "usr1" and "password_for_usr1". In the dashboard log I can see that the user correctly authenticated:

2014-08-25 18:27:14,039 7259 DEBUG openstack_auth.backend Authentication completed for user "usr1".
2014-08-25 18:27:14,039 7259 INFO openstack_auth.forms Login successful for user "usr1".

Next, problems with nova:

2014-08-25 18:27:14,093 7260 DEBUG openstack_dashboard.api.nova novaclient connection created using token "xxxxx" and url "http://132.246.194.41:8774/v2/xxxxx"
2014-08-25 18:27:14,330 7260 ERROR django.request Internal Server Error: /dashboard/project/
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 38, in dec
    return view_func(request, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 54, in dec
    return view_func(request, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/horizon/decorators.py", line 38, in dec
    return view_func(request, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 69, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/django/views/generic/base.py", line 87, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/horizon/tables/views.py", line 154, in get
    handled = self.construct_tables()
  File "/usr/lib/python2.7/site-packages/horizon ...
(more)
edit retag flag offensive close merge delete

Comments

This https://bugs.launchpad.net/horizon/+bug/1264483 (bug report) looks similar, which seemed to be caused by https://bugs.launchpad.net/nova/+bug/1154809 (this bug). However, setting auth_version=2.0 in nova config doesn't help.

echapin gravatar imageechapin ( 2014-08-25 16:55:45 -0500 )edit

I just added the output of keystone service-list to the body of the question. As you will see, nothing explicit about v3 (I also don't see anything promising in keystone endpoint-list)

echapin gravatar imageechapin ( 2014-08-26 11:01:38 -0500 )edit

I tried this suggestion - I now see the v3 endpoints listed for keystone (in addition to the v2.0 ones), but otherwise I get the same behaviour (I restarted all of the services for good measure...)

echapin gravatar imageechapin ( 2014-08-27 15:09:41 -0500 )edit

Accidentally deleted this comment first time... anyway, information added. Logs look suspicious, anything I should change in the nova.conf?

echapin gravatar imageechapin ( 2014-08-27 15:48:58 -0500 )edit

Install RDO, verify services, make sure you can get in. Setup the v3 endpoint, edit horizon configuration, verify the default domain. Then see if you can login again. I did a similar setup but it wasn't with an automated install, it was by hand + keystone + ldap + v3 api on Havana.

mpetason gravatar imagempetason ( 2014-08-27 15:52:26 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
1

answered 2014-08-27 12:22:23 -0500

updated 2014-08-29 16:04:55 -0500

If you are using policy.v3cloudsample.json , then set admin_domain_id = default in that policy file.

If you do that, then existing v2 users will continue to work and you can create new domain and add users to the domain. (.e) any user who is in default domain with "admin" role is considered cloudadmin. I don't want to confuse you, but horizon needs to support domain scoped tokens. V3 identity operations which relies on domain information ( domin/users/projects) will only work if you use domain scoped token if you use v3 policy file

Update 1:

1) Have you granted any roles for the users in the domain dom1 and projects?

2) Are you able to get token using the newly created user? Nova client doesn't support v3 command line. But you can use cinder client and it support v3. This is just to make sure you are able to get tokens. e.g cinder --debug list. Cinder client takes domain_id etc so you can get v3 tokens

Update 2: I'm going to suggest a different way to debug the problem, This will help us to understand what is going on.

1) Switch keystone to use  uuid token.
https://github.com/openstack/keystone/blob/master/etc/keystone.conf.sample#L1384
(i.e) under token section in keystone.conf
provider = keystone.token.providers.uuid.Provider

2) restart keystone 

3) Now get a token from keystone scoped to the project. Since it is uuid token, you can see the token response which has roles for the project.

4) Now try  your nova command. Since it is UUID token the token validation will come to keystone.  There you can see in keystone.log why the token was rejected.

5) If keystone authorizes the token and you are still getting rejected, then nova expects some role on that token which you are missing for that operation.

6) If the token is going from horizon to nova, then you can find out from the token id, the scope of the token. All the services need project scoped token. If horizon sends domain scoped token, then it is not going to work

Update 3:

  1. If you had enabled debug in kesytone, you would have seen the problem.
  2. Nova is sending V2 token validaton call which understands only "default" domain. Nova should send v3 token validation call. V3 is fully backward compatible with v2.0 3 Enable auth_version = v3 and restart nova
auth_version=v3.0
  1. In Juno, default version is v3 and not v2, so you would not have noticed this with latest code

Update 4: 1. Some services uses .conf and some other services using paste-ini to configure keystone settings. I believe for glance you need to add it in paste.init file https://github.com/openstack/glance/b...

This question is becoming too big. If you think this is resolved, close this question and ask separate question.

edit flag offensive delete link more

Comments

Hi - thanks for the suggestion. I literally changed the line that contained "admin_domain_id" to the following at your suggestion: "cloud_admin": "rule:admin_required and domain_id:default", It had no effect for the particular case I mentioned (still getting 401 from nova)

echapin gravatar imageechapin ( 2014-08-27 14:23:59 -0500 )edit

Sorry, I don't understand your current setup. In your current current setup, are you using default policy file or v3 policy file? Also is the user who is getting 401 is in default domain or in the new domain you have created

Haneef Ali gravatar imageHaneef Ali ( 2014-08-28 11:12:33 -0500 )edit

Yes, I re-wrote the question and it may be confusing. I have a v3 user in a new domain 'dom1', and the default policy.json. I get the same behaviour when I switch to policy.v3cloudsample.json (including your suggestion). Originally I tried setting up domain-admins (this confused matters).

echapin gravatar imageechapin ( 2014-08-28 13:30:35 -0500 )edit

Just noticed Update 1. I have added to the question body. I can obtain a v3 token for a user, and they have a member roll on a project within the new domain.

echapin gravatar imageechapin ( 2014-08-28 17:49:55 -0500 )edit

I've added a detailed response to your update 2. The failure is clearly in keystone: "WARNING keystone.common.wsgi [-] Authorization failed. The request you have made requires authentication. from 132.246.194.41", only for users in non-default domain.

echapin gravatar imageechapin ( 2014-08-29 13:16:06 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2014-08-25 13:42:52 -0500

Seen: 2,265 times

Last updated: Aug 29 '14