Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

(Ocata) OpenStack High Availability Setup (glance issue)

What is causing "Unauthorized (HTTP 401)" error?

I am setting up OpenStack HA environment using three controllers, two compute nodes, three ceph nodes, and two HAproxy nodes. I am doing manual installation (not using tripleO / Ansible). I successfully deployed ceph, mariadb-galera-cluster, rabbitmq-cluster, haproxy, and keystone). I have provided output of "openstack token issue" below to show keystone is working fine.

[root@controller1 ~]# cat keystonerc_admin
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin123
export OS_AUTH_URL=http://10.0.0.11:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2
export PS1='[\u@\h \W (admin)]# '

[root@controller1 ~]# source keystonerc_admin
[root@controller1 ~ (admin)]# openstack token issue
+------------+---------------------------------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                                                     |
+------------+---------------------------------------------------------------------------------------------------------------------------+
| expires    | 2017-06-20T17:04:26+0000                                                                                                  |
| id         | gAAAAABZSUeKZLEtrDl54hxjOxgc4HAkj2Hgo-bXk4yerjt6cL4e-                                                                     |
|            | QCsUBM_aQWtyZLfoEWHID8SgXoBGd6eCmguG8MsJ9_9Ci2Oy4btBtchr0ul8YcGHASRTB9rTnGxwF0lsoS7nEcBwF9cYKQoJ-SISCQ-                   |
|            | nbD_W6atbeN4R3gfAN8C-KBmKgI                                                                                               |
| project_id | 96c9b397c67047269b9a59470cf35622                                                                                          |
| user_id    | 35e74dfcd02946d1ae95467f14c3bfb8                                                                                          |
+------------+---------------------------------------------------------------------------------------------------------------------------+

But when I try to list images in glance, I get unauthorized error.

[root@controller1 ~ (admin)]# openstack image list
Unauthorized (HTTP 401)
[root@controller1 ~ (admin)]# glance image-list
Invalid OpenStack Identity credentials.

I don't see any issue with glance user credentials in /etc/glance/glance-api.conf and /etc/glance/glance-registry.conf.

[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_url = http://10.0.0.11:35357
memcached_servers = controller1:11211,controller2:11211,controller3:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = GLANCE_PASS

OpenStack Endpoint Details:

[root@controller1 ~ (admin)]# openstack endpoint list
+----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                        |
+----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------+
| 50577ac48e7446beabcd4336ddd3ad2f | RegionOne | glance       | image        | True    | internal  | http://10.0.0.11:9292      |
| 56a6b82afcd3441fbd71499d62b9d3c1 | RegionOne | glance       | image        | True    | admin     | http://10.0.0.11:9292      |
| 7616f05adf54471e85a944e89786e5e4 | RegionOne | keystone     | identity     | True    | public    | http://10.0.0.11:5000/v3/  |
| 8f4702a609ff4d19a28a4a8e49691586 | RegionOne | glance       | image        | True    | public    | http://10.0.0.11:9292      |
| a8198d52bb674832b39ea5868021c90f | RegionOne | keystone     | identity     | True    | admin     | http://10.0.0.11:35357/v3/ |
| f1ecf62bf9aa42b7863e550f3a676826 | RegionOne | keystone     | identity     | True    | internal  | http://10.0.0.11:5000/v3/  |
+----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------+

HAProxy Configuration File (/etc/haproxy/haproxy.cfg)

global
    log         127.0.0.1 local0
    log         127.0.0.1 local1 notice
    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4096
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    log                     global
    mode                    http
    option                  tcplog
    option                  dontlognull
    retries                 3
    option                  redispatch
    maxconn                 4096
    timeout connect         50000ms
    timeout client          50000ms
    timeout server          50000ms

listen stats 0.0.0.0:80
  mode http
  stats enable
  stats uri /stats
  stats realm HAProxy\ Statistics
  stats auth admin:openstack


listen galera 0.0.0.0:3306
        mode tcp
        balance roundrobin
        option tcpka
        option mysql-check user haproxy
        server controller1 10.0.0.12:3306 check weight 1
        server controller2 10.0.0.13:3306 check weight 1
        server controller3 10.0.0.14:3306 check weight 1

listen keystone_admin_cluster
  bind 0.0.0.0:35357
  balance  source
  option  tcpka
  option  httpchk
  option  tcplog
  server controller1 10.0.0.12:35357 check inter 2000 rise 2 fall 5
  server controller2 10.0.0.13:35357 check inter 2000 rise 2 fall 5
  server controller3 10.0.0.14:35357 check inter 2000 rise 2 fall 5

 listen keystone_public_internal_cluster
  bind 0.0.0.0:5000
  balance  source
  option  tcpka
  option  httpchk
  option  tcplog
  server controller1 10.0.0.12:5000 check inter 2000 rise 2 fall 5
  server controller2 10.0.0.13:5000 check inter 2000 rise 2 fall 5
  server controller3 10.0.0.14:5000 check inter 2000 rise 2 fall 5

listen glance_api_cluster
  bind 0.0.0.0:9292
  balance  source
  option  tcpka
  option httpchk GET /versions
  option  tcplog
  server controller1 10.0.0.12:9292 check inter 2000 rise 2 fall 5
  server controller2 10.0.0.13:9292 check inter 2000 rise 2 fall 5
  server controller3 10.0.0.14:9292 check inter 2000 rise 2 fall 5

listen glance_registry_cluster
  bind 0.0.0.0:9191
  balance  source
  option  tcpka
  option  tcplog
  server controller1 10.0.0.12:9191 check inter 2000 rise 2 fall 5
  server controller2 10.0.0.13:9191 check inter 2000 rise 2 fall 5
  server controller3 10.0.0.14:9191 check inter 2000 rise 2 fall 5

What is causing "Unauthorized (HTTP 401)" error?