(Ocata) OpenStack High Availability Setup (glance issue)
What is causing "Unauthorized (HTTP 401)" error?
I am setting up OpenStack HA environment using three controllers, two compute nodes, three ceph nodes, and two HAproxy nodes. I am doing manual installation (not using tripleO / Ansible). I successfully deployed ceph, mariadb-galera-cluster, rabbitmq-cluster, haproxy, and keystone). I have provided output of "openstack token issue" below to show keystone is working fine.
[root@controller1 ~]# cat keystonerc_admin
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin123
export OS_AUTH_URL=http://10.0.0.11:35357/v3
export OS_IDENTITY_API_VERSION=3
export OS_IMAGE_API_VERSION=2
export PS1='[\u@\h \W (admin)]# '
[root@controller1 ~]# source keystonerc_admin
[root@controller1 ~ (admin)]# openstack token issue
+------------+---------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+------------+---------------------------------------------------------------------------------------------------------------------------+
| expires | 2017-06-20T17:04:26+0000 |
| id | gAAAAABZSUeKZLEtrDl54hxjOxgc4HAkj2Hgo-bXk4yerjt6cL4e- |
| | QCsUBM_aQWtyZLfoEWHID8SgXoBGd6eCmguG8MsJ9_9Ci2Oy4btBtchr0ul8YcGHASRTB9rTnGxwF0lsoS7nEcBwF9cYKQoJ-SISCQ- |
| | nbD_W6atbeN4R3gfAN8C-KBmKgI |
| project_id | 96c9b397c67047269b9a59470cf35622 |
| user_id | 35e74dfcd02946d1ae95467f14c3bfb8 |
+------------+---------------------------------------------------------------------------------------------------------------------------+
But when I try to list images in glance, I get unauthorized error.
[root@controller1 ~ (admin)]# openstack image list
Unauthorized (HTTP 401)
[root@controller1 ~ (admin)]# glance image-list
Invalid OpenStack Identity credentials.
I don't see any issue with glance user credentials in /etc/glance/glance-api.conf and /etc/glance/glance-registry.conf.
[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_url = http://10.0.0.11:35357
memcached_servers = controller1:11211,controller2:11211,controller3:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = glance
password = GLANCE_PASS
OpenStack Endpoint Details:
[root@controller1 ~ (admin)]# openstack endpoint list
+----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------+
| ID | Region | Service Name | Service Type | Enabled | Interface | URL |
+----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------+
| 50577ac48e7446beabcd4336ddd3ad2f | RegionOne | glance | image | True | internal | http://10.0.0.11:9292 |
| 56a6b82afcd3441fbd71499d62b9d3c1 | RegionOne | glance | image | True | admin | http://10.0.0.11:9292 |
| 7616f05adf54471e85a944e89786e5e4 | RegionOne | keystone | identity | True | public | http://10.0.0.11:5000/v3/ |
| 8f4702a609ff4d19a28a4a8e49691586 | RegionOne | glance | image | True | public | http://10.0.0.11:9292 |
| a8198d52bb674832b39ea5868021c90f | RegionOne | keystone | identity | True | admin | http://10.0.0.11:35357/v3/ |
| f1ecf62bf9aa42b7863e550f3a676826 | RegionOne | keystone | identity | True | internal | http://10.0.0.11:5000/v3/ |
+----------------------------------+-----------+--------------+--------------+---------+-----------+----------------------------+
HAProxy Configuration File (/etc/haproxy/haproxy.cfg)
global
log 127.0.0.1 local0
log 127.0.0.1 local1 notice
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4096
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
log global
mode http
option tcplog
option dontlognull
retries 3
option redispatch
maxconn 4096
timeout connect 50000ms
timeout client 50000ms
timeout server 50000ms
listen stats 0.0.0.0:80
mode http
stats enable
stats uri /stats
stats realm HAProxy\ Statistics
stats auth admin:openstack
listen galera 0.0.0.0:3306
mode tcp
balance roundrobin
option tcpka
option mysql-check user haproxy
server controller1 10.0.0.12:3306 check weight 1
server controller2 10.0.0.13:3306 check weight 1
server controller3 10.0.0.14:3306 check weight 1
listen keystone_admin_cluster
bind 0.0.0.0:35357
balance source
option tcpka
option httpchk
option tcplog
server controller1 10.0.0.12:35357 check inter 2000 rise 2 fall 5
server controller2 10.0.0.13:35357 check inter ...