Keystone authentication failure in a HA Setup
I am running two controllers as active/active under two haproxy nodes as active/passive using a VIP. All the services in controller nodes are load balanced. I am facing a strange situation. Whenever I am trying to keystone command, for the first two attempts it gives me this error : An unexpected error prevented the server from fulfilling your request. (HTTP 500)
, and then when I run the commands continuously without much time gap, it works fine. Then when I try to run the same commands after sometime, again the same situation comes into picture. No significant error message in the logs is noticed.
I am very new to load balancing, so unable to figure out whether it is a load balancing issue or a something else.
This is my haproxy.cfg :
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
contimeout 5000
clitimeout 50000
srvtimeout 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
listen galera 192.168.1.64:3306
balance source
mode tcp
option tcpka
option mysql-check user haproxy
server Controller1 192.168.1.61:3306 check weight 1
server Controller2 192.168.1.62:3306 check weight 1
listen keystone_admin 192.168.1.64:35357
balance source
option tcpka
option httpchk
maxconn 10000
server Controller1 192.168.1.61:35357 check inter 2000 rise 2 fall 5
server Controller2 192.168.1.62:35357 check inter 2000 rise 2 fall 5
listen keystone_api 192.168.1.64:5000
balance source
option tcpka
option httpchk
maxconn 10000
server Controller1 192.168.1.61:5000 check inter 2000 rise 2 fall 5
server Controller2 192.168.1.62:5000 check inter 2000 rise 2 fall 5
listen glance-api 192.168.1.64:9292
balance source
option tcpka
option httpchk
maxconn 10000
server Controller1 192.168.1.61:9292 check inter 2000 rise 2 fall 5
server Controller2 192.168.1.62:9292 check inter 2000 rise 2 fall 5
listen glance-registry 192.168.1.64:9191
balance source
option tcpka
option httpchk
maxconn 10000
server Controller1 192.168.1.61:9191 check inter 2000 rise 2 fall 5
server Controller2 192.168.1.62:9191 check inter 2000 rise 2 fall 5
listen nova_ec2 192.168.1.64:8773
balance source
option tcpka
option httpchk
maxconn 10000
server Controller1 192.168.1.61:8773 check inter 2000 rise 2 fall 5
server Controller2 192.168.1.62:8773 check inter 2000 rise 2 fall 5
listen nova_osapi 192.168.1.64:8774
balance source
option tcpka
option httpchk
maxconn 10000
server Controller1 192.168.1.61:8774 check inter 2000 rise 2 fall 5
server Controller2 192.168 ...
I am facing the same issue in dashboard as well. It gives me such error :
Error: Unable to retrieve usage information.
. Then I have to logout and then again log in to avoid this error. By this, I am pretty much clear the issue is with load balancing.