Ask Your Question
1

nova-consoleauth and high-availability setup issue

asked 2015-02-17 08:19:38 -0500

jpmethot gravatar image

This is an issue that can only happen on high availability setups from what I understand. First of all, let me show my nova config for controller :

vncserver_listen = 10.251.0.201
vncserver_proxyclient_address = 10.251.0.245
auth_strategy = keystone

vncserver_proxyclient_address is the load-balanced IP, vncserver_listen is the controller's ip. It's the same configuration in controller2, except that vncserver_listen has that server's IP.

Now for the compute nodes :

my_ip = 10.251.0.101
vnc_enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = 10.251.0.101
novncproxy_base_url = http://10.251.0.245:6080/vnc_auto.html

This time, vncserver_proxyclient_address and my_ip are the serveR's address and novncproxy_base_url uses the load-balanced ip.

The main issue is that the console in horizon sometimes work and sometimes doesn't. It generally takes a few refresh for the console to work. If I look in the nova-consoleauth logs, we can see messages such as this one :

2015-02-17 08:52:03.727 1868 AUDIT nova.consoleauth.manager [req-65191ec8-93e9-41b3-8f88-fcc70626df7a None None] Checking Token: 822c2afc-3b99-4894-b67d-7bade24c07fd, True
2015-02-17 08:52:07.914 1868 AUDIT nova.consoleauth.manager [req-78892f1f-87b7-432e-bc0a-65abd592acba None None] Checking Token: 822c2afc-3b99-4894-b67d-7bade24c07fd, True
2015-02-17 08:52:11.133 1868 AUDIT nova.consoleauth.manager [req-73f122e7-bf0c-4c48-b810-d63a48a68e12 None None] Checking Token: 822c2afc-3b99-4894-b67d-7bade24c07fd, True
2015-02-17 08:53:17.825 1868 AUDIT nova.consoleauth.manager [req-c24d8d0f-c0d0-4141-85eb-9305c44899fd None None] Checking Token: 9e7eab23-7436-4812-9662-e83ee827ab25, False

As you can see, when the token should normally always be true for the connection, it sometimes returns false. I do have my own guess as to why it does that, but before I explain my thoughts, here's a link explaining the vnc proxy inner workings : http://docs.openstack.org/admin-guide...

What I believe happens here is that, since nova-consoleauth is high-availability, it sends the token to the nova-consoleauth that did not request the connection. This results in the correct controller never receiving the token and thus, not authorizing the connection.

Am I right or am I overlooking something in my interpretation? I may file a bug report if this is the issue.

FTR, here's the HAproxy config for the console load-balancing :

listen novnc 10.251.0.245:6080
        balance source
        option tcpka
        maxconn 10000
        server controller1 10.251.0.201:6080 check inter 2000 rise 2 fall 5
        server controller2 10.251.0.202:6080 check inter 2000 rise 2 fall 5
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
2

answered 2015-02-17 13:54:14 -0500

Hmm. We have 5 controllers running and dont have any issue with 500+ users. There is no bug but more than likely a config issue in your env.

listen novncproxy
        bind 10.34.210.200:6080 ssl crt /etc/ssl/pound.pem
        balance source
        option httplog
        option http-server-close
        fullconn 1024
        capture request header X-Auth-Project-Id len 50
        capture request header User-Agent len 50
        server vmos02 10.34.208.19:6080 check inter 5000
        server vmos03 10.34.208.20:6080 check inter 5000
        server vmos04 10.34.208.21:6080 check inter 5000
        server vmos05 10.34.208.22:6080 check inter 5000
        server vmos06 10.34.208.23:6080 check inter 5000

Can you verify that you actually have novnc proxy running on both controllers and listening on port 6080?

Can you paste the out of nova-manage service list from one of your controllers.

You also need to add novncproxy_base_url directive in to nova.conf on each controller pointing to the VIP.

edit flag offensive delete link more
0

answered 2015-02-17 14:25:20 -0500

jpmethot gravatar image

Thank you very much for that reply. By doing a nova manage service list, I found out that there was an old controller node we were using for debugging, that I had removed from haproxy, was still listed. I don't really understand how it could have caused this, but I removed the services from it and it now works well.

edit flag offensive delete link more

Comments

Good. That is what I was hoping to see from service list. Something not correct. Thanks for the update.

sfcloudman gravatar imagesfcloudman ( 2015-02-17 16:37:32 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2015-02-17 08:19:38 -0500

Seen: 2,545 times

Last updated: Feb 17 '15