Ask Your Question

nova-consoleauth and high-availability setup issue

asked 2015-02-17 08:19:38 -0600

jpmethot gravatar image

This is an issue that can only happen on high availability setups from what I understand. First of all, let me show my nova config for controller :

vncserver_listen =
vncserver_proxyclient_address =
auth_strategy = keystone

vncserver_proxyclient_address is the load-balanced IP, vncserver_listen is the controller's ip. It's the same configuration in controller2, except that vncserver_listen has that server's IP.

Now for the compute nodes :

my_ip =
vnc_enabled = True
vncserver_listen =
vncserver_proxyclient_address =
novncproxy_base_url =

This time, vncserver_proxyclient_address and my_ip are the serveR's address and novncproxy_base_url uses the load-balanced ip.

The main issue is that the console in horizon sometimes work and sometimes doesn't. It generally takes a few refresh for the console to work. If I look in the nova-consoleauth logs, we can see messages such as this one :

2015-02-17 08:52:03.727 1868 AUDIT nova.consoleauth.manager [req-65191ec8-93e9-41b3-8f88-fcc70626df7a None None] Checking Token: 822c2afc-3b99-4894-b67d-7bade24c07fd, True
2015-02-17 08:52:07.914 1868 AUDIT nova.consoleauth.manager [req-78892f1f-87b7-432e-bc0a-65abd592acba None None] Checking Token: 822c2afc-3b99-4894-b67d-7bade24c07fd, True
2015-02-17 08:52:11.133 1868 AUDIT nova.consoleauth.manager [req-73f122e7-bf0c-4c48-b810-d63a48a68e12 None None] Checking Token: 822c2afc-3b99-4894-b67d-7bade24c07fd, True
2015-02-17 08:53:17.825 1868 AUDIT nova.consoleauth.manager [req-c24d8d0f-c0d0-4141-85eb-9305c44899fd None None] Checking Token: 9e7eab23-7436-4812-9662-e83ee827ab25, False

As you can see, when the token should normally always be true for the connection, it sometimes returns false. I do have my own guess as to why it does that, but before I explain my thoughts, here's a link explaining the vnc proxy inner workings :

What I believe happens here is that, since nova-consoleauth is high-availability, it sends the token to the nova-consoleauth that did not request the connection. This results in the correct controller never receiving the token and thus, not authorizing the connection.

Am I right or am I overlooking something in my interpretation? I may file a bug report if this is the issue.

FTR, here's the HAproxy config for the console load-balancing :

listen novnc
        balance source
        option tcpka
        maxconn 10000
        server controller1 check inter 2000 rise 2 fall 5
        server controller2 check inter 2000 rise 2 fall 5
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted

answered 2015-02-17 13:54:14 -0600

Hmm. We have 5 controllers running and dont have any issue with 500+ users. There is no bug but more than likely a config issue in your env.

listen novncproxy
        bind ssl crt /etc/ssl/pound.pem
        balance source
        option httplog
        option http-server-close
        fullconn 1024
        capture request header X-Auth-Project-Id len 50
        capture request header User-Agent len 50
        server vmos02 check inter 5000
        server vmos03 check inter 5000
        server vmos04 check inter 5000
        server vmos05 check inter 5000
        server vmos06 check inter 5000

Can you verify that you actually have novnc proxy running on both controllers and listening on port 6080?

Can you paste the out of nova-manage service list from one of your controllers.

You also need to add novncproxy_base_url directive in to nova.conf on each controller pointing to the VIP.

edit flag offensive delete link more

answered 2015-02-17 14:25:20 -0600

jpmethot gravatar image

Thank you very much for that reply. By doing a nova manage service list, I found out that there was an old controller node we were using for debugging, that I had removed from haproxy, was still listed. I don't really understand how it could have caused this, but I removed the services from it and it now works well.

edit flag offensive delete link more


Good. That is what I was hoping to see from service list. Something not correct. Thanks for the update.

sfcloudman gravatar imagesfcloudman ( 2015-02-17 16:37:32 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools



Asked: 2015-02-17 08:19:38 -0600

Seen: 2,630 times

Last updated: Feb 17 '15