Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

no data received from console (juno)

Hi,

I have set up a Juno deployment of OpenStack. I can launch my cirros image:

[root@ost-mgmt-el7-001 ~]# nova list
+-----------+----------+--------+------------+-------------+------------------------+
| ID         | Name     | Status | Task State | Power State | Networks              
+-----------+----------+--------+------------+-------------+------------------------+
| xxxxxxx | test-007 | ACTIVE | -          | Running     | public-59=10.29.103.13
+-----------+----------+--------+------------+-------------+------------------------+
[root@ost-mgmt-el7-001 ~]# ssh 10.29.103.13 -l cirros
cirros@10.29.103.13's password: 
Permission denied, please try again.
cirros@10.29.103.13's password: 
$ ls -l
total 0
$ pwd
/home/cirros
$ exit
Connection to 10.29.103.13 closed.

But when I attempt to access the console like so ...

[root@ost-mgmt-el7-001 ~]# nova get-vnc-console test-007 novnc
+-------+---------------------------------------------------------------------------------------------+
| Type  | Url                                                                                         |
+-------+---------------------------------------------------------------------------------------------+
| novnc | https://ost-el7.exaple.com:6080/vnc_auto.html?token=xxxxx |
+-------+---------------------------------------------------------------------------------------------+
[root@ost-mgmt-el7-001 ~]# curl -v https://ost-el7.exaple.com:6080/vnc_auto.html?token=xxxxx

... I get "Empty reply from server" ...

* About to connect() to ost-el7.exaple.com port 6080 (#0)
*   Trying 10.29.103.39...
* Connected to ost-el7.exaple.com (10.29.103.39) port 6080 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*   subject: CN=ost-el7.exaple.com,O="Example Systems, Inc.",L=San Jose,ST=CA,C=US
*   start date: Feb 18 21:54:13 2015 GMT
*   expire date: Feb 18 21:54:00 2017 GMT
*   common name: ost-el7.exaple.com
*   issuer: CN=HydrantID SSL ICA G2,O=HydrantID (Avalanche Cloud Corporation),C=US
> GET /vnc_auto.html?token=5612703c-06cb-497f-a205-bbebbebb6d9c HTTP/1.1
> User-Agent: curl/7.29.0
> Host: ost-el7.exaple.com:6080
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host ost-el7.exaple.com left intact
curl: (52) Empty reply from server

Same thing happens if I log on to Horizon and attempt to access the console that way.

I am not sure how to really debug this. I do not see anything in the /var/log/httpd/*log files that is obviously wrong.

no data received from console (juno)

Hi,

I have set up a Juno deployment of OpenStack. I can launch my cirros image:

[root@ost-mgmt-el7-001 ~]# nova list
+-----------+----------+--------+------------+-------------+------------------------+
| ID         | Name     | Status | Task State | Power State | Networks              
+-----------+----------+--------+------------+-------------+------------------------+
| xxxxxxx | test-007 | ACTIVE | -          | Running     | public-59=10.29.103.13
+-----------+----------+--------+------------+-------------+------------------------+
[root@ost-mgmt-el7-001 ~]# ssh 10.29.103.13 -l cirros
cirros@10.29.103.13's password: 
Permission denied, please try again.
cirros@10.29.103.13's password: 
$ ls -l
total 0
$ pwd
/home/cirros
$ exit
Connection to 10.29.103.13 closed.

But when I attempt to access the console like so ...

[root@ost-mgmt-el7-001 ~]# nova get-vnc-console test-007 novnc
+-------+---------------------------------------------------------------------------------------------+
| Type  | Url                                                                                         |
+-------+---------------------------------------------------------------------------------------------+
| novnc | https://ost-el7.exaple.com:6080/vnc_auto.html?token=xxxxx |
+-------+---------------------------------------------------------------------------------------------+
[root@ost-mgmt-el7-001 ~]# curl -v https://ost-el7.exaple.com:6080/vnc_auto.html?token=xxxxx

... I get "Empty reply from server" ...

* About to connect() to ost-el7.exaple.com port 6080 (#0)
*   Trying 10.29.103.39...
* Connected to ost-el7.exaple.com (10.29.103.39) port 6080 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*   subject: CN=ost-el7.exaple.com,O="Example Systems, Inc.",L=San Jose,ST=CA,C=US
*   start date: Feb 18 21:54:13 2015 GMT
*   expire date: Feb 18 21:54:00 2017 GMT
*   common name: ost-el7.exaple.com
*   issuer: CN=HydrantID SSL ICA G2,O=HydrantID (Avalanche Cloud Corporation),C=US
> GET /vnc_auto.html?token=5612703c-06cb-497f-a205-bbebbebb6d9c HTTP/1.1
> User-Agent: curl/7.29.0
> Host: ost-el7.exaple.com:6080
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host ost-el7.exaple.com left intact
curl: (52) Empty reply from server

Same thing happens if I log on to Horizon and attempt to access the console that way.

I am not sure how to really debug this. I do not see anything in the /var/log/httpd/*log files that is obviously wrong.

UPDATE: Here's my log file, /var/log/nova/nova-novncproxy.log :

# cat /var/log/nova/nova-novncproxy.log 
2015-09-28 16:46:20.922 15365 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-28 16:46:20.924 15365 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-28 16:46:20.925 15365 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-28 16:59:33.722 15365 INFO nova.console.websocketproxy [-] Got SIGTERM, exiting
2015-09-28 16:59:33.793 15365 INFO nova.console.websocketproxy [-] In exit
2015-09-28 16:59:33.793 15365 DEBUG nova.console.websocketproxy [-] Closing socket listening at 0.0.0.0:6080 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-28 17:01:01.350 819 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-28 17:01:01.370 819 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-28 17:01:01.370 819 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-28 17:01:01.371 819 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-28 17:01:01.371 819 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-28 17:01:01.371 819 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-28 17:01:01.373 819 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-28 20:40:39.688 819 INFO nova.console.websocketproxy [-] Got SIGTERM, exiting
2015-09-28 20:40:39.689 819 INFO nova.console.websocketproxy [-] In exit
2015-09-28 20:40:39.689 819 DEBUG nova.console.websocketproxy [-] Closing socket listening at 0.0.0.0:6080 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-28 20:40:41.383 4876 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-28 20:40:41.383 4876 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-28 20:40:41.386 4876 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-29 15:48:14.118 4876 INFO nova.console.websocketproxy [-] Got SIGTERM, exiting
2015-09-29 15:48:14.120 4876 INFO nova.console.websocketproxy [-] In exit
2015-09-29 15:48:14.121 4876 DEBUG nova.console.websocketproxy [-] Closing socket listening at 0.0.0.0:6080 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:48:49.488 805 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-29 15:48:49.547 805 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-29 15:48:49.550 805 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-29 15:56:39.269 805 DEBUG nova.console.websocketproxy [-] 127.0.0.1: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:56:42.278 3826 INFO nova.console.websocketproxy [-] 127.0.0.1: ignoring socket not ready
2015-09-29 15:56:42.281 805 DEBUG nova.console.websocketproxy [-] Reaing zombies, active child count is 0 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:02.168 805 DEBUG nova.console.websocketproxy [-] 10.29.103.51: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:05.175 3828 INFO nova.console.websocketproxy [-] 10.29.103.51: ignoring socket not ready
2015-09-29 15:57:05.179 805 DEBUG nova.console.websocketproxy [-] Reaing zombies, active child count is 0 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:09.506 805 DEBUG nova.console.websocketproxy [-] 10.29.103.51: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:11.170 3830 INFO nova.console.websocketproxy [-] 10.29.103.51: non-SSL connection received but disallowed
2015-09-29 15:57:11.173 805 DEBUG nova.console.websocketproxy [-] Reaing zombies, active child count is 0 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828

I am not sure what I am looking at there :)

no data received from console (juno)

Hi,

I have set up a Juno deployment of OpenStack. I can launch my cirros image:

[root@ost-mgmt-el7-001 ~]# nova list
+-----------+----------+--------+------------+-------------+------------------------+
| ID         | Name     | Status | Task State | Power State | Networks              
+-----------+----------+--------+------------+-------------+------------------------+
| xxxxxxx | test-007 | ACTIVE | -          | Running     | public-59=10.29.103.13
+-----------+----------+--------+------------+-------------+------------------------+
[root@ost-mgmt-el7-001 ~]# ssh 10.29.103.13 -l cirros
cirros@10.29.103.13's password: 
Permission denied, please try again.
cirros@10.29.103.13's password: 
$ ls -l
total 0
$ pwd
/home/cirros
$ exit
Connection to 10.29.103.13 closed.

But when I attempt to access the console like so ...

[root@ost-mgmt-el7-001 ~]# nova get-vnc-console test-007 novnc
+-------+---------------------------------------------------------------------------------------------+
| Type  | Url                                                                                         |
+-------+---------------------------------------------------------------------------------------------+
| novnc | https://ost-el7.exaple.com:6080/vnc_auto.html?token=xxxxx |
+-------+---------------------------------------------------------------------------------------------+
[root@ost-mgmt-el7-001 ~]# curl -v https://ost-el7.exaple.com:6080/vnc_auto.html?token=xxxxx

... I get "Empty reply from server" ...

* About to connect() to ost-el7.exaple.com port 6080 (#0)
*   Trying 10.29.103.39...
* Connected to ost-el7.exaple.com (10.29.103.39) port 6080 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
* Server certificate:
*   subject: CN=ost-el7.exaple.com,O="Example Systems, Inc.",L=San Jose,ST=CA,C=US
*   start date: Feb 18 21:54:13 2015 GMT
*   expire date: Feb 18 21:54:00 2017 GMT
*   common name: ost-el7.exaple.com
*   issuer: CN=HydrantID SSL ICA G2,O=HydrantID (Avalanche Cloud Corporation),C=US
> GET /vnc_auto.html?token=5612703c-06cb-497f-a205-bbebbebb6d9c HTTP/1.1
> User-Agent: curl/7.29.0
> Host: ost-el7.exaple.com:6080
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host ost-el7.exaple.com left intact
curl: (52) Empty reply from server

Same thing happens if I log on to Horizon and attempt to access the console that way.

I am not sure how to really debug this. I do not see anything in the /var/log/httpd/*log files that is obviously wrong.

UPDATE: Here's my log file, /var/log/nova/nova-novncproxy.log :

# cat /var/log/nova/nova-novncproxy.log 
2015-09-28 16:46:20.922 15365 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-28 16:46:20.923 15365 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-28 16:46:20.924 15365 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-28 16:46:20.925 15365 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-28 16:59:33.722 15365 INFO nova.console.websocketproxy [-] Got SIGTERM, exiting
2015-09-28 16:59:33.793 15365 INFO nova.console.websocketproxy [-] In exit
2015-09-28 16:59:33.793 15365 DEBUG nova.console.websocketproxy [-] Closing socket listening at 0.0.0.0:6080 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-28 17:01:01.350 819 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-28 17:01:01.370 819 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-28 17:01:01.370 819 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-28 17:01:01.371 819 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-28 17:01:01.371 819 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-28 17:01:01.371 819 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-28 17:01:01.373 819 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-28 20:40:39.688 819 INFO nova.console.websocketproxy [-] Got SIGTERM, exiting
2015-09-28 20:40:39.689 819 INFO nova.console.websocketproxy [-] In exit
2015-09-28 20:40:39.689 819 DEBUG nova.console.websocketproxy [-] Closing socket listening at 0.0.0.0:6080 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-28 20:40:41.383 4876 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-28 20:40:41.383 4876 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-28 20:40:41.384 4876 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-28 20:40:41.386 4876 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-29 15:48:14.118 4876 INFO nova.console.websocketproxy [-] Got SIGTERM, exiting
2015-09-29 15:48:14.120 4876 INFO nova.console.websocketproxy [-] In exit
2015-09-29 15:48:14.121 4876 DEBUG nova.console.websocketproxy [-] Closing socket listening at 0.0.0.0:6080 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:48:49.488 805 INFO nova.console.websocketproxy [-] WebSocket server settings:
2015-09-29 15:48:49.547 805 INFO nova.console.websocketproxy [-]   - Listen on 0.0.0.0:6080
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - Flash security policy server
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - Web server. Web root: /usr/share/novnc
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - SSL/TLS support
2015-09-29 15:48:49.548 805 INFO nova.console.websocketproxy [-]   - Deny non-SSL/TLS connections
2015-09-29 15:48:49.550 805 INFO nova.console.websocketproxy [-]   - proxying from 0.0.0.0:6080 to None:None
2015-09-29 15:56:39.269 805 DEBUG nova.console.websocketproxy [-] 127.0.0.1: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:56:42.278 3826 INFO nova.console.websocketproxy [-] 127.0.0.1: ignoring socket not ready
2015-09-29 15:56:42.281 805 DEBUG nova.console.websocketproxy [-] Reaing zombies, active child count is 0 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:02.168 805 DEBUG nova.console.websocketproxy [-] 10.29.103.51: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:05.175 3828 INFO nova.console.websocketproxy [-] 10.29.103.51: ignoring socket not ready
2015-09-29 15:57:05.179 805 DEBUG nova.console.websocketproxy [-] Reaing zombies, active child count is 0 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:09.506 805 DEBUG nova.console.websocketproxy [-] 10.29.103.51: new handler Process vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828
2015-09-29 15:57:11.170 3830 INFO nova.console.websocketproxy [-] 10.29.103.51: non-SSL connection received but disallowed
2015-09-29 15:57:11.173 805 DEBUG nova.console.websocketproxy [-] Reaing zombies, active child count is 0 vmsg /usr/lib/python2.7/site-packages/websockify/websocket.py:828

I am not sure what I am looking at there :)

UPDATE 2: I changed my ssl_only setting to false ...

#RED change back ssl_only = true
ssl_only = false

... in my /etc/nova/nova.conf on my management nodes and I now able to get to the console. But I suspect that my groups infosec team will want me to change it back to true for production so I still need to get the console to work when I set ssl_only back to true.