shoubam's profile - activity

I am following this guide, and have trouble doing anything with glance. I first stumpled across this when I tried to add an image like this:

[root@cloud ~]# glance -d image-create --is-public true --disk-format qcow2 --container-format bare --name "Ubuntu" <precise-server-cloudimg-amd64.img 
   HTTP/1.1 404 Not Found
    date: Fri, 05 Jul 2013 15:27:46 GMT
    content-length: 52
    content-type: text/plain; charset=UTF-8
    x-openstack-request-id: req-ba4ea9c8-ccab-486c-ada2-5a8ac00d8d5c

    404 Not Found

    The resource could not be found.



    Request returned failure status.
    404 Not Found
    The resource could not be found.
        (HTTP 404)

Doing an glance image-list gives the same error. In /var/log/glance/api.log, I find

   2013-07-05 17:27:46.613 1653 DEBUG glance.api.middleware.version_negotiation [-] Determining version of request: POST //v1/images Accept:  process_request /usr/lib/python2.7/site-packages/glance/api/middleware/version_negotiation.py:46
    2013-07-05 17:27:46.618 1653 DEBUG glance.api.middleware.version_negotiation [-] Using url versioning process_request /usr/lib/python2.7/site-packages/glance/api/middleware/version_negotiation.py:59
    2013-07-05 17:27:46.619 1653 DEBUG glance.api.middleware.version_negotiation [-] Matched version: v1 process_request /usr/lib/python2.7/site-packages/glance/api/middleware/version_negotiation.py:71
    2013-07-05 17:27:46.620 1653 DEBUG glance.api.middleware.version_negotiation [-] new uri /v1/images process_request /usr/lib/python2.7/site-packages/glance/api/middleware/version_negotiation.py:72

While doing the image-create command, I can see in top that glance is working for a while, being at the top of the process list. So it seems that glance is connecting and working, but then somehow fails.

On a sidenote, not sure if thats related: when stopping the glance daemons, I get

    2013-07-05 16:05:39.703 3624 CRITICAL glance [-] [Errno 13] Permission denied
    2013-07-05 16:05:39.703 3624 TRACE glance Traceback (most recent call last):
    2013-07-05 16:05:39.703 3624 TRACE glance   File "/usr/bin/glance-api", line 65, in <modu
    le>
    2013-07-05 16:05:39.703 3624 TRACE glance     server.wait()
    2013-07-05 16:05:39.703 3624 TRACE glance   File "/usr/lib/python2.7/site-packages/glance
    /common/wsgi.py", line 271, in wait
    2013-07-05 16:05:39.703 3624 TRACE glance     self.wait_on_children()
    2013-07-05 16:05:39.703 3624 TRACE glance   File "/usr/lib/python2.7/site-packages/glance
    /common/wsgi.py", line 244, in wait_on_children
    2013-07-05 16:05:39.703 3624 TRACE glance     pid, status = os.wait()
    2013-07-05 16:05:39.703 3624 TRACE glance   File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 210, in kill_children
    2013-07-05 16:05:39.703 3624 TRACE glance     os.killpg(pgid, signal.SIGTERM)
    2013-07-05 16:05:39.703 3624 TRACE glance OSError: [Errno 13] Permission denied
    2013-07-05 16:05:39.703 3624 TRACE glance

I had a look at the code, and its the os.killpg(pgid, signal.SIGTERM that causes this, but nonetheless, the daemon is stopped.

I am using devstack on ubuntu 12.04. When starting the rejoin-stack.sh after a reboot, the scripts hangs at the command

cd /opt/stack/cinder && /usr/local/bin/cinder-volume --config-file /etc/cinder/cinder.conf

with the message

2013-07-08 16:55:12.935 ERROR cinder.openstack.common.rpc.common   [req-94a7b477-794b-4679-864d-dd6deb7ba5e8 None None] AMQP server on localhost:5672 is  unreachable: Socket closed. Trying again in 1 seconds.
2013-07-08 16:55:13.939 INFO cinder.openstack.common.rpc.common [req-94a7b477-794b-4679-864d-dd6deb7ba5e8 None None] Reconnecting to AMQP server on localhost:5672
2013-07-08 16:55:14.006 DEBUG amqp [-] Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL.  See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2011 VMware, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'consumer_cancel_notify': True, u'publisher_confirms': True, u'basic.nack': True}, u'platform': u'Erlang/OTP', u'version': u'2.7.1'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US'] from (pid=4554) _start /usr/local/lib/python2.7/dist-packages/amqp/connection.py:706

It is true that there is no one listening (at least on ipv4):

root@devstack:~# netstat -tulpen |grep 5672
tcp6       0      0 :::5672                 :::*                    LISTEN      104        9733        1285/beam.smp

What am I missing? What could I do to further debug this?

I am following this guide and have configured everything like in the guide. When I try something like glance image-list, I get

Authorization Failed: [Errno 104] Connection reset by peer

As I am new to openstack, I am not quite sure how to debug this. Whom is glance connecting to, keystone, or the mysql database? If its keystone, how can I check which credentials glance is using (I guess the ones provided in /etc/glance/glance-api.conf and /etc/glance/glance-registry.conf ?) How can I check the credentials with the verifyer directly? ... any hints would be appreciated.

I also found:

# keystone user-list
WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored).
[Errno 104] Connection reset by peer

Maybe there is a connection to the glance authorization error?

Using grizzly on centos 6.4, I have this strange thing with horizon: when I try to view the console of an instance (novncproxy), all I get is

Instance Console

If console is not responding to keyboard input: click the grey status bar below. Click here to show only console

And then nothing. I can "Click here", which gets me a new browser window, with the message

Server disconnected (code: 1006)

and a greyed out Send CtrlAltDel button. Now when I reload this window (typing Ctrl+Shift+r in firefox), I get a functioning console.

When I do this with chrome, I get the console window within horizon, but still "Failed to connect to server (code: 1006)". Then I do the "click here" thingy, a new browser tab opens, and the console is working.

So while I have the basic functionality, I would greatly apreciate to have the console wokring _inside_ horizon _without_ the need to reload. Any hints? Anything I could debug?

So it turns out this issue is related to myself sitting in an office with a webproxy. I have firefox configured to automatic detect proxy setting for the network. In general it works fine, I can connect to outside websites AND openstack horizon. horizon itself works fine, but it does not show the console window. I can load the console in an extra window, still no conncetion. But if I disable webproxy alltogether, and then reload the window with the console, it works.

So its kind of solved, but I would really like to know why this works like this. I will make a sperate question.

Following up on my earlier question: I have openstack grizzly on centos 6.4. Things are working, except for console connection to the instances. Instances are up and running, and I can connect to their console with a vnc viewer, e.g. remmina. But I can not connect to their console via novnc. All it gives me is a "Server disconnected (code: 1006)".

hosts involved:

192.168.222.10 haproxy
192.168.222.17 all nova services except nova-compute
192.168.222.140 nova-compute

From an experimental devstack install, I can see that novncproxy is connecting directly to the vnc port of the instance. With this system, this is not the case, novnc is not attempting to connect to the instances console, as I can see from an empty tcpdump on the compute host:

tcpdump -i br-admin-222 port 5907
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-admin-222, link-type EN10MB (Ethernet), capture size 65535 bytes

I am sure its the right port, because I can connect to that port with remmina and see the console. Also, I can connect to that port with telnet from the host running nova-novncproxy:

$ telnet 192.168.222.140 5907
Trying 192.168.222.140...
Connected to 192.168.222.140.
Escape character is '^]'.
RFB 003.008

So its not a networking issue.

nova.conf on nova-novncproxy host is

novncproxy_port=6080
novncproxy_base_url=http://192.168.222.10:6080/vnc_auto.html
novncproxy_host=192.168.222.17

nova.conf on nova-compute host is

vncserver_listen=0.0.0.0
vncserver_proxyclient_address=192.168.222.140
novncproxy_base_url=http://192.168.222.10:6080/vnc_auto.html

If someone could answer any of the following questions, I would be greatfull:

• how does nova-novncproxy know about which host and which port to connect to for a certain instance? Where is this stored, and how does that information get to novnc?
• I have started nova-novncproxy in the foreground to get more info with --debug and --verbose, but won't get any errors or warnings. How can I get more info on where exactly this is failing?
• Any other hints on how to debug this?

I did `nova-manage db sync`. It did not help with the error message and also not with the actual problem.