Ask Your Question
0

Unable run instance on second node with error Timed out waiting for RPC response

asked 2012-07-06 06:30:31 -0600

xiamianderen gravatar image

I've installed openstack essex on 2 nodes with CentOS 6.2. One has all nova services called node1 and the other called node2 has only compute service running. I could launch instance successfully with a single node (node1), it failed while I'd added node2 to the cloud. When I launch instance, it's assigned on node2 and turns to ERROR state after a while. Errors could be found on both network.log on node1 and compute.log on node2.

network.log on node 1: 2012-07-03 14:43:00 ERROR nova.rpc.common [-] Timed out waiting for RPC response: timed out 2012-07-03 14:43:00 TRACE nova.rpc.common Traceback (most recent call last): 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/nova/rpc/impl_kombu.py", line 490, in ensure 2012-07-03 14:43:00 TRACE nova.rpc.common return method(args, kwargs) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/nova/rpc/impl_kombu.py", line 567, in _consume 2012-07-03 14:43:00 TRACE nova.rpc.common return self.connection.drain_events(timeout=timeout) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/kombu/connection.py", line 139, in drain_events 2012-07-03 14:43:00 TRACE nova.rpc.common return self.transport.drain_events(self.connection, *kwargs) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 223, in drain_events 2012-07-03 14:43:00 TRACE nova.rpc.common return connection.drain_events(kwargs) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 56, in drain_events 2012-07-03 14:43:00 TRACE nova.rpc.common return self.wait_multi(self.channels.values(), timeout=timeout) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 62, in wait_multi 2012-07-03 14:43:00 TRACE nova.rpc.common chanmap.keys(), allowed_methods, timeout=timeout) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 119, in _wait_multiple 2012-07-03 14:43:00 TRACE nova.rpc.common channel, method_sig, args, content = read_timeout(timeout) 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/kombu/transport/pyamqplib.py", line 93, in read_timeout 2012-07-03 14:43:00 TRACE nova.rpc.common return self.method_reader.read_method() 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/amqplib/client_0_8/method_framing.py", line 215, in read_method 2012-07-03 14:43:00 TRACE nova.rpc.common raise m 2012-07-03 14:43:00 TRACE nova.rpc.common timeout: timed out

compute.log on node 2:

2012-07-03 14:43:00 ERROR nova.rpc.common [req-76b47004-835b-4875-8cb0-f4714776aa2b b74e7826ac8c46f1968653e71871d69e 7df8530e806740428d4ae230f6a50cad] Timed out waiting for RPC response: timed out 2012-07-03 14:43:00 TRACE nova.rpc.common Traceback (most recent call last): 2012-07-03 14:43:00 TRACE nova.rpc.common File "/usr/lib/python2.6/site-packages/nova/rpc/impl_kombu.py", line 490, in ensure ... (more)

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2012-07-09 14:23:39 -0600

nilanjan-r gravatar image

Can you check the rabbitmq log files? I have faced similar issue of RPC timed out while listing floating IPs and the problem boiled down to rabbbitmq-server's queue was full. I restarted the service and blocked the source which is making numerous aqmp requests ( https://answers.launchpad.net/nova/+question/202628 (https://answers.launchpad.net/nova/+q...) ).

I think the servers are intercommunicating with hostname. Also try setting the flag

novnc_enabled=true vncserver_listen=0.0.0.0

edit flag offensive delete link more
0

answered 2012-07-10 07:29:07 -0600

xiamianderen gravatar image

Hi, Nilanjan

Thanks for your reply. However, I have to reinstall openstack after trying for many times. That I came up with another question.

I will take your advice facing the same problem. And if you have some pieces, could you please take a look at my concern on page https://answers.launchpad.net/nova/+question/202631 (https://answers.launchpad.net/nova/+q...) now?

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-07-06 06:30:31 -0600

Seen: 68 times

Last updated: Jul 10 '12