Ask Your Question
0

Failed to run instance in second nova-compute node

asked 2012-05-20 07:38:36 -0500

tungns-inf gravatar image

I'm trying to install Openstack with 2 nodes.

Each node has only one network interface eth0. I config nova-network by VlanManage mode.

Node 1 (a Physic machine with IP: 172.17.2.203) : installs all components keystone, glance, nova (also nova-compute) and horizon. I can run instance on this host, everything okay. Then I install another nova-compute on node 2.

Node 2 (a Virtual machine on ESX with IP: 172.17.2.202) : installs only nova-compute. When I check by # nova-manage service list -> it's return nova-compute service on node 2 works with :-)

But in dashboard, the services list don't have compute service from node 2 and when I try to run a new instance on node 2. It's return error when 'Spawning' instance.

For example when I run an instance with Ubuntu lucid UEC image, then I check the information of this instance by $ nova show lucid2. I got this error: (this instance was created on node 2)

{u'message': u'libvirtError', u'code': 500, u'created': u'2012-05-20T07:26:44Z'}

So I check nova-compute.log on Node 2 and here is the error part:

2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] Traceback (most recent call last): 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in _spawn 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] self._legacy_nw_info(network_info), block_device_info) 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] return f(args, *kw) 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 922, in spawn 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] self._create_new_domain(xml) 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1575, in _create_new_domain 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] domain.createWithFlags(launch_flags) 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] libvirtError: internal error Process exited while reading console log output: char device redirected to /dev/pts/2 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] inet_listen_opts: bind(ipv4,172.17.2.203,5900): Cannot assign requested address 2012-05-20 14:26:44 TRACE nova.compute.manager [instance: a053fae3-0798-4fef-b596-e08d32756322] inet_listen_opts: FAILED

So I think the problem comes from node 2 because libvirt can not create and run a new virtual machine on that (because node 2 is virtual machine on ESX too) but I config nova-compute to run: QEMU not KVM, and I think ... (more)

edit retag flag offensive close merge delete

7 answers

Sort by ยป oldest newest most voted
0

answered 2012-05-24 08:50:41 -0500

juanfra-rc gravatar image

This is the link where multihost option is mentioned:

http://docs.openstack.org/trunk/openstack-compute/admin/content/existing-ha-networking-options.html (http://docs.openstack.org/trunk/opens...)

edit flag offensive delete link more
0

answered 2012-05-23 12:08:13 -0500

juanfra-rc gravatar image

Hi neo:

I have the same problem... hence, my nove-compute nodes run KVM. I've try to delete '/var/lib/nova/_base' directory, but it does not solve the problem.

I also observed different forms to reference to Glance service in 'nova.conf' file:

--image_service=nova.image.glance.GlanceImageService OR --image_service=nova.image.glance.GlanceImage

Which is the correct form? both of them?

does anyone know how to solve this error (mentioned by neo above)?

thanks!

edit flag offensive delete link more
0

answered 2012-05-23 15:24:43 -0500

juanfra-rc gravatar image

sorry, i forgot to attach the error log:

instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] Instance failed to spawn 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] Traceback (most recent call last): 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in _spawn 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] self._legacy_nw_info(network_info), block_device_info) 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] return f(args, *kw) 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 922, in spawn 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] self._create_new_domain(xml) 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1575, in _create_new_domain 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] domain.createWithFlags(launch_flags) 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self) 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] libvirtError: internal error Process exited while reading console log output: char device redirected to /dev/pts/0 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] inet_listen_opts: bind(ipv4,163.117.148.131,5900): Cannot assign requested address 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] inet_listen_opts: FAILED 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] 2012-05-23 17:18:02 TRACE nova.compute.manager [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] 2012-05-23 17:18:02 DEBUG nova.compute.manager [req-fa3714d3-3ed1-4810-8cef-20b1be3d4192 dc570e363f03431b9c4d1fc0c2a92991 02fd2450caeb4b429ca8cbb41f1f96be] [instance: f1d6eabb-aae2-4500-9b77-e18bdeff6277] Deallocating network for instance from (pid=2161) _deallocate_network /usr/lib/python2.7/dist-packages/nova/compute/manager.py:616 2012-05-23 17:18:02 DEBUG nova.rpc.amqp [req-fa3714d3-3ed1-4810-8cef-20b1be3d4192 dc570e363f03431b9c4d1fc0c2a92991 02fd2450caeb4b429ca8cbb41f1f96be] Making asynchronous cast on network... from (pid=2161) cast /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:346 2012-05-23 17:18:03 ERROR nova.rpc.amqp [req-fa3714d3-3ed1-4810-8cef-20b1be3d4192 dc570e363f03431b9c4d1fc0c2a92991 02fd2450caeb4b429ca8cbb41f1f96be] Exception during message handling 2012-05-23 17:18:03 TRACE nova.rpc.amqp Traceback (most recent call last): 2012-05-23 17:18:03 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 252, in _process_data 2012-05-23 17:18:03 TRACE nova.rpc.amqp rval = node_func(context=ctxt, *node_args) 2012-05-23 17:18:03 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped 2012-05-23 17:18:03 TRACE nova.rpc.amqp return f(args, *kw) 2012-05-23 17:18:03 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 177, in decorated_function 2012-05-23 17:18:03 TRACE nova.rpc.amqp sys.exc_info()) 2012-05-23 17:18:03 TRACE nova.rpc.amqp File ... (more)

edit flag offensive delete link more
0

answered 2012-05-23 18:37:06 -0500

juanfra-rc gravatar image

Hi again:

I think the problem is related to novnc service. I've removed novnc service (and dependences) and I've launched a new instance. The instance was deployed with no poblems.

The error line 'inet_listen_opts: bind(ipv4,172.17.2.203,5900): Cannot assign requested address' is due to VNC Server. It cannot bind new VNC session at same IP:port (5900-tcp). I suggest a possible solution would be to assign new ports automatically for each new connection.

Regards!

edit flag offensive delete link more
0

answered 2012-05-24 05:31:13 -0500

tungns-inf gravatar image

@Juan: You're great! When I remove novnc, it's work. Thank you!

I only have one problem is: in nova-network, for example I have instance_1 in node1(controller + nova-compute) with fixed IP 10.0.1.2 and instance_2 in node2 (only nova-compute) with fixed IP 10.0.1.3.

In node1 (controller) I can ping to these 10.0.1.x IP, but in node2 I can't. For floating IP, it's okay.

When I run: $ ip addr

In node1:

br1: <broadcast,multicast,up,lower_up> mtu 1500 qdisc noqueue state UP link/ether fa:16:3e:7b:66:79 brd ff:ff:ff:ff:ff:ff inet 10.0.1.1/24 brd 10.0.1.255 scope global br1 inet6 fe80::64f2:cfff:fe89:b0ec/64 scope link valid_lft forever preferred_lft forever

but in node2:

br1: <broadcast,multicast,up,lower_up> mtu 1500 qdisc noqueue state UP link/ether 08:00:27:f3:57:4f brd ff:ff:ff:ff:ff:ff inet6 fe80::980e:edff:fe79:da7e/64 scope link valid_lft forever preferred_lft forever

I config nova-network by VlanManager. How can I make node2 get fixed IP vlan?

edit flag offensive delete link more
0

answered 2012-05-24 05:31:20 -0500

tungns-inf gravatar image

Thanks Juan F. Rodriguez, that solved my question.

edit flag offensive delete link more
0

answered 2012-05-24 08:49:18 -0500

juanfra-rc gravatar image

Hi neo0:

I also answered myself the same question. I think multihost option could be a solution since 'node2' (worker node) will also need 'nova-network' service. I'm going to try this configuration. I will add a comment with the results, ok?

Regards!

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-05-20 07:38:36 -0500

Seen: 134 times

Last updated: May 24 '12