Problems launching instances
Hi everyone, last week I managed to configure a two node OpenStack cloud and it was working fine but today is not working!!! I don't know what is the problem, I restarted the CloudController, I restarted the services but nothing worked, here is the output of nova-compute log:
2011-05-16 17:24:19,823 DEBUG nova.rpc [-] received {u'_context_request_id': u'2S-U0RPA6VPLVL73BB07', u'_context_read_deleted': False, u'args': {u'instance_id': 18, u'injected_files': None, u'availability_zone': None}, u'_context_is_admin': True, u'_context_timestamp': u'2011-05-16T16:24:19Z', u'_context_user': u'myuser', u'method': u'run_instance', u'_context_project': u'proj1', u'_context_remote_address': u'192.168.1.30'} from (pid=1048) _receive /usr/lib/pymodules/python2.6/nova/rpc.py:177 2011-05-16 17:24:19,824 DEBUG nova.rpc [-] unpacked context: {'timestamp': u'2011-05-16T16:24:19Z', 'remote_address': u'192.168.1.30', 'project': u'proj1', 'is_admin': True, 'user': u'myuser', 'request_id': u'2S-U0RPA6VPLVL73BB07', 'read_deleted': False} from (pid=1048) _unpack_context /usr/lib/pymodules/python2.6/nova/rpc.py:350 2011-05-16 17:24:19,895 AUDIT nova.compute.manager [2S-U0RPA6VPLVL73BB07 myuser proj1] instance 18: starting... 2011-05-16 17:24:20,042 DEBUG nova.rpc [-] Making asynchronous call on network.openstack-in-a-box-1rev85 ... from (pid=1048) call /usr/lib/pymodules/python2.6/nova/rpc.py:370 2011-05-16 17:24:20,043 DEBUG nova.rpc [-] MSG_ID is 1d4c30b82a514def8264d5a37a449879 from (pid=1048) call /usr/lib/pymodules/python2.6/nova/rpc.py:373 2011-05-16 17:24:23,646 INFO nova.compute.manager [-] Found instance 'instance-00000012' in DB but no VM. State=0, so setting state to shutoff. 2011-05-16 17:24:23,647 INFO nova.compute.manager [-] DB/VM state mismatch. Changing state from '0' to '5' 2011-05-16 17:25:23,856 INFO nova.compute.manager [-] Found instance 'instance-00000012' in DB but no VM. State=5, so setting state to shutoff. 2011-05-16 17:26:23,925 INFO nova.compute.manager [-] Found instance 'instance-00000012' in DB but no VM. State=5, so setting state to shutoff.
From what I'm uderstanding reading the log the spawning process is interrupted after making asynchronous call on network, nova computer manager waits for 3 seconds and because there is no response to the network call it changes the state of the VM to shutoff. Here is the nova.conf of the cloud controller:
--dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --verbose --s3_host=192.168.1.234 --rabbit_host=192.168.1.234 --cc_host=192.168.1.234 --ec2_url=http://192.168.1.234:8773/services/Cloud --fixed_range=10.0.0.0/12 --network_size=2048 --network_manager=nova.network.manager.FlatDHCPManager --flat_interface=eth0 --routing_source_ip=192.168.1.234 --FAKE_subdomain=ec2 --image_service=nova.image.glance.GlanceImageService --glance_host=192.168.1.234 --glance_port=9292 --sql_connection=mysql://root:nova@192.168.1.234/nova
And here is the output of ifconfig on the cloud controller:
eth0 Link encap:Ethernet HWaddr 00:24:8c:2e:50:88
inet addr:192.168.1.234 Bcast:192.168.1.255 Mask:255.255 ...