Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

Network Not Coming Up in Instances

I've done a pretty thorough search of the Q&A's here using as many keywords I can think of and can't seem to find an answer to my issue (though I'm finding a lot of people with similar issues but no resolution)

I've followed the Quickstart Guide in RDO for creating an allinone Openstack installation and configured it to work with an existing network. I have an existing network (192.168.1.0/24) on my LAN to which this new CentOS 7 running OpenStack Kilo is connected. I followed the guide and made modifications to the startup scripts of my host OS to configure the br-ex bridge and enp7s0 interface as an interface on that bridge. I've also created a private network (192.168.100.0/24 following the defaults of the guide). I've imported 3 different images (CentOS 7, Fedora 22 and CirrOS qcow2 images from the official repositories and started instances assigning private and public interfaces to all. In all cases, I cannot access the VM instances from my host (either via ICMP ping or SSH). I can access the console but cannot login as the images don't have passwords. The trick to override the passwords either via the Post-Creation script (password: mypassword) or the API ('nova root-password <instance-id>') don't seem to work (the last one resulting in a:

ERROR (HTTPNotImplemented): Unable to set password on instance (HTTP 501) (Request-ID: req-65fa3132-4003-45dd-ab15-4070404ed2f2)
). From the Fedora console-log, it looks like the network interfaces aren't coming up:
A start job is running for LSB: Bring up/down networking (1min 6s / 5min 3s)[K[[32m  OK  [0m] Started LSB: Bring up/down networking.
[[32m  OK  [0m] Reached target Network is Online.
         Starting Initial cloud-init job (metadata service crawler)...
[   70.053248] cloud-init[614]: Cloud-init v. 0.7.6 running 'init' at Fri, 19 Jun 2015 20:02:43 +0000. Up 70.03 seconds.
[   70.098909] cloud-init[614]: 2015-06-19 20:02:43,197 - util.py[WARNING]: Route info failed: Unexpected error while running command.
[   70.099204] cloud-init[614]: Command: ['netstat', '-rn']
[   70.099384] cloud-init[614]: Exit code: 1
[   70.099542] cloud-init[614]: Reason: -
[   70.099700] cloud-init[614]: Stdout: 'Kernel IP routing table\nDestination     Gateway         Genmask         Flags   MSS Window  irtt Iface\n'
[   70.099859] cloud-init[614]: Stderr: ''
[   70.122800] cloud-init[614]: ci-info: +++++++++++++++++++++++Net device info++++++++++++++++++++++++
[   70.130341] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[   70.136699] cloud-init[614]: ci-info: | Device |   Up  |  Address  |    Mask   |     Hw-Address    |
[   70.136877] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[   70.137122] cloud-init[614]: ci-info: |  lo:   |  True | 127.0.0.1 | 255.0.0.0 |         .         |
[   70.137325] cloud-init[614]: ci-info: | eth1:  | False |     .     |     .     | fa:16:3e:8d:1e:65 |
[   70.137525] cloud-init[614]: ci-info: | eth0:  |  True |     .     |     .     | fa:16:3e:50:a2:a6 |
[   70.137717] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[   70.137905] cloud-init[614]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[   70.408174] cloud-init[614]: 2015-06-19 20:02:43,506 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(101, 'Network is unreachable'))]

(the last line repeated several times)

I'm not sure what the issue is. I haven't touched the default iptables on my CentOS 7 host. Everything is either the default or configured by neutron/nova.

Any help would be appreciated

Network Not Coming Up in Instances

I've done a pretty thorough search of the Q&A's here using as many keywords I can think of and can't seem to find an answer to my issue (though I'm finding a lot of people with similar issues but no resolution)

I've followed the Quickstart Guide in RDO for creating an allinone Openstack installation and configured it to work with an existing network. I have an existing network (192.168.1.0/24) on my LAN to which this new CentOS 7 running OpenStack Kilo is connected. I followed the guide and made modifications to the startup scripts of my host OS to configure the br-ex bridge and enp7s0 interface as an interface on that bridge. I've also created a private network (192.168.100.0/24 following the defaults of the guide). I've imported 3 different images (CentOS 7, Fedora 22 and CirrOS qcow2 images from the official repositories and started instances assigning private and public interfaces to all. In all cases, I cannot access the VM instances from my host (either via ICMP ping or SSH). I can access the console but cannot login as the images don't have passwords. The trick to override the passwords either via the Post-Creation script (password: mypassword) or the API ('nova root-password <instance-id>') don't seem to work (the last one resulting in a:

ERROR (HTTPNotImplemented): Unable to set password on instance (HTTP 501) (Request-ID: req-65fa3132-4003-45dd-ab15-4070404ed2f2)
). From the Fedora console-log, it looks like the network interfaces aren't coming up:
A start job is running for LSB: Bring up/down networking (1min 6s / 5min 3s)[K[[32m  OK  [0m] Started LSB: Bring up/down networking.
[[32m  OK  [0m] Reached target Network is Online.
  Starting Initial cloud-init job (metadata service crawler)...
[  70.053248] cloud-init[614]: Cloud-init v. 0.7.6 running 'init' at Fri, 19 Jun 2015 20:02:43 +0000. Up 70.03 seconds.
[  70.098909] cloud-init[614]: 2015-06-19 20:02:43,197 - util.py[WARNING]: Route info failed: Unexpected error while running command.
[  70.099204] cloud-init[614]: Command: ['netstat', '-rn']
[  70.099384] cloud-init[614]: Exit code: 1
[  70.099542] cloud-init[614]: Reason: -
[  70.099700] cloud-init[614]: Stdout: 'Kernel IP routing table\nDestination  Gateway  Genmask  Flags  MSS Window  irtt Iface\n'
[  70.099859] cloud-init[614]: Stderr: ''
[  70.122800] cloud-init[614]: ci-info: +++++++++++++++++++++++Net device info++++++++++++++++++++++++
[  70.130341] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[  70.136699] cloud-init[614]: ci-info: | Device |  Up  |  | Address  |    | Mask   |     | Hw-Address  |
[  70.136877] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[  70.137122] cloud-init[614]: ci-info: |  lo:   |  | True | 127.0.0.1 | 255.0.0.0 |  .  |
[  70.137325] cloud-init[614]: ci-info: | eth1:  | False |  .     |     | .  | fa:16:3e:8d:1e:65 |
[  70.137525] cloud-init[614]: ci-info: | eth0:  |  | True |  .     |     | .  | fa:16:3e:50:a2:a6 |
[  70.137717] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[  70.137905] cloud-init[614]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[  70.408174] cloud-init[614]: 2015-06-19 20:02:43,506 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(101, 'Network is unreachable'))]

(the last line repeated several times)

I'm not sure what the issue is. I haven't touched the default iptables on my CentOS 7 host. Everything is either the default or configured by neutron/nova.

Any help would be appreciated

Update 2015-07-05:

I was able to log in to the cirros image, but I was only able to confirm that the image wasn't getting its IP address from the OpenStack DHCP server. Any recommendations on troubleshooting this? Also, I noticed from the OpenStack Dashboard Network Topology that the cirros instance has two ports/interfaces facing the two networks, but I'm only seeing one network interface (eth0). Is eth0 connected to both networks? If not, which network is it connected to and how do I access the second network?

Network Not Coming Up in Instances

I've done a pretty thorough search of the Q&A's here using as many keywords I can think of and can't seem to find an answer to my issue (though I'm finding a lot of people with similar issues but no resolution)

I've followed the Quickstart Guide in RDO for creating an allinone Openstack installation and configured it to work with an existing network. I have an existing network (192.168.1.0/24) on my LAN to which this new CentOS 7 running OpenStack Kilo is connected. I followed the guide and made modifications to the startup scripts of my host OS to configure the br-ex bridge and enp7s0 interface as an interface on that bridge. I've also created a private network (192.168.100.0/24 following the defaults of the guide). I've imported 3 different images (CentOS 7, Fedora 22 and CirrOS qcow2 images from the official repositories and started instances assigning private and public interfaces to all. In all cases, I cannot access the VM instances from my host (either via ICMP ping or SSH). I can access the console but cannot login as the images don't have passwords. The trick to override the passwords either via the Post-Creation script (password: mypassword) or the API ('nova root-password <instance-id>') don't seem to work (the last one resulting in a:

ERROR (HTTPNotImplemented): Unable to set password on instance (HTTP 501) (Request-ID: req-65fa3132-4003-45dd-ab15-4070404ed2f2)
). From the Fedora console-log, it looks like the network interfaces aren't coming up:
A start job is running for LSB: Bring up/down networking (1min 6s / 5min 3s)[K[[32m OK [0m] Started LSB: Bring up/down networking.
[[32m OK [0m] Reached target Network is Online.
 Starting Initial cloud-init job (metadata service crawler)...
[ 70.053248] cloud-init[614]: Cloud-init v. 0.7.6 running 'init' at Fri, 19 Jun 2015 20:02:43 +0000. Up 70.03 seconds.
[ 70.098909] cloud-init[614]: 2015-06-19 20:02:43,197 - util.py[WARNING]: Route info failed: Unexpected error while running command.
[ 70.099204] cloud-init[614]: Command: ['netstat', '-rn']
[ 70.099384] cloud-init[614]: Exit code: 1
[ 70.099542] cloud-init[614]: Reason: -
[ 70.099700] cloud-init[614]: Stdout: 'Kernel IP routing table\nDestination Gateway Genmask Flags MSS Window irtt Iface\n'
[ 70.099859] cloud-init[614]: Stderr: ''
[ 70.122800] cloud-init[614]: ci-info: +++++++++++++++++++++++Net device info++++++++++++++++++++++++
[ 70.130341] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.136699] cloud-init[614]: ci-info: | Device | Up | Address | Mask | Hw-Address |
[ 70.136877] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.137122] cloud-init[614]: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . |
[ 70.137325] cloud-init[614]: ci-info: | eth1: | False | . | . | fa:16:3e:8d:1e:65 |
[ 70.137525] cloud-init[614]: ci-info: | eth0: | True | . | . | fa:16:3e:50:a2:a6 |
[ 70.137717] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.137905] cloud-init[614]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[ 70.408174] cloud-init[614]: 2015-06-19 20:02:43,506 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(101, 'Network is unreachable'))]

(the last line repeated several times)

I'm not sure what the issue is. I haven't touched the default iptables on my CentOS 7 host. Everything is either the default or configured by neutron/nova.

Any help would be appreciated

Update 2015-07-05:

I was able to log in to the cirros image, but I was only able to confirm that the image wasn't getting its IP address from the OpenStack DHCP server. Any recommendations on troubleshooting this? Also, I noticed from the OpenStack Dashboard Network Topology that the cirros instance has two ports/interfaces facing the two networks, but I'm only seeing one network interface (eth0). Is eth0 connected to both networks? If not, which network is it connected to and how do I access the second network?

Update 2015-07-05b:

Finally! I was able to get some network access. I set the IP address manually and I had to set it to the IP address of the external_network. I tried setting the interface to the IP address of the internal_network, but I couldn't ping the router. Setting it to the external_network allowed access to all of my machines (and vice-verse). Investigating further.

Network Not Coming Up in Instances

I've done a pretty thorough search of the Q&A's here using as many keywords I can think of and can't seem to find an answer to my issue (though I'm finding a lot of people with similar issues but no resolution)

I've followed the Quickstart Guide in RDO for creating an allinone Openstack installation and configured it to work with an existing network. I have an existing network (192.168.1.0/24) on my LAN to which this new CentOS 7 running OpenStack Kilo is connected. I followed the guide and made modifications to the startup scripts of my host OS to configure the br-ex bridge and enp7s0 interface as an interface on that bridge. I've also created a private network (192.168.100.0/24 following the defaults of the guide). I've imported 3 different images (CentOS 7, Fedora 22 and CirrOS qcow2 images from the official repositories and started instances assigning private and public interfaces to all. In all cases, I cannot access the VM instances from my host (either via ICMP ping or SSH). I can access the console but cannot login as the images don't have passwords. The trick to override the passwords either via the Post-Creation script (password: mypassword) or the API ('nova root-password <instance-id>') don't seem to work (the last one resulting in a:

ERROR (HTTPNotImplemented): Unable to set password on instance (HTTP 501) (Request-ID: req-65fa3132-4003-45dd-ab15-4070404ed2f2)
). From the Fedora console-log, it looks like the network interfaces aren't coming up:
A start job is running for LSB: Bring up/down networking (1min 6s / 5min 3s)[K[[32m OK [0m] Started LSB: Bring up/down networking.
[[32m OK [0m] Reached target Network is Online.
 Starting Initial cloud-init job (metadata service crawler)...
[ 70.053248] cloud-init[614]: Cloud-init v. 0.7.6 running 'init' at Fri, 19 Jun 2015 20:02:43 +0000. Up 70.03 seconds.
[ 70.098909] cloud-init[614]: 2015-06-19 20:02:43,197 - util.py[WARNING]: Route info failed: Unexpected error while running command.
[ 70.099204] cloud-init[614]: Command: ['netstat', '-rn']
[ 70.099384] cloud-init[614]: Exit code: 1
[ 70.099542] cloud-init[614]: Reason: -
[ 70.099700] cloud-init[614]: Stdout: 'Kernel IP routing table\nDestination Gateway Genmask Flags MSS Window irtt Iface\n'
[ 70.099859] cloud-init[614]: Stderr: ''
[ 70.122800] cloud-init[614]: ci-info: +++++++++++++++++++++++Net device info++++++++++++++++++++++++
[ 70.130341] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.136699] cloud-init[614]: ci-info: | Device | Up | Address | Mask | Hw-Address |
[ 70.136877] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.137122] cloud-init[614]: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . |
[ 70.137325] cloud-init[614]: ci-info: | eth1: | False | . | . | fa:16:3e:8d:1e:65 |
[ 70.137525] cloud-init[614]: ci-info: | eth0: | True | . | . | fa:16:3e:50:a2:a6 |
[ 70.137717] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.137905] cloud-init[614]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[ 70.408174] cloud-init[614]: 2015-06-19 20:02:43,506 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(101, 'Network is unreachable'))]

(the last line repeated several times)

I'm not sure what the issue is. I haven't touched the default iptables on my CentOS 7 host. Everything is either the default or configured by neutron/nova.

Any help would be appreciated

Update 2015-07-05:

I was able to log in to the cirros image, but I was only able to confirm that the image wasn't getting its IP address from the OpenStack DHCP server. Any recommendations on troubleshooting this? Also, I noticed from the OpenStack Dashboard Network Topology that the cirros instance has two ports/interfaces facing the two networks, but I'm only seeing one network interface (eth0). Is eth0 connected to both networks? If not, which network is it connected to and how do I access the second network?

Update 2015-07-05b:

Finally! I was able to get some network access. I set the IP address manually and I had to set it to the IP address of the external_network. I tried setting the interface to the IP address of the internal_network, but I couldn't ping the router. Setting it to the external_network allowed access to all of my machines (and vice-verse). Investigating further.

Update 2015-07-05c:

Getting close to a resolution. So the cirros guess OS is setting out a DHCPDISCOVER request AND my network DHCP server (not part of the OpenStack, err, stack) is getting it and sending out a DHCPOFFER response. It's this response, however, that is not getting to my virtual machine. It's either the physical interface on my OpenStack host not picking up messages addressed to the virtual machine's MAC address (the cirros one) or there's some kind of firewall configuration on the CentOS host that is blocking it. I'm doubting it's the second, but is there any way to confirm the first? Is there some special command that tells the TCP/IP stack on the OpenStack host machine to switch to promiscuous mode and pick up messages addressed to the virtual machine MAC address?

Network Not Coming Up in Instances

I've done a pretty thorough search of the Q&A's here using as many keywords I can think of and can't seem to find an answer to my issue (though I'm finding a lot of people with similar issues but no resolution)

I've followed the Quickstart Guide in RDO for creating an allinone Openstack installation and configured it to work with an existing network. I have an existing network (192.168.1.0/24) on my LAN to which this new CentOS 7 running OpenStack Kilo is connected. I followed the guide and made modifications to the startup scripts of my host OS to configure the br-ex bridge and enp7s0 interface as an interface on that bridge. I've also created a private network (192.168.100.0/24 following the defaults of the guide). I've imported 3 different images (CentOS 7, Fedora 22 and CirrOS qcow2 images from the official repositories and started instances assigning private and public interfaces to all. In all cases, I cannot access the VM instances from my host (either via ICMP ping or SSH). I can access the console but cannot login as the images don't have passwords. The trick to override the passwords either via the Post-Creation script (password: mypassword) or the API ('nova root-password <instance-id>') don't seem to work (the last one resulting in a:

ERROR (HTTPNotImplemented): Unable to set password on instance (HTTP 501) (Request-ID: req-65fa3132-4003-45dd-ab15-4070404ed2f2)
). From the Fedora console-log, it looks like the network interfaces aren't coming up:
A start job is running for LSB: Bring up/down networking (1min 6s / 5min 3s)[K[[32m OK [0m] Started LSB: Bring up/down networking.
[[32m OK [0m] Reached target Network is Online.
 Starting Initial cloud-init job (metadata service crawler)...
[ 70.053248] cloud-init[614]: Cloud-init v. 0.7.6 running 'init' at Fri, 19 Jun 2015 20:02:43 +0000. Up 70.03 seconds.
[ 70.098909] cloud-init[614]: 2015-06-19 20:02:43,197 - util.py[WARNING]: Route info failed: Unexpected error while running command.
[ 70.099204] cloud-init[614]: Command: ['netstat', '-rn']
[ 70.099384] cloud-init[614]: Exit code: 1
[ 70.099542] cloud-init[614]: Reason: -
[ 70.099700] cloud-init[614]: Stdout: 'Kernel IP routing table\nDestination Gateway Genmask Flags MSS Window irtt Iface\n'
[ 70.099859] cloud-init[614]: Stderr: ''
[ 70.122800] cloud-init[614]: ci-info: +++++++++++++++++++++++Net device info++++++++++++++++++++++++
[ 70.130341] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.136699] cloud-init[614]: ci-info: | Device | Up | Address | Mask | Hw-Address |
[ 70.136877] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.137122] cloud-init[614]: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . |
[ 70.137325] cloud-init[614]: ci-info: | eth1: | False | . | . | fa:16:3e:8d:1e:65 |
[ 70.137525] cloud-init[614]: ci-info: | eth0: | True | . | . | fa:16:3e:50:a2:a6 |
[ 70.137717] cloud-init[614]: ci-info: +--------+-------+-----------+-----------+-------------------+
[ 70.137905] cloud-init[614]: ci-info: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Route info failed!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[ 70.408174] cloud-init[614]: 2015-06-19 20:02:43,506 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(101, 'Network is unreachable'))]

(the last line repeated several times)

I'm not sure what the issue is. I haven't touched the default iptables on my CentOS 7 host. Everything is either the default or configured by neutron/nova.

Any help would be appreciated

Update 2015-07-05:

I was able to log in to the cirros image, but I was only able to confirm that the image wasn't getting its IP address from the OpenStack DHCP server. Any recommendations on troubleshooting this? Also, I noticed from the OpenStack Dashboard Network Topology that the cirros instance has two ports/interfaces facing the two networks, but I'm only seeing one network interface (eth0). Is eth0 connected to both networks? If not, which network is it connected to and how do I access the second network?

Update 2015-07-05b:

Finally! I was able to get some network access. I set the IP address manually and I had to set it to the IP address of the external_network. I tried setting the interface to the IP address of the internal_network, but I couldn't ping the router. Setting it to the external_network allowed access to all of my machines (and vice-verse). Investigating further.

Update 2015-07-05c:

Getting close to a resolution. So the cirros guess OS is setting out a DHCPDISCOVER request AND my network DHCP server (not part of the OpenStack, err, stack) is getting it and sending out a DHCPOFFER response. It's this response, however, that is not getting to my virtual machine. It's either the physical interface on my OpenStack host not picking up messages addressed to the virtual machine's MAC address (the cirros one) or there's some kind of firewall configuration on the CentOS host that is blocking it. I'm doubting it's the second, but is there any way to confirm the first? Is there some special command that tells the TCP/IP stack on the OpenStack host machine to switch to promiscuous mode and pick up messages addressed to the virtual machine MAC address?address? Also, should I set up a DHCP server on the OpenStack nova/neutron platform and be the DHCP server for all VMs? Would this interfere with my physical network's DHCP server?