Revision history [back]

click to hide/show revision 1
initial version

Running instance unable to ping gateway

Hi folks,

I am running a Juno installation on Centos7 on 3 nodes. The compute node is physical and the controller and network nodes are KVM virtual machines. I have followed the OpenStack installation guide for Centos 7.

I am stuck at the point of my launched VM instance being unable to hit it's gateway on the demo-router.

@192.168.1.7 # ping 192.168.1.1
100% packet loss

Everything else seems to be in order -

I can ping the external interface and I can ping the internal gateway when using an ip netns command as per:

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.021 ms

The router interface is receiving arp requests

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff tcpdump -i qr-71aad27a-68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qr-71aad27a-68, link-type EN10MB (Ethernet), capture size 65535 bytes
10:30:02.477929 ARP, Request who-has network tell 192.168.1.7, length 28
10:30:02.477937 ARP, Reply network is-at fa:16:3e:bc:45:20 (oui Unknown), length 28

and is successfully getting the MAC for the interface

@network ~]#ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.7              ether   fa:16:3e:37:fa:b9   C                     qr

The qrouter routing table looks correct.

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.10.110.1     0.0.0.0         UG        0 0          0 qg-5a67ac82-1c
10.10.110.0     0.0.0.0         255.255.255.0   U         0 0          0 qg-5a67ac82-1c
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 qr-71aad27a-68

The arp replies are never getting back to the host and I am unable to explain why. So my vm's are completely cut off from the external network but can communicate between themselves on the 192.168.1.0 network.

Some details -

Neutron looks good to me:

[@controller ~]\:admin$ neutron agent-list
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host       | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| 5a4588f7-9985-48d7-be85-e655f6586e27 | DHCP agent         | network    | :-)   | True           | neutron-dhcp-agent        |
| 5ef0bf1c-7917-45c6-9818-42d11db61bd6 | Open vSwitch agent | ostack     | :-)   | True           | neutron-openvswitch-agent |
| 77d05a25-b7e3-4e7a-9d14-ead1002a4966 | Open vSwitch agent | network    | :-)   | True           | neutron-openvswitch-agent |
| a9cb467f-c9d5-44f5-88cf-f11293812aa2 | Metadata agent     | controller | :-)   | True           | neutron-metadata-agent    |
| d46a99c4-92a0-4afe-b7bd-0a177075deb3 | Metadata agent     | network    | :-)   | True           | neutron-metadata-agent    |
| eb18ef39-1779-41a8-944a-92d1c69fa72a | L3 agent           | network    | :-)   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+

Network list:

@controller ~]\:admin$ neutron net-list
+--------------------------------------+----------+-----------------------------------------------------+
| id                                   | name     | subnets                                             |
+--------------------------------------+----------+-----------------------------------------------------+
| febb46dd-0891-4061-a11e-5879a7558238 | demo-net | e2b5a508-c5a0-4ca5-be50-40aa3347b49c 192.168.1.0/24 |
| 26dd5f73-8bfd-46cb-9f7d-cb57c0b1d851 | ext-net  | 1f3b04b6-186a-45fb-b752-af7996e0e903 10.10.110.0/24 |
+--------------------------------------+----------+-----------------------------------------------------+

I am baffled and would really like some help here please!

Regards,

Marcus.

Running instance unable to ping gateway

Hi folks,

I am running a Juno installation on Centos7 on 3 nodes. The compute node is physical and the controller and network nodes are KVM virtual machines. I have followed the OpenStack installation guide for Centos 7.

I am stuck at the point of my launched VM instance being unable to hit it's gateway on the demo-router.

@192.168.1.7 # ping 192.168.1.1
100% packet loss

Everything else seems to be in order -

I can ping the external interface and I can ping the internal gateway when using an ip netns command as per:

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.021 ms

The router interface is receiving arp requests

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff tcpdump -i qr-71aad27a-68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qr-71aad27a-68, link-type EN10MB (Ethernet), capture size 65535 bytes
10:30:02.477929 ARP, Request who-has network tell 192.168.1.7, length 28
10:30:02.477937 ARP, Reply network is-at fa:16:3e:bc:45:20 (oui Unknown), length 28

and is successfully getting the MAC for the interface

@network ~]#ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff arp
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.7              ether   fa:16:3e:37:fa:b9   C                     qr

The qrouter routing table looks correct.

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.10.110.1     0.0.0.0         UG        0 0          0 qg-5a67ac82-1c
10.10.110.0     0.0.0.0         255.255.255.0   U         0 0          0 qg-5a67ac82-1c
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 qr-71aad27a-68

The arp replies are never getting back to the host and I am unable to explain why. So my vm's are completely cut off from the external network but can communicate between themselves on the 192.168.1.0 network.

Some details -

Security Group is open:

Ingress IPv4    Any -   default 
Egress  IPv4    Any -   0.0.0.0/0 (CIDR)

Neutron looks good to me:

[@controller ~]\:admin$ neutron agent-list
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host       | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| 5a4588f7-9985-48d7-be85-e655f6586e27 | DHCP agent         | network    | :-)   | True           | neutron-dhcp-agent        |
| 5ef0bf1c-7917-45c6-9818-42d11db61bd6 | Open vSwitch agent | ostack     | :-)   | True           | neutron-openvswitch-agent |
| 77d05a25-b7e3-4e7a-9d14-ead1002a4966 | Open vSwitch agent | network    | :-)   | True           | neutron-openvswitch-agent |
| a9cb467f-c9d5-44f5-88cf-f11293812aa2 | Metadata agent     | controller | :-)   | True           | neutron-metadata-agent    |
| d46a99c4-92a0-4afe-b7bd-0a177075deb3 | Metadata agent     | network    | :-)   | True           | neutron-metadata-agent    |
| eb18ef39-1779-41a8-944a-92d1c69fa72a | L3 agent           | network    | :-)   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+

Network list:

@controller ~]\:admin$ neutron net-list
+--------------------------------------+----------+-----------------------------------------------------+
| id                                   | name     | subnets                                             |
+--------------------------------------+----------+-----------------------------------------------------+
| febb46dd-0891-4061-a11e-5879a7558238 | demo-net | e2b5a508-c5a0-4ca5-be50-40aa3347b49c 192.168.1.0/24 |
| 26dd5f73-8bfd-46cb-9f7d-cb57c0b1d851 | ext-net  | 1f3b04b6-186a-45fb-b752-af7996e0e903 10.10.110.0/24 |
+--------------------------------------+----------+-----------------------------------------------------+

I am baffled and would really like some help here please!

Regards,

Marcus.

Running instance unable to ping gateway

Hi folks,

I am running a Juno installation on Centos7 on 3 nodes. The compute node is physical and the controller and network nodes are KVM virtual machines. I have followed the OpenStack installation guide for Centos 7.

I am stuck at the point of my launched VM instance being unable to hit it's gateway on the demo-router.

@192.168.1.7 # ping 192.168.1.1
100% packet loss

Everything else seems to be in order -

I can ping the external interface and I can ping the internal gateway when using an ip netns command as per:

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.021 ms

The router interface is receiving arp requests

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff tcpdump -i qr-71aad27a-68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qr-71aad27a-68, link-type EN10MB (Ethernet), capture size 65535 bytes
10:30:02.477929 ARP, Request who-has network tell 192.168.1.7, length 28
10:30:02.477937 ARP, Reply network is-at fa:16:3e:bc:45:20 (oui Unknown), length 28

and is successfully getting the MAC for the interface

 @network ~]#ip ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff arp
arp 192.168.1.7
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.7              ether   fa:16:3e:37:fa:b9   C                     qr
qr-71aad27a-68

The qrouter routing table looks correct.

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.10.110.1     0.0.0.0         UG        0 0          0 qg-5a67ac82-1c
10.10.110.0     0.0.0.0         255.255.255.0   U         0 0          0 qg-5a67ac82-1c
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 qr-71aad27a-68

The router interface details are here.

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff ifconfig qr-71aad27a-68
qr-71aad27a-68: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1400
        inet 192.168.1.1  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::f816:3eff:febc:4520  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:bc:45:20  txqueuelen 0  (Ethernet)
        RX packets 31966  bytes 1344198 (1.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32005  bytes 1362126 (1.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The RX and TX packet count on the router goes up every time an arp request/reply is received/sent while on the vm node only the TX packet count increments, the RX count stays the same.

So, the arp replies are never getting back to the host and I am unable to explain why. So my why.

My vm's are completely cut off from the external network but can communicate between themselves on the 192.168.1.0 network.

Some details -

Security Group is open:

Ingress IPv4    Any -   default 
Egress  IPv4    Any -   0.0.0.0/0 (CIDR)

Neutron looks good to me:

[@controller ~]\:admin$ neutron agent-list
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host       | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| 5a4588f7-9985-48d7-be85-e655f6586e27 | DHCP agent         | network    | :-)   | True           | neutron-dhcp-agent        |
| 5ef0bf1c-7917-45c6-9818-42d11db61bd6 | Open vSwitch agent | ostack     | :-)   | True           | neutron-openvswitch-agent |
| 77d05a25-b7e3-4e7a-9d14-ead1002a4966 | Open vSwitch agent | network    | :-)   | True           | neutron-openvswitch-agent |
| a9cb467f-c9d5-44f5-88cf-f11293812aa2 | Metadata agent     | controller | :-)   | True           | neutron-metadata-agent    |
| d46a99c4-92a0-4afe-b7bd-0a177075deb3 | Metadata agent     | network    | :-)   | True           | neutron-metadata-agent    |
| eb18ef39-1779-41a8-944a-92d1c69fa72a | L3 agent           | network    | :-)   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+

Network list:

@controller ~]\:admin$ neutron net-list
+--------------------------------------+----------+-----------------------------------------------------+
| id                                   | name     | subnets                                             |
+--------------------------------------+----------+-----------------------------------------------------+
| febb46dd-0891-4061-a11e-5879a7558238 | demo-net | e2b5a508-c5a0-4ca5-be50-40aa3347b49c 192.168.1.0/24 |
| 26dd5f73-8bfd-46cb-9f7d-cb57c0b1d851 | ext-net  | 1f3b04b6-186a-45fb-b752-af7996e0e903 10.10.110.0/24 |
+--------------------------------------+----------+-----------------------------------------------------+

I am baffled and would really like some help here please!

Regards,

Marcus.

Running instance unable to ping gateway

Hi folks,

I am running a Juno installation on Centos7 on 3 nodes. The compute node is physical and the controller and network nodes are KVM virtual machines. I have followed the OpenStack installation guide for Centos 7.

I am stuck at the point of my launched VM instance being unable to hit it's gateway on the demo-router.

@192.168.1.7 # ping 192.168.1.1
100% packet loss

Everything else seems to be in order -

I can ping the external interface and I can ping the internal gateway when using an ip netns command as per:

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.021 ms

The router interface is receiving arp requests

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff tcpdump -i qr-71aad27a-68
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on qr-71aad27a-68, link-type EN10MB (Ethernet), capture size 65535 bytes
10:30:02.477929 ARP, Request who-has network tell 192.168.1.7, length 28
10:30:02.477937 ARP, Reply network is-at fa:16:3e:bc:45:20 (oui Unknown), length 28

and is successfully getting the MAC for the interface

 @network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff arp 192.168.1.7
Address                  HWtype  HWaddress           Flags Mask            Iface
192.168.1.7              ether   fa:16:3e:37:fa:b9   C                     qr-71aad27a-68

The qrouter routing table looks correct.

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff netstat -nr
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.10.110.1     0.0.0.0         UG        0 0          0 qg-5a67ac82-1c
10.10.110.0     0.0.0.0         255.255.255.0   U         0 0          0 qg-5a67ac82-1c
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 qr-71aad27a-68

The router interface details are here.

@network ~]# ip netns exec qrouter-e77947ea-d03c-4b90-aebe-2d394faf10ff ifconfig qr-71aad27a-68
qr-71aad27a-68: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1400
        inet 192.168.1.1  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::f816:3eff:febc:4520  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:bc:45:20  txqueuelen 0  (Ethernet)
        RX packets 31966  bytes 1344198 (1.2 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 32005  bytes 1362126 (1.2 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The RX and TX packet count on the router goes up every time an arp request/reply is received/sent while on the vm node only the TX packet count increments, the RX count stays the same.

So, the arp replies are never getting back to the host and I am unable to explain why.

My vm's are completely cut off from the external network but can communicate between themselves on the 192.168.1.0 network.

Some details -

Security Group is open:

Ingress IPv4    Any -   default 
Egress  IPv4    Any -   0.0.0.0/0 (CIDR)

Neutron looks good to me:

[@controller ~]\:admin$ neutron agent-list
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| id                                   | agent_type         | host       | alive | admin_state_up | binary                    |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+
| 5a4588f7-9985-48d7-be85-e655f6586e27 | DHCP agent         | network    | :-)   | True           | neutron-dhcp-agent        |
| 5ef0bf1c-7917-45c6-9818-42d11db61bd6 | Open vSwitch agent | ostack     | :-)   | True           | neutron-openvswitch-agent |
| 77d05a25-b7e3-4e7a-9d14-ead1002a4966 | Open vSwitch agent | network    | :-)   | True           | neutron-openvswitch-agent |
| a9cb467f-c9d5-44f5-88cf-f11293812aa2 | Metadata agent     | controller | :-)   | True           | neutron-metadata-agent    |
| d46a99c4-92a0-4afe-b7bd-0a177075deb3 | Metadata agent     | network    | :-)   | True           | neutron-metadata-agent    |
| eb18ef39-1779-41a8-944a-92d1c69fa72a | L3 agent           | network    | :-)   | True           | neutron-l3-agent          |
+--------------------------------------+--------------------+------------+-------+----------------+---------------------------+

Network list:

@controller ~]\:admin$ neutron net-list
+--------------------------------------+----------+-----------------------------------------------------+
| id                                   | name     | subnets                                             |
+--------------------------------------+----------+-----------------------------------------------------+
| febb46dd-0891-4061-a11e-5879a7558238 | demo-net | e2b5a508-c5a0-4ca5-be50-40aa3347b49c 192.168.1.0/24 |
| 26dd5f73-8bfd-46cb-9f7d-cb57c0b1d851 | ext-net  | 1f3b04b6-186a-45fb-b752-af7996e0e903 10.10.110.0/24 |
+--------------------------------------+----------+-----------------------------------------------------+

I am baffled and would really like some help here please!

Regards,

Marcus.