Poor network performance for traffic that hit "Floating IP"

asked 2015-07-28 09:39:33 -0600

Herr-Herner gravatar image

updated 2015-07-28 10:12:02 -0600

We have installed OpenStack Kilo in our vSphere environment on Ubuntu Trusty 14.04. Everything is up and running, but I have a network bottleneck somewhere in my environment and I need some help to find it. Traffic from "Floating IPs" to my external server is satifactory:

OpenStack VM -> External Server

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.00 GBytes   862 Mbits/sec  366             sender
[  4]   0.00-10.00  sec  1.00 GBytes   860 Mbits/sec                  receiver

But the other direction provides insufficient performance:

External Server -> OpenStack VM

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   226 MBytes   190 Mbits/sec  2681             sender
[  4]   0.00-10.00  sec   225 MBytes   189 Mbits/sec                  receiver

I am trying to find out where the bootleneck exists. I started iperf3 on the network node and on the tenant's external router:

ip netns exec qrouter-a190687c-f795-4814-8f7a-d943e3df7315 iperf3 -s

External Server -> qrouter-a190687c-f795-4814-8f7a-d943e3df7315

[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   896 MBytes   752 Mbits/sec  3030             sender
[  4]   0.00-10.00  sec   894 MBytes   750 Mbits/sec                  receiver

There are two connections more to check:

  • External Server -> Network Node -> GRE -> Compute Node
  • Compute Node -> OpenStack VM

Can anybody help me to check these two connections? I do not know how to measure network throughput on the compute node when messages get transferred via the GRE tunnel and how can I check the network performance between the hosting compute node and the guest OpenStack VM?

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2015-07-29 09:07:54 -0600

echiu gravatar image

Some ideas to try:

  1. You can run traceroute on the compute node to the guest VM IPs. This will show the number of hops and the time per hop.
  2. Generate a report from your routes on network traffic

Is the network latency symptom intermittent or consistent?

edit flag offensive delete link more


It seems that the problem is caused by vSphere. Our "Promiscous Mode" interface gets all packets that should reach a OpenStack VM twice. Unfortunately, we have not found a persistent solution. After some time, we get duplicated packets, even when we got it working for a while.

Herr-Herner gravatar imageHerr-Herner ( 2015-07-31 03:08:01 -0600 )edit

The phenomenon is described here: https://ask.openstack.org/en/question...

Herr-Herner gravatar imageHerr-Herner ( 2015-07-31 03:08:56 -0600 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools



Asked: 2015-07-28 09:39:33 -0600

Seen: 648 times

Last updated: Jul 29 '15