Ask Your Question
0

udp performance is bad in neutron

asked 2015-01-26 21:14:12 -0500

Lance gravatar image

updated 2015-01-27 23:24:09 -0500

hi All I've done some network performance tests with neutron. With TCP, all seems fine, VM in/out is OK(about 800~900Mbps). But with UDP, if VM connects out with UDP, the performance is bad(only 200Mbps). However, if external clients connects and send udp packets to VM, the performance is OK, over 800Mps.

Any thoughts? I've changed the MTU size to 1454. Also changed GRO on network node(don't have big impact to me)

On guest node:
root@smta1:/home/media# ethtool --show-offload eth0
Features for eth0:
rx-checksumming: off
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: off [fixed]
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]

On network node: 
root@media-network /etc# ethtool --show-offload eth1 (this is the NIC for internal link)
Features for eth1:
rx-checksumming: on
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
-----------------------------------------

root@media-network /etc# ethtool --show-offload eth0 (this NIC connect to the external network)
Features for eth0:
rx-checksumming: on
tx-checksumming: on
        tx-checksum-ipv4: on
        tx-checksum-ip-generic: off [fixed]
        tx-checksum-ipv6: on
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: on
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-mpls-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: on
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]

Thanks

edit retag flag offensive close merge delete

Comments

I'm using GRE tunnels, since I've only 2 NIC, so the management network and data network go through the same eth0. I use iperf to test the performances

Lance gravatar imageLance ( 2015-01-26 21:16:33 -0500 )edit

try running iperf in the router namespace and test from the guest and from outside.

darragh-oreilly gravatar imagedarragh-oreilly ( 2015-02-04 14:54:35 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
-1

answered 2015-01-27 02:34:27 -0500

Lance gravatar image

What I don't understand is why the performance of sending udp package into VM is good, while sending package from VM to external pc is bad. The trace looks similar to me.

edit flag offensive delete link more
0

answered 2015-01-26 22:43:10 -0500

There is no TCP offload for UDP so GRE tunnels will be slow....

edit flag offensive delete link more

Comments

Any thing can do about this? Or do I have to change the UDP buffer size in applications? I tried to send each UDP package size smaller than an MTU(1426), the speed is about 300Mbps, still not good enough to me

Lance gravatar imageLance ( 2015-01-26 23:33:42 -0500 )edit

There as GRE over head added to each frame (about 30 bytes). Make sure that your guest MTU is not bigger than physical MTU .

Other than that there is not much you can do to get better performance outside of adding 10gig NICs plus jumbo frames. Still UDP will be slower.

sfcloudman gravatar imagesfcloudman ( 2015-01-27 00:32:38 -0500 )edit

Can you run ethtool ethX in the guest and on the HV hosting the guest.

sfcloudman gravatar imagesfcloudman ( 2015-01-27 22:34:26 -0500 )edit

I tried, if I turn tso off in VM, the performance of tcp will lower. gso/gro/tso changes on network node seems no impact. Info updated in the question

Lance gravatar imageLance ( 2015-01-27 23:21:06 -0500 )edit

Gro and gso are on the guest but off on the HV.

You need to make changes on the HV not just guest.

sfcloudman gravatar imagesfcloudman ( 2015-01-27 23:40:22 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

3 followers

Stats

Asked: 2015-01-26 21:14:12 -0500

Seen: 1,721 times

Last updated: Jan 27 '15