Ask Your Question

Mathias Ewald's profile - activity

2016-08-09 05:07:05 -0500 received badge  Famous Question (source)
2016-08-09 05:07:05 -0500 received badge  Notable Question (source)
2015-08-21 03:43:26 -0500 received badge  Good Question (source)
2015-08-21 03:29:45 -0500 received badge  Nice Question (source)
2015-08-19 08:31:03 -0500 received badge  Famous Question (source)
2015-08-19 08:31:03 -0500 received badge  Notable Question (source)
2015-02-11 00:27:17 -0500 received badge  Self-Learner (source)
2015-02-11 00:26:13 -0500 received badge  Famous Question (source)
2015-02-06 02:53:11 -0500 received badge  Popular Question (source)
2015-02-05 16:27:45 -0500 received badge  Notable Question (source)
2015-02-05 12:24:44 -0500 asked a question Nova Cells: Question about Networking and Storage

Hi, I am just wrapping my head around the concept of cells. Here two questions I cannot figure out:

Networking: From a couple of Youtube videos and slidedecks on Slideshare I noticed, that every child cell runs its own instance of Neutron. So I end up running kind of the same Nova service but multiple independent Neutron setups. So when I request a new instance from the parent cells nova-api service, I am going to pass it the ID of the network to attach to. But as nova schedules the instance to any cell it controls, how does it work? Is the network created in every child cell with the same ID? But as far as I know Neutron is not aware of any cells. The other option I am seeing is to run a single instance of neutron and share it accross cells.

Storage: Pretty much the same question as before but this time to be answered for volumes.

Any help is appreciated :)

2015-02-05 12:15:24 -0500 commented answer Making OpenStack API public

Thanks for the steps :) Your set up is assuming users from the inside can access the public IPs, right? There is no way to make that unnecessary, is there?

2015-02-05 02:53:43 -0500 received badge  Popular Question (source)
2015-02-04 12:21:54 -0500 asked a question Making OpenStack API public

Hi, I am trying to find a good way to give the public (Internet) access to an existing OpenStack cloud. Let's assume the environment was set up using private IP addresses, say 192.168.10.0/24 for all Open Stack services. Now I would like to allow customers to deploy instances and work with them from the internet. There are 2 problem I am seeing from the top of my head if I tried to NAT OpenStack services to public IP addresses:

1) Keystone's service catalog points to URLs (endpoints) that cannot be reached from the internet. All URLs point to somewhere on the 192.168.10.0/24 network.

2) The URL that is returned by nova for VNC access also points to an internal address.

Well, one way would be to deploy OpenStack controller(s) in a DMZ right from the beginning but that is a very ugly way in my opinion.

So my questions: How is that typically handled?

2015-01-15 09:40:25 -0500 received badge  Nice Question (source)
2014-11-25 16:00:28 -0500 received badge  Famous Question (source)
2014-11-17 07:13:28 -0500 received badge  Famous Question (source)
2014-10-01 13:26:28 -0500 received badge  Taxonomist
2014-09-19 10:21:26 -0500 received badge  Notable Question (source)
2014-09-19 10:21:26 -0500 received badge  Famous Question (source)
2014-09-19 10:20:33 -0500 received badge  Popular Question (source)
2014-09-18 07:52:21 -0500 received badge  Famous Question (source)
2014-09-17 08:56:06 -0500 received badge  Notable Question (source)
2014-09-04 06:46:50 -0500 received badge  Popular Question (source)
2014-08-25 15:47:32 -0500 received badge  Necromancer (source)
2014-07-30 12:20:43 -0500 received badge  Famous Question (source)
2014-07-15 10:37:38 -0500 commented answer Neutron gre with mtu 1454 cause windows network speed too low

I think I might have found a more elegant way:

/etc/nova/nova.conf

...
network_device_mtu = 9000
...

The instance just created was set to 9000, removing the line and creating a new instance gives mit 1500 MTU on the tap device. Seems to be working!

But thinking about it more closely: GRE happens later so this should not be helping with the MTU issues. Just nice to know ^^

2014-07-15 09:13:47 -0500 asked a question Irregular Network Latency to Instances

Hi, I noticed repeating high network latencies to my instances:

64 bytes from XXX.XXX.XXX.XXX: icmp_req=44 ttl=56 time=1387 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=45 ttl=56 time=380 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=46 ttl=56 time=4.79 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=47 ttl=56 time=4.64 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=48 ttl=56 time=5.07 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=49 ttl=56 time=4.61 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=50 ttl=56 time=4.77 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=51 ttl=56 time=5.21 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=52 ttl=56 time=4.73 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=53 ttl=56 time=4.54 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=54 ttl=56 time=1596 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=55 ttl=56 time=597 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=56 ttl=56 time=4.82 ms
64 bytes from XXX.XXX.XXX.XXX: icmp_req=57 ttl=56 time=4.78 ms
^C
--- XXX.XXX.XXX.XXX ping statistics ---
65 packets transmitted, 65 received, 0% packet loss, time 64109ms
rtt min/avg/max/mdev = 4.441/75.162/1596.820/277.746 ms, pipe 2

This ping goes through a router VM, a neutron network node to an instance via floating IP. I am not seeing this to the router-VM (392ms happend only once):

49 packets transmitted, 49 received, 0% packet loss, time 48080ms
rtt min/avg/max/mdev = 3.131/11.272/392.985/55.095 ms

But I can even see this between instances on the same GRE-based network:

64 bytes from 192.168.10.2: icmp_req=15 ttl=64 time=0.604 ms
64 bytes from 192.168.10.2: icmp_req=16 ttl=64 time=584 ms
64 bytes from 192.168.10.2: icmp_req=17 ttl=64 time=0.622 ms
64 bytes from 192.168.10.2: icmp_req=18 ttl=64 time=0.714 ms
64 bytes from 192.168.10.2: icmp_req=19 ttl=64 time=0.687 ms
64 bytes from 192.168.10.2: icmp_req=20 ttl=64 time=0.676 ms
64 bytes from 192.168.10.2: icmp_req=21 ttl=64 time=0.635 ms
64 bytes from 192.168.10.2: icmp_req=22 ttl=64 time=0.612 ms
64 bytes from 192.168.10.2: icmp_req=23 ttl=64 time=1458 ms
64 bytes from 192.168.10.2: icmp_req=24 ttl=64 time=464 ms
64 bytes from 192.168.10.2: icmp_req=25 ttl=64 time=0.631 ms
64 bytes from 192.168.10.2: icmp_req=26 ttl=64 time=1074 ms
64 bytes from 192.168.10.2: icmp_req=27 ...
(more)
2014-07-15 08:57:39 -0500 commented answer Neutron gre with mtu 1454 cause windows network speed too low

I think I didn't explain myself properly. I know how to set an MTU permanently in Linux. But my question was related to tap* devices created by nova as they seem to keep me from setting the OVS mtu.

It is interesting that there is to view information on this topic. It really seems like everyone is either in a lab and satisfied with a working ping or sets the MTU in the guest via DHCP.

2014-07-15 02:43:51 -0500 commented answer Neutron gre with mtu 1454 cause windows network speed too low

It seems like I solved my problem by running a script that set mtu = 9000 for everyhing ip link show sees. Now br-transport shows 9000 and my VMs show proper behavior with mtu 1500. Any suggestions how to automate this behavior? How do I make sure newly provisioned instances get 9000 on their vifs so that the bridge mtu does not go down to 1500?

cheers Mathias

2014-07-14 10:47:18 -0500 commented answer Neutron gre with mtu 1454 cause windows network speed too low

Unfortunately, ip doesnt to the job either:

root@node00:~# ip link set mtu 9000 br-transport
root@node00:~# ip lin show br-transport
7: br-transport: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/ether 4e:41:9e:77:58:82 brd ff:ff:ff:ff:ff:ff
root@node00:~# ip lin show ovs-infra
9: ovs-infra: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/ether 00:25:90:9a:c6:54 brd ff:ff:ff:ff:ff:ff
root@node00:~# ip link set mtu 9000 ovs-infra
root@node00:~# ip link show ovs-infra
9: ovs-infra: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/ether 00:25:90:9a:c6:54 brd ff:ff:ff:ff:ff:ff
root@node00:~#

As br-transport is a fake bridge I though I'd give ovs-infra (the parent) a try but ... (more)

2014-07-14 10:03:03 -0500 commented answer Neutron gre with mtu 1454 cause windows network speed too low

Hi SamYaple, first I'd like to note that anything beyond 1500 Bytes MTU size is considered a jumbo frame - not only 8000 and above. Secondly, and more importantly, there is no use in configuring a 10Gbit or high connection with a higher MTU than 1500 by default. The setting itself is useless as long as nobody on the network actually makes use of it. The one exception is with overlay networking as GRE or VXLAN when normally-sized traffic for example from VMs has to be encapsulated without the knowledge of those. In this case additional header information has to be put in place increasing the MTU size of the physically transported frame.

The problem I am having is one of more specific nature. I am running Neutron with ML2 (OVS and GRE) and experience a complete loss of connectivity once I put some load on the line. Reestablishing the ... (more)

2014-07-14 06:26:18 -0500 answered a question Neutron gre with mtu 1454 cause windows network speed too low

The solution proposed in the varios documentations of settings the guests MTU to 1400 is complete non-sense in my opinion. Rather than being even more invasive into the guest operating system than we already are (cloud-init, etc) we should move the solution of this to the networking infrastructure: Increasing the MTU of the tunnel / transport VLAN should fix it without lowering the guest MTU. I am still trying to figure out which interfaces and bridges need to be reconfigured.

2014-07-09 11:08:17 -0500 answered a question What's the recommended way to snapshot images for cloud-init?

Unfortunately, it isn't. I tested with Ubuntun 12.04 and 14.04 cloud images. Here is what I do:

  1. deploy from original image with SSH keypair
  2. SSH into the instance works
  3. snapshot the instance
  4. deploy from the snapshot (on boot the screen is black so no information to get from there)
  5. SSH not working anymore.

From the serial console I can see this:

cloud-init start-local running: Wed, 09 Jul 2014 16:08:11 +0000. up 3.16 seconds
no instance data found in start-local
cloud-init-nonet waiting 120 seconds for a network device.
ci-info: lo    : 1 127.0.0.1       255.0.0.0       .
ci-info: eth0  : 1 192.168.10.114  255.255.255.0   fa:16:3e:22:89:7f
ci-info: route-0: 0.0.0.0         192.168.10.1    0.0.0.0         eth0   UG
ci-info: route-1: 169.254.0.0     0.0.0.0         255.255.0.0     eth0   U
ci-info: route-2: 192.168.10.0    0.0.0.0         255.255.255.0   eth0   U
cloud-init start running: Wed, 09 Jul 2014 16:08:25 +0000. up 16.80 seconds
2014-07-09 16:08:28,906 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [2/120s]: url error [[Errno 113] No route to host]
2014-07-09 16:08:31,906 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [5/120s]: url error [[Errno 113] No route to host]
2014-07-09 16:07:03,707 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [-82/120s]: url error [[Errno 113] No route to host]
2014-07-09 16:07:06,707 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [-79/120s]: url error [[Errno 113] No route to host]
2014-07-09 16:07:09,707 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [-76/120s]: url error [[Errno 113] No route to host]

It looks like the metadata service was not working. But I know it is as any newly spawned instance (ubuntu or cirros) can connect to it and retrieve their metadata.

Any ideas?

EDIT: I am sorry I have to admit that between steps 2 and 3 I installed xubuntu-desktop. I just did the plain procedure as above and it worked. Any suggestions why is failes with xubuntu-desktop installed?

2014-07-08 13:32:19 -0500 received badge  Famous Question (source)
2014-07-07 04:25:57 -0500 received badge  Notable Question (source)
2014-07-06 23:02:55 -0500 received badge  Popular Question (source)
2014-07-05 07:54:42 -0500 asked a question Multiple L3 Agents - Failed to bind port on host

Hi, I was struggling with multiple L3 Agents to support multiple external networks for a while now when I hit an error message I dont understand. But here my configuration first:

on the controller node:

root@controller:~# cat /etc/hosts
127.0.0.1       localhost

192.168.10.1            r1.lab.invalid                r1
192.168.10.10           controller.lab.invalid        controller
192.168.10.11           neutron-net.lab.invalid       neutron-net
192.168.10.11           neutron-net-l3-1.lab.invalid  neutron-net-l3-1
192.168.10.11           neutron-net-l3-2.lab.invalid  neutron-net-l3-2
192.168.10.100          node00.lab.invalid            node00


# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

root@controller:~# neutron net-list
+--------------------------------------+---------+-------------------------------------------------------+
| id                                   | name    | subnets                                               |
+--------------------------------------+---------+-------------------------------------------------------+
| f55ded23-08b1-4671-850a-50f134be406d | public1 | 88ccce32-d1d9-43f5-9160-733b4c33281f 88.198.249.48/28 |
| faaad07e-0466-44c2-b457-8de3876b82a2 | public0 | e36a335c-c265-4b11-96c3-07a0ee3abf0e 88.198.244.96/29 |
+--------------------------------------+---------+-------------------------------------------------------+

on my network node:

root@neutron-net:~# cat /etc/hosts
127.0.0.1       localhost

192.168.10.1            r1.lab.invalid                r1
192.168.10.10           controller.lab.invalid        controller
192.168.10.11           neutron-net.lab.invalid       neutron-net
192.168.10.11           neutron-net-l3-1.lab.invalid  neutron-net-l3-1
192.168.10.11           neutron-net-l3-2.lab.invalid  neutron-net-l3-2
192.168.10.100          node00.lab.invalid            node00


# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

root@neutron-net:~# cat /etc/neutron/l3_agent.ini | grep -v -E "^#" | grep -v -E "^\s*$"
[DEFAULT]
verbose = True
host = neutron-net-l3-1
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True
gateway_external_network_id = faaad07e-0466-44c2-b457-8de3876b82a2
handle_internal_only_routers = True
external_network_bridge = br-ex

root@neutron-net:~# cat /etc/neutron/l3_agent2.ini | grep -v -E "^#" | grep -v -E "^\s*$"
[DEFAULT]
verbose = True
host = neutron-net-l3-2
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True
gateway_external_network_id = f55ded23-08b1-4671-850a-50f134be406d
handle_internal_only_routers = False
external_network_bridge = br-ex1
root@neutron-net:~#

root@neutron-net:~# cat /etc/init/neutron-l3-agent.conf 
# vim:set ft=upstart ts=2 et:
description "Neutron L3 Agent"
author "Chuck Short <zulcss@ubuntu.com>"

start on runlevel [2345]
stop on runlevel [!2345]

respawn

chdir /var/run

pre-start script
  mkdir -p /var/run/neutron
  chown neutron:root /var/run/neutron
  # Check to see if openvswitch plugin in use by checking
  # status of cleanup upstart configuration
  if status neutron-ovs-cleanup; then
    start wait-for-state WAIT_FOR=neutron-ovs-cleanup WAIT_STATE=running WAITER=neutron-l3-agent
  fi
end script

exec start-stop-daemon --start --chuid neutron --exec /usr/bin/neutron-l3-agent -- \
  --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/l3_agent.ini \
  --config-file=/etc/neutron/fwaas_driver.ini --log-file=/var/log/neutron/l3-agent.log

root@neutron-net:~# cat /etc/init/neutron-l3-agent2.conf 
# vim:set ft=upstart ts=2 et:
description "Neutron L3 Agent 2"
author "Chuck Short <zulcss@ubuntu.com>"

start on runlevel [2345]
stop on runlevel [!2345]

respawn

chdir /var/run

pre-start script
  mkdir -p /var/run/neutron
  chown neutron:root /var/run/neutron
  # Check to see if openvswitch plugin in use by checking
  # status of cleanup upstart configuration
  if status neutron-ovs-cleanup; then
    start wait-for-state WAIT_FOR=neutron-ovs-cleanup WAIT_STATE=running WAITER=neutron-l3-agent2
  fi
end script

exec start-stop-daemon --start --chuid neutron --exec /usr/bin/neutron-l3-agent -- \
  --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/l3_agent2.ini \
  --config-file=/etc/neutron/fwaas_driver.ini --log-file=/var/log ...
(more)
2014-06-13 06:54:11 -0500 received badge  Famous Question (source)
2014-05-18 17:31:33 -0500 received badge  Famous Question (source)
2014-05-12 08:10:57 -0500 received badge  Popular Question (source)