took recent grizzly quantum patch and it broke our namespace [closed]

asked 2013-11-05 10:52:26 -0500

B.Callanan gravatar image

updated 2013-11-06 16:43:47 -0500

I updated our openstack Nova/Quantum compute server with a recent patch and now the network namespace is not being created properly. The config files haven't changed but now the namespace for a newly created network and VM(s) is not accessible.

# /sbin/ip netns exec qdhcp-4fd25554-a549-4ffe-82b8-9c9bf6d06d7a ssh -i /root/temp_key.pem
Cannot open network namespace: No such file or directory

The server is set to use namespace but the namespace is not getting created.

Something broke and I can't figure out what it is.

Update:

The symptoms we are seeing are transient. I dont know if the issue is related to the Scheduling issue. I read the bug report and reviewed the code. From what I've read the fix at:

seems to say that it can be ignored.

Yes, there are namespaces that have been created and those still exist. We applied a patch and now we can't consistently create namespaces. Its hosed and we cant back out the patches that caused it. Seems the namespace for any new networks is not even being created.

/var/log/openvswitch/ovs-vswitchd.log is saying for every new network device. So that's atleast consistent in that the network isn't being given a namespace.

2013-11-05T20:05:54Z|00700|netdev_linux|WARN|ethtool command ETHTOOL_GSET on network device tapb221ea79-0d failed: No such device

Seeing these errors in /var/log/messages

 Nov  6 09:10:34 wma-dev02 object-replicator Object replication complete. (0.00 minutes)
 Nov  6 09:10:36 wma-dev02 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --timeout=5 -- --may-exist add-port br-int tapd545cfce-49 -- set Interface tapd545cfce-49 "external-ids:attached-mac=\"FA:16:3E:8C:EB:3F\"" -- set Interface tapd545cfce-49 "external-ids:iface-id=\"d545cfce-4943-4e54-abc5-8fdef4857a82\"" -- set Interface tapd545cfce-49 "external-ids:vm-id=\"1dbd3d7c-f90f-4519-bfa3-2a11c4e06fd1\"" -- set Interface tapd545cfce-49 external-ids:iface-status=active
 Nov  6 09:10:37 wma-dev02 kernel: device tapd545cfce-49 entered promiscuous mode
 **Nov  6 09:10:37 wma-dev02 NetworkManager[3250]: <warn> /sys/devices/virtual/net/tapd545cfce-49: couldn't determine device driver; ignoring...
 Nov  6 09:10:37 wma-dev02 qemu-kvm: Could not find keytab file: /etc/qemu/krb5.tab: No such file or directory**

These are the current patches

Linux version 2.6.32-358.118.1.openstack.el6.x86_64 (mockbuild@x86-007.build.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Aug 14 13:18:08 EDT 2013

~(keystone_admin)]# rpm --all -q | grep openstack
kernel-2.6.32-358.118.1.openstack.el6.x86_64
openstack-swift-1.8.0-2.el6.noarch
openstack-nova-compute-2013.1.4-1.el6.noarch
kernel-2.6.32-358.111.1.openstack.el6.x86_64
openstack-swift-container-1.8.0-2.el6.noarch
openstack-nova-common-2013.1.4-1.el6.noarch
openstack-dashboard-2013.1.4-1.el6.noarch
kernel-devel-2.6.32-358.114.1.openstack.el6.gre.2.x86_64
kernel-devel-2.6.32-358.118.1.openstack.el6.x86_64
openstack-nova-novncproxy-2013.1.4-1.el6.noarch
kernel-headers-2.6.32-358.123.2.openstack.el6.x86_64
kernel-2.6.32-358.114.1.openstack.el6.gre.2.x86_64
openstack-utils-2013.2-1.el6.1.noarch
openstack-nova-scheduler-2013.1.4-1.el6.noarch
openstack-packstack-2013.1.1-0.29.dev683.el6 ...
(more)
edit retag flag offensive reopen merge delete

Closed for the following reason question is not relevant or outdated by rbowen
close date 2016-09-27 11:16:32.456393

Comments

Got this in the Quantum Server log 2013-11-05 11:13:07 WARNING [quantum.db.agentschedulers_db] Fail scheduling network {'status': u'ACTIVE', 'subnets': [u'83c983c0-7d4d-4610-9b53-122e41bad833'], 'name': u'ESO__IMSTest1__77370876-badc-4232-91f9-167f54693e7b', 'provider:physical_network': u'compute', 'admin_state_up': True, 'tenant_id': u'9281f9022a0f413db45f6a34fe9f8bb5', 'provider:network_type': u'vlan', 'router:external': False, 'shared': False, 'id': u'4c761d85-0bd9-4327-8915-f94da980700f', 'provider:segmentation_id': 8L} It says "fail sceduling network." anyone ever se this?????

B.Callanan gravatar imageB.Callanan ( 2013-11-05 12:29:46 -0500 )edit
1

Does `ip netns` by itself even *show* any network namespaces? Also, you're better off updating your *question* with new information rather than replying in comments (because you have more flexible formatting options).

larsks gravatar imagelarsks ( 2013-11-05 14:21:25 -0500 )edit

Closing: This question relates to a version of OpenStack which has been marked End Of Life. If you are still experiencing this problem with a more recent version of OpenStack, please open a new question with updated details. Thanks!

rbowen gravatar imagerbowen ( 2016-09-27 11:16:28 -0500 )edit