Ask Your Question
0

VM supposed to have two NIC, can't bring up second one

asked 2017-04-26 18:24:04 -0600

tiger gravatar image

Sigh.. I know this has been asked a LOT, and I have sifted through lots of answers, but can't find one that addresses my situation exactly. Running Packstack / Kilo on a server with 24x CPU and 96GB RAM / Centos7

I have two networks, management (192.168.71.0/24) and signaling (192.168.70.0/24)

[root@kvm7 ~(keystone_admin)]# openstack subnet list
+--------------------------------------+-------------------+--------------------------------------+-----------------+
| ID                                   | Name              | Network                              | Subnet          |
+--------------------------------------+-------------------+--------------------------------------+-----------------+
| 118fa505-cc8e-4c66-b8e1-c3fbf44316bf | public_subnet     | 472535c7-8df5-404f-a79a-5a410e402441 | 172.24.4.0/24   |
| 157c9abb-2cbc-4f65-ab12-b11a62a8e032 | Management_Subnet | 93d14ef9-b384-4e49-b42d-b4711b813e39 | 192.168.71.0/24 |
| 68ecb2ea-2f93-47c5-ab98-2dc963e5dd56 | private_subnet    | aea9b0ae-bf25-4406-bcff-472d098befb3 | 10.0.0.0/24     |
| 7471e981-9305-4490-8377-bf5dc43ed03b | Signalling_Subnet | 9eb3ebfa-0738-4875-ac91-30301dda53f7 | 192.168.70.0/24 |
+--------------------------------------+-------------------+--------------------------------------+-----------------+

I booted a vendor-supplied QCOW2 (Ubuntu 14.04.2 LTS), referencing those two networks:

nova boot --image b9368940-be4e-4e23-8246-8e4ef2d6a2d5 --meta ims_home_domain="yourmom.net" --security-groups default --flavor 3 --config-drive true --nic net-id=**93d14ef9-b384-4e49-b42d-b4711b813e39**,v4-fixed-ip="192.168.71.4" --nic net-id=**9eb3ebfa-0738-4875-ac91-30301dda53f7**,v4-fixed-ip="192.168.70.4" --key-name admin tiger123

Booted fine. From Horizon:

[root@kvm7 ~(keystone_admin)]# openstack server list
+--------------------------------------+-----------+--------+----------------------------------------------------+--------------+
| ID                                   | Name      | Status | Networks                                           | Image Name   |
+--------------------------------------+-----------+--------+----------------------------------------------------+--------------+
| e342b269-1624-49ab-a420-975ceadfefc5 | tiger123  | ACTIVE | Signalling=192.168.70.4; Management=192.168.71.4   | tiger123     |
+--------------------------------------+-----------+--------+----------------------------------------------------+--------------+

Great. I log in at the console, and check the network situation:

tiger@tiger123:~$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fa:16:3e:21:db:9d brd ff:ff:ff:ff:ff:ff
    inet **192.168.71.4/24** brd 192.168.71.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe21:db9d/64 scope link 
       valid_lft forever preferred_lft forever

Sigh.. OK, I create a /etc/network/interfaces.d/eth1.conf with the 192.168.70.4 IP address in it:

auto eth1
  iface eth1 inet static
  address 192.168.70.4/24
  gateway 192.168.70.1

The second NIC is not being propagated to the newly booted instance:

clearwater@ralf:/etc/network/interfaces.d$ sudo ifup eth1
Cannot find device "eth1"
Failed to bring up eth1.

I'm out of guesses here. Earlier, on another VM I was able to bring up eth1 but it was unusable, i.e. when I tried to SSH to it from the host, I got "destination host unreachable". So to recap, two 'nic' are added to the 'nova boot' line, one ethernet interface survives the instantiation process. Manually trying to bring up the second, fails.

Any help is appreciated.

edit retag flag offensive close merge delete

Comments

What you did would work on qcow2 image before uploading to glance and launching VM

dbaxps gravatar imagedbaxps ( 2017-04-27 02:04:17 -0600 )edit

You can't expect to access fixed IP addresses from the controllers or compute hosts, which explains your problem reaching an earlier VM. For the current VM, which doesn't see eth1, assuming libvirt hypervisor I would virsh dumpxml the VM and check if the network is there. Or check the console log.

Bernd Bausch gravatar imageBernd Bausch ( 2017-04-27 03:40:26 -0600 )edit

Yes, the VM has two 'bridge' in the XML. This forum won't let me paste the XML for some reason. Each has model type 'virtio' and each has a unique alias name 'net0' and 'net1'

tiger gravatar imagetiger ( 2017-04-27 09:31:21 -0600 )edit

You can paste longer files in paste.openstack.org. Perhaps something is wrong with the bridges. You should have a /var/log/libvirt directory with a libvirt log and qemu.log, I think. Anything in there? Also, does this happen to this particular instance only or several or all instances?

Bernd Bausch gravatar imageBernd Bausch ( 2017-04-27 09:59:46 -0600 )edit

2 answers

Sort by ยป oldest newest most voted
1

answered 2017-04-27 02:13:49 -0600

dbaxps gravatar image

updated 2017-04-28 13:34:19 -0600

UPDATE 28/04/17
Done as advised for F24 and CentOS 7.3 overcloud VM RDO Ocata HA TripleO QS deployment ( use case )
http://dbaxps.blogspot.com/2017/04/ad...
As appears echo 'GATEWAYDEV=eth0' >> /etc/sysconfig/network is a critical part for painless
adding second VNIC. The most recent device eth(X) created via attachment of port on the last created private network automatically becomes default gateway device.

END UPDATE

I would expect to work on Kilo Release :-
http://blog.scottlowe.org/2015/10/21/...
https://thornelabs.net/2014/09/03/con...
Just test per second link :- Ubuntu style updating interface files , but after command bellow succeed

$ neutron port-create net02 --fixed-ip ip_address=<IP-ADDRESS>
remember the id returned when the command completes.
Attach the OpenStack Instance VMX to the newly created Neutron Port 
VMX is the name of the OpenStack Instance down here
$ nova interface-attach --port-id $PORT_ID VMX
edit flag offensive delete link more

Comments

What terrifies me is having GUI and CLI activity and getting the system in an out-of-sync state. From the GUI, it looks like the VM has networks 192.168.70.4 and 192.168.71.4. What you describe above is a two-step process. Can it be said that what I have is one of the two steps, hence the confusion?

tiger gravatar imagetiger ( 2017-04-27 09:33:33 -0600 )edit

You are forced to use Neutron CLI to create port on second subnet && Nova CLI to attach this port to already launched cloud VM.
You wrote What terrifies me is having GUI and CLI activity and getting the system in an out-of-sync state
Sounds strange.

dbaxps gravatar imagedbaxps ( 2017-04-27 11:09:30 -0600 )edit

I have already dealt with the GUI saying I had 50GB out of 50GB memory used. That required going into MySQL and editing nova.quota_usage table. I had deleted two failed instances and apparently OpenStack does not update what I perceives as the memory in use. That's why I fear another "out of sync"

tiger gravatar imagetiger ( 2017-04-27 12:33:34 -0600 )edit

"What terrifies me is having GUI and CLI activity and getting the system in an out-of-sync state." There is no out-of-sync. There is only one DB, used by both CLI and GUI. To confirm this, display the network topology and change some networks using CLI. The NW topology is updated in realtime.

Bernd Bausch gravatar imageBernd Bausch ( 2017-04-27 23:03:05 -0600 )edit

Thank you Boris for the detailed example with screenshots! For anyone else having a similar issue I highly recommend his link here: http://dbaxps.blogspot.com/2017/04/adding-second-vnic-to-running-in.html (http://dbaxps.blogspot.com/2017/04/ad...)

tiger gravatar imagetiger ( 2017-04-28 19:46:08 -0600 )edit
0

answered 2017-04-27 12:36:58 -0600

tiger gravatar image

I am at my wit's end with this. The only thing that the 192.168.71.0/24 has that the other does not is it was created first. It is assigned eth0 in the guest. I have already tried to create eth1.conf and ifup eth1, nothing works...

nova interface-list 47a1dd2e-e234-40dc-9b3d-06edf3297349
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID                              | Net ID                               | IP addresses | MAC Addr          |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE     | 328c3425-f606-4a72-b7b3-d3a6c67bae62 | 9eb3ebfa-0738-4875-ac91-30301dda53f7 | 192.168.70.4 | fa:16:3e:47:68:b7 |
| ACTIVE     | 56bcdbda-71a8-4325-9c9e-bca78d2a44c4 | 93d14ef9-b384-4e49-b42d-b4711b813e39 | 192.168.71.4 | fa:16:3e:21:db:9d |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

neutron port-list
+--------------------------------------+------+----------------------------------+-------------------+----------------------------------------------+
| id                                   | name | tenant_id                        | mac_address       | fixed_ips                                    |
+--------------------------------------+------+----------------------------------+-------------------+----------------------------------------------+
| 328c3425-f606-4a72-b7b3-d3a6c67bae62 |      | ba33d16139934955b28183a0e7a71cf4 | fa:16:3e:47:68:b7 | {"subnet_id":                                |
|                                      |      |                                  |                   | "7471e981-9305-4490-8377-bf5dc43ed03b",      |
|                                      |      |                                  |                   | "ip_address": "192.168.70.4"}                |
| 56bcdbda-71a8-4325-9c9e-bca78d2a44c4 |      | ba33d16139934955b28183a0e7a71cf4 | fa:16:3e:21:db:9d | {"subnet_id": "157c9abb-2cbc-                |
|                                      |      |                                  |                   | 4f65-ab12-b11a62a8e032", "ip_address":       |
|                                      |      |                                  |                   | "192.168.71.4"}                              |
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2017-04-26 18:24:04 -0600

Seen: 1,657 times

Last updated: Apr 28 '17