After a hard reboot of nodes network is not starting

asked 2020-05-03 12:35:44 -0500

Erin Sims gravatar image

updated 2020-05-04 04:16:21 -0500

well I tried to fix the formatting so people could understand the problem. but I do not have 10 points to upload anything. Seriously you guy

After we restarted all the nodes after some networking issues on the hardware in the DC. The openvswitch seems to be messed up and using the wrong interface? We attempted to start the instances on the nodes but it gets a permission denied error. I notices the following errors on the ovs-vsctl show command. also dpdk does not work either. Please let me know what other logs i can get.

failed services: ovs-vswitchd.service loaded failed failed Open vSwitch Forwarding Unit ● sdkserver.service loaded failed failed zVM SDK API server ● systemd-networkd-wait-online.service loaded failed failed Wait for Network to be Configured

logs: 2020-05-03T16:21:04.152Z|00007|dpdk|INFO|Using DPDK 18.11.2 2020-05-03T16:21:04.152Z|00008|dpdk|INFO|DPDK Enabled - initializing... 2020-05-03T16:21:04.152Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2020-05-03T16:21:04.152Z|00010|dpdk|INFO|IOMMU support for vhost-user-client disabled. 2020-05-03T16:21:04.152Z|00011|dpdk|INFO|POSTCOPY support for vhost-user-client disabled. 2020-05-03T16:21:04.152Z|00012|dpdk|INFO|Per port memory for DPDK devices disabled. 2020-05-03T16:21:04.152Z|00013|dpdk|INFO|EAL ARGS: ovs-vswitchd -c 0x03 --socket-mem 1024,1024 --socket-limit 1024,1024. 2020-05-03T16:21:04.153Z|00014|dpdk|INFO|EAL: Detected 48 lcore(s) 2020-05-03T16:21:04.153Z|00015|dpdk|INFO|EAL: Detected 2 NUMA nodes 2020-05-03T16:21:04.160Z|00016|dpdk|INFO|EAL: Multi-process socket /var/run/dpdk/rte/mp_socket 2020-05-03T16:21:04.178Z|00017|dpdk|WARN|EAL: No free hugepages reported in hugepages-2048kB 2020-05-03T16:21:04.178Z|00018|dpdk|WARN|EAL: No free hugepages reported in hugepages-2048kB 2020-05-03T16:21:04.178Z|00019|dpdk|WARN|EAL: No free hugepages reported in hugepages-2048kB 2020-05-03T16:21:04.178Z|00020|dpdk|WARN|EAL: No free hugepages reported in hugepages-1048576kB 2020-05-03T16:21:04.178Z|00021|dpdk|ERR|EAL: Cannot get hugepage information. 2020-05-03T16:21:04.178Z|00022|dpdk|EMER|Unable to initialize DPDK: Permission denied 2020-05-03T16:21:04.203Z|00002|daemon_unix|ERR|fork child died before signaling startup (killed (Aborted), core dumped) 2020-05-03T16:21:04.203Z|00003|daemon_unix|EMER|could not detach from foreground session 2020-05-03T16:21:04.614Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovs-vswitchd.log 2020-05-03T16:21:04.617Z|00002|ovs_numa|INFO|Discovered 24 CPU cores on NUMA node 1 2020-05-03T16:21:04.617Z|00003|ovs_numa|INFO|Discovered 24 CPU cores on NUMA node 0 2020-05-03T16:21:04.617Z|00004|ovs_numa|INFO|Discovered 2 NUMA nodes and 48 CPU cores 2020-05-03T16:21:04.617Z|00005|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connecting... 2020-05-03T16:21:04.617Z|00006|reconnect|INFO|unix:/var/run/openvswitch/db.sock: connected 2020-05-03T16:21:04.619Z|00007|dpdk|INFO|Using DPDK 18.11.2 2020-05-03T16:21:04.619Z|00008|dpdk|INFO|DPDK Enabled - initializing... 2020-05-03T16:21:04.619Z|00009|dpdk|INFO|No vhost-sock-dir provided - defaulting to /var/run/openvswitch 2020-05-03T16:21:04.619Z ... (more)

edit retag flag offensive close merge delete

2 answers

Sort by » oldest newest most voted

answered 2020-05-04 10:29:41 -0500

Please share detail server logs and nova-conductor logs. Looks some file ownership issue for that instance.

edit flag offensive delete link more


sure Ill find some.

Erin Sims gravatar imageErin Sims ( 2020-05-04 10:59:34 -0500 )edit

root@web2:/var/log/nova# ls nova-api-metadata.log nova-api-metadata.log.3.gz nova-compute.log.1 nova-compute.log.4.gz nova-manage.log.2.gz privsep-helper.log.1 privsep-helper.log.4.gz nova-api-metadata.log.1 nova-api-metadata.log.4.gz nova-compute.log.2.gz nova-manage.log

Erin Sims gravatar imageErin Sims ( 2020-05-04 11:03:28 -0500 )edit

which ones?

Erin Sims gravatar imageErin Sims ( 2020-05-04 11:03:35 -0500 )edit

since everytime i try to past something it looks like crap i am going to post links sorry, it wont let me paste from a file

Erin Sims gravatar imageErin Sims ( 2020-05-04 11:38:16 -0500 )edit

is there some way I can just rebuild these networks?

Erin Sims gravatar imageErin Sims ( 2020-05-04 11:48:56 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2020-05-03 12:35:44 -0500

Seen: 118 times

Last updated: May 04