VM instance cannot be pinged or SSH

asked 2020-06-17 10:07:48 -0500

star5 gravatar image

updated 2020-06-18 03:57:00 -0500

I have used a provider network based on the tutorial setup for Rocky Installation. I have 2 VMs fully working where I can ping them and also SSH into them using the keypair.

When I try creating my third VM, the instance gets an IP but I am unable to ping. When I keep on deleting an creating new instances, sometimes I am able to ping the new instance but still unable to ssh. I get connection refused error.

When I looked into the IPtables of the compute node I see that the new entries are not added when I create an instance. If it gets added I am able to ping the VM but still cannot SSH.

The current setup is using DHCP I assume: https://imgur.com/0zTswHZ The ports are also created: https://imgur.com/vr8ry3A instance creation : openstack server create --flavor mi.large --image ubuntuServer --nic net-id=111fd1bd-b3bb-4085-8022-d1354c7d016f --security-group default --key-name n1 n1

Could anyone please provide an idea as to what is happening? Update: I am still able to create the instance without any error. But when I checked the logs I found it had 2 errors:

  1. amqp server not reachable controller:5672 error. So I created a new openstack rabbitmq user, updated all the config files to use the new user and then this error disappeared.
  2. The second error in the compute log was:

2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager [req-25b3f5d8-2831-4070-a08f-e101b6dc5a2e - - - - -] Error updating resources for node compute2.: DiskNotFound: No disk at /var/lib/nova/instances/4c9c7d22-b0c0-4342-9b02-d9956923065f/disk 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager Traceback (most recent call last): 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 7736, in _update_available_resource_for_node 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager rt.update_available_resource(context, nodename) 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py", line 708, in update_available_resource 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager resources = self.driver.get_available_resource(nodename) 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 6540, in get_available_resource 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager disk_over_committed = self._get_disk_over_committed_size_total() 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 8162, in _get_disk_over_committed_size_total 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager err_ctxt.reraise = False 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager self.force_reraise() 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-06-18 12:43:22.762 1364 ERROR nova.compute.manager six.reraise(self.type_, self.value, self ... (more)

edit retag flag offensive close merge delete

Comments

Provider networks usually don’t serve dhcp requests, depending on how your instance is configured to obtain its IP it could be necessary to use config-drive for the boot command instead of dhcp. Can you provide more details about dhcp disabled on the provider network and the instance configuration?

eblock gravatar imageeblock ( 2020-06-17 10:18:49 -0500 )edit

The current setup is using DHCP I assume: https://imgur.com/0zTswHZ The ports are also created: https://imgur.com/vr8ry3A instance creation : openstack server create --flavor mi.large --image ubuntuServer --nic net-id=111fd1bd-b3bb-4085-8022-d1354c7d016f --security-group default --key-name n1 n1

star5 gravatar imagestar5 ( 2020-06-17 12:30:00 -0500 )edit

Do you have an external dhcp server for the provider network? It could be a different issue but I would try it this way: in our company we can only use config-drive for provider networks. So DHCP is disabled for that subnet and config-drive applies the IP address to the VM that openstack assigned.

eblock gravatar imageeblock ( 2020-06-18 01:48:49 -0500 )edit

No I don't have any external dhcp server for provider network. My setup has 1 controller and 2 compute nodes. RabbitMQ is installed on controller.

star5 gravatar imagestar5 ( 2020-06-18 03:50:40 -0500 )edit

1 answer

Sort by » oldest newest most voted
0

answered 2020-06-18 04:10:38 -0500

star5 gravatar image

updated 2020-06-18 04:13:50 -0500

UPDATE 2: I saw some issues within the console.log of both instances. In the new VM the allocated IP and the console.log IP doesn't match.

Old VM working (ping and SSH) : IP : 192.168.20.107 Console.log:

> [[0;32m  OK  [0m] Started Wait for Network to be Configured.
         Starting Initial cloud-init job (metadata service crawler)...
[   23.465958] cloud-init[974]: Cloud-init v. 19.3-41-gc4735dd3-0ubuntu1~18.04.1 running 'init' at Thu, 18 Jun 2020 08:37:25 +0000. Up 23.26 seconds.
[   23.468715] cloud-init[974]: ci-info: +++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
[   23.471014] cloud-init[974]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[   23.473189] cloud-init[974]: ci-info: | Device |  Up  |           Address            |      Mask     | Scope  |     Hw-Address    |
[   23.475444] cloud-init[974]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[   23.477741] cloud-init[974]: ci-info: |  ens3  | True |        **192.168.20.107**        | 255.255.255.0 | global | fa:16:3e:21:1c:ce |
[   23.479997] cloud-init[974]: ci-info: |  ens3  | True | fe80::f816:3eff:fe21:1cce/64 |       .       |  link  | fa:16:3e:21:1c:ce |
[   23.482230] cloud-init[974]: ci-info: |   lo   | True |          127.0.0.1           |   255.0.0.0   |  host  |         .         |
[   23.484520] cloud-init[974]: ci-info: |   lo   | True |           ::1/128            |       .       |  host  |         .         |
[   23.486847] cloud-init[974]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[   23.489042] cloud-init[974]: ci-info: +++++++++++++++++++++++++++++++++Route IPv4 info++++++++++++++++++++++++++++++++++
[   23.491028] cloud-init[974]: ci-info: +-------+-----------------+----------------+-----------------+-----------+-------+
[   23.493062] cloud-init[974]: ci-info: | Route |   Destination   |    Gateway     |     Genmask     | Interface | Flags |
[   23.495053] cloud-init[974]: ci-info: +-------+-----------------+----------------+-----------------+-----------+-------+
[   23.497220] cloud-init[974]: ci-info: |   0   |     0.0.0.0     |  192.168.20.1  |     0.0.0.0     |    ens3   |   UG  |
[   23.499747] cloud-init[974]: ci-info: |   1   | 169.254.169.254 | 192.168.20.100 | 255.255.255.255 |    ens3   |  UGH  |
[   23.501819] cloud-init[974]: ci-info: |   2   |   192.168.20.0  |    0.0.0.0     |  255.255.255.0  |    ens3   |   U   |
[   23.503832] cloud-init[974]: ci-info: |   3   |   192.168.20.0  |  192.168.20.1  |  255.255.255.0  |    ens3   |   UG  |
[   23.505875] cloud-init[974]: ci-info: +-------+-----------------+----------------+-----------------+-----------+-------+
[   23.507947] cloud-init[974]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
[   23.509594] cloud-init[974]: ci-info: +-------+-------------+---------+-----------+-------+
[   23.511226] cloud-init[974]: ci-info: | Route | Destination | Gateway | Interface | Flags |
[   23.512822] cloud-init[974]: ci-info: +-------+-------------+---------+-----------+-------+
[   23.514446] cloud-init[974]: ci-info: |   1   |  fe80::/64  |    ::   |    ens3   |   U   |
[   23.516046] cloud-init[974]: ci-info: |   3   |    local    |    ::   |    ens3   |   U   |
[   23.517665] cloud-init[974]: ci-info: |   4   |   ff00::/8  |    ::   |    ens3   |   U   |
[   23.519311] cloud-init[974]: ci-info: +-------+-------------+---------+-----------+-------+
[[0;32m  OK  [0m] Started Initial cloud-init job (metadata service crawler).
[[0;32m  OK  [0m] Reached target System Initialization.
[[0;32m  OK  [0m] Started ACPI Events Check.
[[0;32m  OK  [0m] Started Daily Cleanup of Temporary Directories.

2)New VM: IP Address: 192.168.20.109

>        Starting Initial cloud-init job (metadata service crawler)...
[   35.460465] cloud-init[781]: Cloud-init v. 18.4-0ubuntu1~18.04.1 running 'init' at Thu, 18 Jun 2020 08:43:40 +0000. Up 22.30 seconds.
[   35.466694] cloud-init[781]: ci-info: +++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
[   35.469793] cloud-init[781]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[   35.471926] cloud-init[781]: ci-info: | Device |  Up  |           Address            |      Mask     | Scope  |     Hw-Address    |
[   35.474050] cloud-init[781]: ci-info: +--------+------+------------------------------+---------------+--------+-------------------+
[   35.476172] cloud-init[781]: ci-info: |  ens3  | True |        192.168.20.170        | 255.255.255.0 | global | fa:16:3e:90:6b:8c |
[   35.478309] cloud-init[781]: ci-info: |  ens3  | True ...
(more)
edit flag offensive delete link more

Comments

I used the console to login to ubuntu and then change the IP of ens3 in the VM to match that of the IP provided and I am able to ping. But had no internet connection so had to add the gateway manually. But I can't see any bridge interface created on the controller.

star5 gravatar imagestar5 ( 2020-06-18 08:08:35 -0500 )edit

That confirms my previous suspicion, the IPs don’t match because openstack has to assign one, but the instance can’t apply it. Use config-drive to configure the network (including gateway). Provider networks work directly on compute nodes that’s why nothing is created on control node.

eblock gravatar imageeblock ( 2020-06-18 10:32:30 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2020-06-17 10:07:48 -0500

Seen: 107 times

Last updated: Jun 18