Unable to launch an instance

asked 2018-06-01 05:36:57 -0500

Abhi1988 gravatar image

Failing while creating an instance as below

2018-06-01 12:07:11.218 6800 ERROR nova.scheduler.utils [req-d7decb2e-b75f-4f73-8666-799badbb0e41 15f5b7bee619432880240dbbd5a42c89 98c2bf7cd43f4f108de86b6eee207ecd - - -] [instance: 334b80c0-ab52-4bfe-9947-63ca02b0897f] Error from last host: http://slabnode1210.netact.nsn-rdnet.net (node http://slabnode1210.netact.nsn-rdnet.net): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1787, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1985, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 334b80c0-ab52-4bfe-9947-63ca02b0897f was re-scheduled: The request you have made requires authentication. (HTTP 401) (Request-ID: req-a1969249-59f9-4173-b377-72e74cb5f813)\n']

Any idea on what kind of authentication it is asking for? There is already password-less authentication between all the compute nodes and the controller node back and fro.

edit retag flag offensive close merge delete


There should be more information in the compute log file on the compute host slabnode1210.netact.nsn-rdnet.net. In any case, this has nothing to do with ssh, but OpenStack authentication.

My guess: Incorrect config in nova.conf, section keystone_authtoken, on that compute host.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-01 05:57:54 -0500 )edit

For an example of how this should be configured, see https://docs.openstack.org/nova/queen....

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-01 05:58:14 -0500 )edit

Tried adding configuration details to nova.conf under keystone_authtoken section on that compute host. But still didn't help!

Abhi1988 gravatar imageAbhi1988 ( 2018-06-04 04:16:01 -0500 )edit

Are you sure your authentication data is correct? Check the compute log for more info.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-04 10:01:04 -0500 )edit

There is no problem in the authentication data. I double checked it.

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 02:29:50 -0500 )edit

The problem is, whenever i am creating an instance, the space is not being taken from the external storage attached to the blade. But is being taken from the controller node's local storage. Don't know how to map the storage to consider external storage.

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 02:30:09 -0500 )edit

How do you know that?

I am puzzled, by the way. I thought your problem was that the instance didn’t start. I don’t see how an instance that doesn’t start uses up storage space.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 03:26:09 -0500 )edit

To change the location of instance ephemeral storage, use the instances_path setting.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 03:30:48 -0500 )edit

To solve the authentication problem, I am sure you find relevant messages in the compute log.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 03:31:51 -0500 )edit

The error message is as below when I did nove show for that instance.

image | Attempt to boot from volume - no image supplied

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 04:49:02 -0500 )edit

But the image still exists [root@slabnode1202 ~]# glance image-list +--------------------------------------+-------------------+ | ID | Name | +--------------------------------------+-------------------+ | 58d65160-453c-459b-8449-0839991f7906 | rhel7_in

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 04:49:37 -0500 )edit

Asso my another finding is that, there is no iscsi lun created for any volume I create [root@slabnode1202 ~]# targetcli

/iscsi> ls o- iscsi .............................................................................................................. [Targets: 0] /iscsi>

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 04:51:30 -0500 )edit

And in lsblk output, not able to see any LVM mount points where it belongs to cinder group. Currently it belongs to controller node. And hence, considering storage space from controller node

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 04:54:48 -0500 )edit

Attempt to boot from volume - no image supplied is not an error message. It indicates that the instance boots from a volume instead of an image. Why it does that? Because the user wanted it.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 07:19:45 -0500 )edit

If you use LVM as Cinder backend, the LVM volume is on the host where cinder-volume runs. It may well be on the controller in your case. On the compute node, you should see the corresponding iscsi LUN. There are no mount points involved.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 07:22:34 -0500 )edit

Now, this has nothing to do with the original. I suggest you check the compute log and look for an answer there. If you need more help with Cinder, create another question.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 07:24:27 -0500 )edit

This is actually related to the question. I am able to successfully deploy an instance of smaller size where the space is considered from the controller node's "/" storage. But an instance of larger size(larger than / size of controller node), it fails as said in the question.

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 08:41:42 -0500 )edit

The instance boots from a volume. If the volume service is implemented on the controller, and you use the LVM backend, it's normal that storage is allocated on the controller. If you want storage to be on the compute node, don't boot from a volume.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 09:50:44 -0500 )edit

If you expect storage to be on some external disk array, and it is allocated in the controller's filesystem instead, your volume driver or LVM is misconfigured. In this case, check cinder.conf and the output of pvs, vgs and lvs (or share it here).

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 09:52:14 -0500 )edit

I still wonder how a storage issue can create an authentication problem. Please check the compute log; I am sure it contains more information.

Bernd Bausch gravatar imageBernd Bausch ( 2018-06-06 09:53:18 -0500 )edit

Log from /var/log/nova/nova-conductor.log 2018-06-07 07:48:16.735 7127 WARNING nova.scheduler.utils [req-70399438-d6 - - -] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available.

Abhi1988 gravatar imageAbhi1988 ( 2018-06-06 23:58:49 -0500 )edit

2018-06-07 07:16:37.417 6976 DEBUG nova.scheduler.filters.disk_filter [req-415ffa17-ee20d - - -] (http://slabnode1210.netact.nsn-rdnet.net, http://slabnode1210.netact.nsn-rdnet.net) ram: 130303MB disk: 48128MB io_ops: 0 instances: 0 does not have 81920 MB usable disk space before overcommit, it only has 50176 MB.

Abhi1988 gravatar imageAbhi1988 ( 2018-06-07 00:23:46 -0500 )edit