Disk & Memory usage differs in MySQL and 'openstack host show'

asked 2018-07-12 03:15:03 -0500

rvaa gravatar image

Hi,

I tried to migrate a guest to another host but it failed with a message saying there's not enough capacity on the target host even though the server should me nearly empty. The guest I'm trying to move needs 4 cores, 50 GB of disk and 4 GB of memory.

osuser@controller:~$ openstack server migrate --block-migration --live compute1 e896652e-1a0d-46dd-b40f-c04285bd8958
No valid host was found. Unable to move instance e896652e-1a0d-46dd-b40f-c04285bd8958 to host compute1. There is not enough capacity on the host for the instance. (HTTP 400) (Request-ID: req-02128b04-c980-4245-a2c5-af20fe715648)

In /var/log/nova/nova-conductor.log I can see that the problem is the disk usage:

2018-07-10 12:42:53.218 61212 WARNING nova.scheduler.client.report [req-02128b04-c980-4245-a2c5-af20fe715648 d375a9c2aa544cd7a8d89cd1e8a93210 04a697566bc646e39d2f7ee1563e8ee0 - default default] Unable to submit allocation for instance e896652e-1a0d-46dd-b40f-c04285bd8958 (409 <html>
 <head>
  <title>409 Conflict</title>
 </head>
 <body>
  <h1>409 Conflict</h1>
  There was a conflict when trying to complete your request.<br /><br />
Unable to allocate inventory: Unable to create allocation for 'DISK_GB' on resource provider '931c2057-1d6d-4cd0-b85b-1efdc029e758'. The requested amount would exceed the capacity.
 </body>
</html>)

However, when I check the host status I can see that there's plenty of space (4 cpus, 4 G RAM, 50 G disk required):

osuser@controller:~$ openstack host show compute1
+----------+----------------------------------+-----+-----------+---------+
| Host     | Project                          | CPU | Memory MB | Disk GB |
+----------+----------------------------------+-----+-----------+---------+
| compute1 | (total)                          |  20 |    128888 |     261 |
| compute1 | (used_now)                       |   4 |      8704 |      50 |
| compute1 | (used_max)                       |   4 |      8192 |      50 |
| compute1 | 5ad784a0b8cd459c9959811435b66ccf |   4 |      8192 |      50 |
+----------+----------------------------------+-----+-----------+---------+

Then I found a link *) with instructions how to check the usage situation from the MySQL DB. I entered the following:

MariaDB [nova_api]> SELECT
  p.uuid,
  p.name,
  CASE
    when i.resource_class_id = 0 then 'VCPU'
    when i.resource_class_id = 1 then 'MEMORY_MB'
    when i.resource_class_id = 2 then 'DISK_GB'
  END as resource_name,
  i.total,
  i.reserved,
  a.used
FROM inventories i
JOIN resource_providers p ON i.resource_provider_id = p.id
LEFT JOIN allocations a ON i.resource_provider_id = a.resource_provider_id AND i.resource_class_id = a.resource_class_id;

And got the results:

+---------------------------+----------+---------------+--------+----------+-------+
| uuid                      | name     | resource_name | total  | reserved | used  |
+---------------------------+----------+---------------+--------+----------+-------+
| ae7cac06-...-a47b1b4c7cbb | compute2 | VCPU          |     20 |        0 |     4 |
| ae7cac06-...-a47b1b4c7cbb | compute2 | MEMORY_MB     | 128888 |      512 |  4096 |
| ae7cac06-...-a47b1b4c7cbb | compute2 | DISK_GB       |    261 |        0 |    50 |

| 931c2057-...-1efdc029e758 | compute1 | VCPU          |     20 |        0 |    20 |
| 931c2057-...-1efdc029e758 | compute1 | MEMORY_MB     | 128888 |      512 | 40960 |
| 931c2057-...-1efdc029e758 | compute1 | DISK_GB       |    261 |        0 |   250 |
+---------------------------+----------+---------------+--------+----------+-------+

According to this printout there's only 11 GB of disk and no free cores left. When I compared these to the openstack host show results I noticed that the allocation on the compute1 is 5 times what it really is. The compute2 has only the guest I'm trying to migrate to compute1.

Both guests were originally running on compute2 so they should fit on one host. I did make a few attempts to resize the guest that now runs on compute1 but for some reason they failed and by default the resize tries to restart the resized guest on a different host (compute1). In the end I was able to do the resize on the same host (compute2). I was wondering if the resize attempts messed up the compute1 resource ... (more)

edit retag flag offensive close merge delete