Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

nova delete does not remove servers

I am having trouble bringing up servers, so I end up with a list that looks like this:

root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | None       | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+

Attempting to delete the servers doesn't work, it looks like this:

root@xxx:~# nova delete precise1
The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-86e8e658-a18e-47ee-9447-b953d9497882)
ERROR: Unable to delete any of the specified servers.
root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | deleting   | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+
root@xxx:~#

Then the server stays there forever. I fixed a problem with glance although my new servers still don't work, at least I can delete them. So how do I purge the data base of these two?

click to hide/show revision 2
No.2 Revision

What causes "The server has either erred or is incapable of performing the requested operation." when using nova delete does not to remove serversservers?

I am having trouble bringing up servers, so I end up with a list that looks like this:

root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | None       | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+

Attempting to delete the servers doesn't work, it looks like this:

root@xxx:~# nova delete precise1
The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-86e8e658-a18e-47ee-9447-b953d9497882)
ERROR: Unable to delete any of the specified servers.
root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | deleting   | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+
root@xxx:~#

Then the server stays there forever. I fixed a problem with glance although my new servers still don't work, at least I can delete them. So how do I purge the data base of these two?

What causes "The server has either erred or is incapable of performing the requested operation." when using nova delete to remove servers?

I am having trouble bringing up servers, so I end up with a list that looks like this:

root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | None       | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+

Attempting to delete the servers doesn't work, it looks like this:

root@xxx:~# nova delete precise1
The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-86e8e658-a18e-47ee-9447-b953d9497882)
ERROR: Unable to delete any of the specified servers.
root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | deleting   | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+
root@xxx:~#

Then the server stays there forever. I fixed a problem with glance although my new servers still don't work, at least I can delete them. So how do I purge the data base of these two?

UPDATE:

When the nova delete command attempts to execute the nova-compute.log does not get any new entries. However the nova-api.log gets an extensive traceback starting with:

2013-11-18 12:53:53.789 19195 WARNING nova.compute.api [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] [instance: edfde014-e489-413d-93e9-ab861a2dfbdd] instance's host xxx is down, deleting from database
2013-11-18 12:54:53.862 19195 ERROR nova.api.openstack [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] Caught error: Timeout while waiting on RPC response - topic: "network.xxx", RPC method: "deallocate_for_instance" info: "<unknown>"
2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack Traceback (most recent call last):

((omitted lots of calls))

2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack     exc.info, real_topic, msg.get('method'))
2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack Timeout: Timeout while waiting on RPC response - topic: "network.xxx", RPC method: "deallocate_for_instance" info: "<unknown>"
2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack 
2013-11-18 12:54:53.865 19195 INFO nova.api.openstack [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] http://xxx:8774/v2/3405c884d51f4332a046c57ddfffc828/servers/edfde014-e489-413d-93e9-ab861a2dfbdd returned with HTTP 500
2013-11-18 12:54:53.865 19195 INFO nova.osapi_compute.wsgi.server [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] 65.49.79.26 "DELETE /v2/3405c884d51f4332a046c57ddfffc828/servers/edfde014-e489-413d-93e9-ab861a2dfbdd HTTP/1.1" status: 500 len: 335 time: 60.2047951

So at the end of the day it seems like my compute node isn't responding -- is that right? I found a utility I think is deprecated that shows this:

nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth hv-v-iron                            internal         enabled    :-)   2013-11-18 21:01:07
nova-cert        hv-v-iron                            internal         enabled    :-)   2013-11-18 21:01:07
nova-scheduler   hv-v-iron                            internal         enabled    :-)   2013-11-18 21:01:07
nova-conductor   hv-v-iron                            internal         enabled    :-)   2013-11-18 21:01:08
nova-compute     hv-v-iron                            nova             enabled    XXX   2013-11-11 21:49:07
nova-network     hv-v-iron                            internal         enabled    XXX   2013-11-11 21:49:08

Would that XXX state have anything to do with it?

What causes "The server has either erred or is incapable of performing the requested operation." when using nova delete to remove servers?

I am having trouble bringing up servers, so I end up with a list that looks like this:

root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | None       | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+

Attempting to delete the servers doesn't work, it looks like this:

root@xxx:~# nova delete precise1
The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-86e8e658-a18e-47ee-9447-b953d9497882)
ERROR: Unable to delete any of the specified servers.
root@xxx:~# nova list
+--------------------------------------+----------+--------+------------+-------------+----------+
| ID                                   | Name     | Status | Task State | Power State | Networks |
+--------------------------------------+----------+--------+------------+-------------+----------+
| edfde014-e489-413d-93e9-ab861a2dfbdd | precise1 | ERROR  | deleting   | NOSTATE     |          |
| 9dad0e43-cb2c-4863-8765-ee4af5d6704d | precise2 | ERROR  | None       | NOSTATE     |          |
+--------------------------------------+----------+--------+------------+-------------+----------+
root@xxx:~#

Then the server stays there forever. I fixed a problem with glance although my new servers still don't work, at least I can delete them. So how do I purge the data base of these two?

UPDATE:

When the nova delete command attempts to execute the nova-compute.log does not get any new entries. However the nova-api.log gets an extensive traceback starting with:

2013-11-18 12:53:53.789 19195 WARNING nova.compute.api [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] [instance: edfde014-e489-413d-93e9-ab861a2dfbdd] instance's host xxx is down, deleting from database
2013-11-18 12:54:53.862 19195 ERROR nova.api.openstack [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] Caught error: Timeout while waiting on RPC response - topic: "network.xxx", RPC method: "deallocate_for_instance" info: "<unknown>"
2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack Traceback (most recent call last):

((omitted lots of calls))

2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack     exc.info, real_topic, msg.get('method'))
2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack Timeout: Timeout while waiting on RPC response - topic: "network.xxx", RPC method: "deallocate_for_instance" info: "<unknown>"
2013-11-18 12:54:53.862 19195 TRACE nova.api.openstack 
2013-11-18 12:54:53.865 19195 INFO nova.api.openstack [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] http://xxx:8774/v2/3405c884d51f4332a046c57ddfffc828/servers/edfde014-e489-413d-93e9-ab861a2dfbdd returned with HTTP 500
2013-11-18 12:54:53.865 19195 INFO nova.osapi_compute.wsgi.server [req-c75bf3b4-33b0-4523-8a18-6f00c36ffbb9 de2b62ce53764cd1979f0b39b87acc16 3405c884d51f4332a046c57ddfffc828] 65.49.79.26 "DELETE /v2/3405c884d51f4332a046c57ddfffc828/servers/edfde014-e489-413d-93e9-ab861a2dfbdd HTTP/1.1" status: 500 len: 335 time: 60.2047951

So at the end of the day it seems like my compute node isn't responding -- is that right? I found a utility I think is deprecated that shows this:

nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth hv-v-iron hv-v-xxx                            internal         enabled    :-)   2013-11-18 21:01:07
nova-cert        hv-v-iron hv-v-xxx                            internal         enabled    :-)   2013-11-18 21:01:07
nova-scheduler   hv-v-iron hv-v-xxx                            internal         enabled    :-)   2013-11-18 21:01:07
nova-conductor   hv-v-iron hv-v-xxx                            internal         enabled    :-)   2013-11-18 21:01:08
nova-compute     hv-v-iron hv-v-xxx                            nova             enabled    XXX   2013-11-11 21:49:07
nova-network     hv-v-iron hv-v-xxx                            internal         enabled    XXX   2013-11-11 21:49:08

Would that XXX state have anything to do with it?