Ask Your Question
0

Quota exceeded: code=%(code)s (HTTP 413)

asked 2012-03-17 00:49:22 -0500

swoo gravatar image

I'm running Xenserver and Openstack. I'm following the XenServer Development doc. Everything is installed by I get the Quota error when I try to run an instance.

nova@OSserver2b:/home/localadmin/openstack/nova$ nova image-list +--------------------------------------+---------+--------+ | ID | Name | Status | +--------------------------------------+---------+--------+ | 465442ad-da0a-401a-9efc-4a38a165361e | kernal | ACTIVE | | 6c5f6148-34db-424b-bd8f-82c4688c7d78 | ramdisk | ACTIVE | | 7b17d37e-8a76-4791-bf51-d887146ced52 | kernal | ACTIVE | +--------------------------------------+---------+--------+ nova@OSserver2b:/home/localadmin/openstack/nova$ nova flavor-list +----+-----------+-----------+------+----------+-------+------------+----------+ | ID | Name | Memory_MB | Swap | Local_GB | VCPUs | RXTX_Quota | RXTX_Cap | +----+-----------+-----------+------+----------+-------+------------+----------+ | 1 | m1.tiny | 512 | | 0 | 1 | | | | 2 | m1.small | 2048 | | 20 | 1 | | | | 3 | m1.medium | 4096 | | 40 | 2 | | | | 4 | m1.large | 8192 | | 80 | 4 | | | | 5 | m1.xlarge | 16384 | | 160 | 8 | | | +----+-----------+-----------+------+----------+-------+------------+----------+ nova@OSserver2b:/home/localadmin/openstack/nova$ nova boot test --flavor 1 --image 465442ad-da0a-401a-9efc-4a38a165361e Quota exceeded: code=%(code)s (HTTP 413) nova@OSserver2b:/home/localadmin/openstack/nova$ nova boot test --flavor 1 --image 6c5f6148-34db-424b-bd8f-82c4688c7d78 Quota exceeded: code=%(code)s (HTTP 413) nova@OSserver2b:/home/localadmin/openstack/nova$ nova boot test --flavor 1 --image 7b17d37e-8a76-4791-bf51-d887146ced52 Quota exceeded: code=%(code)s (HTTP 413) nova@OSserver2b:/home/localadmin/openstack/nova$ ./bin/nova-manage project quota openstack /usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py:8: UserWarning: Module nova was already imported from /home/localadmin/openstack/nova/nova/__init__.pyc, but /usr/lib/python2.7/dist-packages is being added to sys.path import pkg_resources

2012-03-16 17:11:47 DEBUG nova.utils [req-f8ec300f-ed76-4b48-b93f-7e0b1623dc8e None None] backend <module 'nova.db.sqlalchemy.api'="" from="" '="" home="" localadmin="" openstack="" nova="" nova="" db="" sqlalchemy="" api.pyc'=""> from (pid=843) __get_backend /home/localadmin/openstack/nova/nova/utils.py:607 metadata_items: 128 instances: 10 injected_file_content_bytes: 10240 injected_files: 5 volumes: 10 gigabytes: 1000 cores: 20 ram: 51200 floating_ips: 10 nova@OSserver2b:/home/localadmin/openstack/nova$

As far as I can tell, I should be able to create unto 10 instances, but it won't even let me create 1.

Am I missing another setting? Any help would be appreciated. Thanks.

edit retag flag offensive close merge delete

14 answers

Sort by ยป oldest newest most voted
0

answered 2012-03-19 16:50:40 -0500

johngarbutt gravatar image

I would recommend you try using DevStack ( http://wiki.openstack.org/XenServer/DevStack (http://wiki.openstack.org/XenServer/D...) ).

I have not seen that error myself, but it looks a lot like a configuration issue.

If you could check the api-server and the scheduler logs they may be able to give you some hints as to why it thinks you have reached your quota. Maybe the way you integrated keystone is not quite working?

It might be worth checking the clocks to ensure it does not simply think there are no resources avaiable (but I thought that would give you a different error).

I hope that is of some help.

edit flag offensive delete link more
0

answered 2012-03-19 19:36:37 -0500

swoo gravatar image

Yes, I am currently using a pull from DevStack.

The nova-api.log file shows: 2012-03-19 12:28:02 INFO nova.wsgi [-] Started metadata on 0.0.0.0:8775 2012-03-19 12:28:11 INFO nova.api.openstack.wsgi [req-e7e8696d-11ac-40a7-9013-11fdb90d025e localadmin openstack] POST http://10.228.24.234:8774/v1.1/openstack/servers (http://10.228.24.234:8774/v1.1/openst...) 2012-03-19 12:28:12 DEBUG nova.utils [req-e7e8696d-11ac-40a7-9013-11fdb90d025e localadmin openstack] backend from (pid=2534) __get_backend /home/localadmin/openstack/nova/nova/utils.py:607 2012-03-19 12:28:14 WARNING nova.compute.api [req-e7e8696d-11ac-40a7-9013-11fdb90d025e localadmin openstack] Quota exceeded for openstack, tried to run 1 instances 2012-03-19 12:28:14 INFO nova.api.openstack.wsgi [req-e7e8696d-11ac-40a7-9013-11fdb90d025e localadmin openstack] HTTP exception thrown: Quota exceeded: code=%(code)s 2012-03-19 12:28:14 INFO nova.api.openstack.wsgi [req-e7e8696d-11ac-40a7-9013-11fdb90d025e localadmin openstack] http://10.228.24.234:8774/v1.1/openstack/servers (http://10.228.24.234:8774/v1.1/openst...) returned with HTTP 413

I didn't notice any activity on the other logs.

edit flag offensive delete link more
0

answered 2012-03-19 19:52:08 -0500

swoo gravatar image

Also checked the clock and they are all in sync.

edit flag offensive delete link more
0

answered 2012-03-19 23:24:17 -0500

swoo gravatar image

Interesting. Just ran a "nova list" and found:

nova@OSserver2b:/home/localadmin/openstack/nova$ nova list +--------------------------------------+------+--------+----------+ | ID | Name | Status | Networks | +--------------------------------------+------+--------+----------+ | 20854cdd-b158-449c-a44a-c676ebdc0f8f | test | ERROR | | | 5a0d5c03-4869-48bd-a15b-60afb6e526c7 | test | ERROR | | | 62c3f5cb-62e0-46de-be46-4aaf3e5d676e | test | ERROR | | | 7b3045bd-2af5-4074-8352-42bc1ef4095b | test | ERROR | | | b1967369-d887-47a2-a53e-d077f5349563 | test | ERROR | | | b481cd67-ef31-4072-9723-48240f4b9ee1 | test | ERROR | | | b4894f33-243f-4f4e-97e4-1f44f9c773b4 | test | ERROR | | | c194ec35-b798-44f8-b6a4-10c43cc66236 | test | ERROR | | | c6049034-8f33-4f3f-8c49-dba4f238406c | test | ERROR | | | c813bc01-c590-4ede-bcf4-fdcbe5414b00 | test | ERROR | | +--------------------------------------+------+--------+----------+

So it seems that instances were being created and I did indeed have 10 of them already. Will delete and they to create again and see what happens.

edit flag offensive delete link more
0

answered 2012-03-19 23:35:13 -0500

swoo gravatar image

Ran "nova delete 20854cdd-b158-449c-a44a-c676ebdc0f8f"

Command didn't return an error and didn't delete the instance.

nova-api output: 2012-03-19 16:31:49 INFO nova.api.openstack.wsgi [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] GET http://10.228.24.234:8774/v1.1/openstack/servers/detail?fresh=1332199909.35 (http://10.228.24.234:8774/v1.1/openst...) 2012-03-19 16:31:49 DEBUG nova.api.openstack.wsgi [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Unrecognized Content-Type provided in request from (pid=3143) get_body /home/localadmin/openstack/nova/nova/api/openstack/wsgi.py:697 2012-03-19 16:31:49 DEBUG nova.compute.api [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Searching by: {'deleted': False, 'fresh': '1332199909.35', 'project_id': 'openstack'} from (pid=3143) get_all /home/localadmin/openstack/nova/nova/compute/api.py:1010 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=spawning. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.api.openstack.common [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3143) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:31:49 DEBUG nova.compute.api [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Searching by: {'uuid': ['7b3045bd-2af5-4074-8352-42bc1ef4095b', '62c3f5cb-62e0-46de-be46-4aaf3e5d676e', 'c6049034-8f33-4f3f-8c49-dba4f238406c', 'c194ec35-b798-44f8-b6a4-10c43cc66236', 'b4894f33-243f-4f4e-97e4-1f44f9c773b4', 'c813bc01-c590-4ede-bcf4-fdcbe5414b00', '20854cdd-b158-449c-a44a-c676ebdc0f8f', '5a0d5c03-4869-48bd-a15b-60afb6e526c7', 'b481cd67-ef31-4072-9723-48240f4b9ee1', 'b1967369-d887-47a2-a53e-d077f5349563']} from (pid=3143) get_all /home/localadmin/openstack/nova/nova/compute/api.py:1010 2012-03-19 16:31:49 DEBUG nova.compute.api [req-bae31505-1f34-4415-8994-7b07a8497ff9 localadmin openstack] Searching by: {'uuid': ['7b3045bd-2af5-4074-8352-42bc1ef4095b ... (more)

edit flag offensive delete link more
0

answered 2012-03-19 23:41:33 -0500

swoo gravatar image

Tried again to delete and now getting a similar, but different error:

2012-03-19 16:36:11 INFO nova.api.openstack.wsgi [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] GET http://10.228.24.234:8774/v1.1/openstack/servers/detail?fresh=1332200171.12 (http://10.228.24.234:8774/v1.1/openst...) 2012-03-19 16:36:11 DEBUG nova.api.openstack.wsgi [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Unrecognized Content-Type provided in request from (pid=3850) get_body /home/localadmin/openstack/nova/nova/api/openstack/wsgi.py:697 2012-03-19 16:36:11 DEBUG nova.compute.api [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Searching by: {'deleted': False, 'fresh': '1332200171.12', 'project_id': 'openstack'} from (pid=3850) get_all /home/localadmin/openstack/nova/nova/compute/api.py:1010 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=spawning. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.api.openstack.common [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Generated ERROR from vm_state=error task_state=deleting. from (pid=3850) status_from_state /home/localadmin/openstack/nova/nova/api/openstack/common.py:96 2012-03-19 16:36:11 DEBUG nova.compute.api [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Searching by: {'uuid': ['7b3045bd-2af5-4074-8352-42bc1ef4095b', '62c3f5cb-62e0-46de-be46-4aaf3e5d676e', 'c6049034-8f33-4f3f-8c49-dba4f238406c', 'c194ec35-b798-44f8-b6a4-10c43cc66236', 'b4894f33-243f-4f4e-97e4-1f44f9c773b4', 'c813bc01-c590-4ede-bcf4-fdcbe5414b00', '20854cdd-b158-449c-a44a-c676ebdc0f8f', '5a0d5c03-4869-48bd-a15b-60afb6e526c7', 'b481cd67-ef31-4072-9723-48240f4b9ee1', 'b1967369-d887-47a2-a53e-d077f5349563']} from (pid=3850) get_all /home/localadmin/openstack/nova/nova/compute/api.py:1010 2012-03-19 16:36:11 DEBUG nova.compute.api [req-30892558-52b1-4d42-89b3-e8675b5710ca localadmin openstack] Searching by: {'uuid': ['7b3045bd-2af5-4074-8352-42bc1ef4095b', '62c3f5cb-62e0-46de-be46-4aaf3e5d676e', 'c6049034-8f33-4f3f-8c49-dba4f238406c', 'c194ec35-b798-44f8-b6a4-10c43cc66236', 'b4894f33-243f-4f4e-97e4-1f44f9c773b4', 'c813bc01-c590-4ede-bcf4-fdcbe5414b00', '20854cdd-b158-449c-a44a-c676ebdc0f8f ... (more)

edit flag offensive delete link more
0

answered 2012-03-20 00:32:34 -0500

swoo gravatar image

The nova-compute output shows:

2012-03-19 17:23:09 DEBUG nova.rpc.common [-] received {u'_context_roles': [u'admin'], u'_context_request_id': u'req-d7d78fd8-093d-4ea9-8cc4-31714b917e37', u'_context_read_deleted': u'no', u'args': {u'instance_uuid': u'c813bc01-c590-4ede-bcf4-fdcbe5414b00'}, u'_context_auth_token': None, u'_context_strategy': u'noauth', u'_context_is_admin': True, u'_context_project_id': u'openstack', u'_context_timestamp': u'2012-03-20T00:23:09.408167', u'_context_user_id': u'localadmin', u'method': u'terminate_instance', u'_context_remote_address': u'127.0.0.1'} from (pid=3939) _safe_log /home/localadmin/openstack/nova/nova/rpc/common.py:144 2012-03-19 17:23:09 DEBUG nova.rpc.common [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] unpacked context: {'request_id': u'req-d7d78fd8-093d-4ea9-8cc4-31714b917e37', 'user_id': u'localadmin', 'roles': [u'admin'], 'timestamp': '2012-03-20T00:23:09.408167', 'is_admin': True, 'auth_token': None, 'project_id': u'openstack', 'remote_address': u'127.0.0.1', 'read_deleted': u'no', 'strategy': u'noauth'} from (pid=3939) unpack_context /home/localadmin/openstack/nova/nova/rpc/amqp.py:186 2012-03-19 17:23:09 INFO nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] check_instance_lock: decorating: |<function terminate_instance="" at="" 0x1c51398="">| 2012-03-19 17:23:09 INFO nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] check_instance_lock: arguments: |<nova.compute.manager.computemanager object="" at="" 0xef1910="">| |<nova.rpc.amqp.rpccontext object="" at="" 0x35f77d0="">| |c813bc01-c590-4ede-bcf4-fdcbe5414b00| 2012-03-19 17:23:09 DEBUG nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] instance c813bc01-c590-4ede-bcf4-fdcbe5414b00: getting locked state from (pid=3939) get_lock /home/localadmin/openstack/nova/nova/compute/manager.py:1577 2012-03-19 17:23:09 INFO nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] check_instance_lock: locked: |False| 2012-03-19 17:23:09 INFO nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] check_instance_lock: admin: |True| 2012-03-19 17:23:09 INFO nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] check_instance_lock: executing: |<function terminate_instance="" at="" 0x1c51398="">| 2012-03-19 17:23:09 AUDIT nova.compute.manager [req-d7d78fd8-093d-4ea9-8cc4-31714b917e37 localadmin openstack] [instance: c813bc01-c590-4ede-bcf4-fdcbe5414b00] Terminating instance 2012-03-19 17:23:09 DEBUG nova.rpc.common [-] Making asynchronous call on network ... from (pid=3939) multicall /home/localadmin/openstack/nova/nova/rpc/amqp.py:319 2012-03-19 17:23:09 DEBUG nova.rpc.common [-] MSG_ID is 738bf27394b34afc996dcdf766ac2f8c from (pid=3939) multicall /home/localadmin/openstack/nova/nova/rpc/amqp.py:322 2012-03-19 17:23:09 DEBUG nova.rpc.common [-] Pool creating new connection from (pid=3939) create /home/localadmin/openstack/nova/nova/rpc/amqp.py:57 2012-03-19 17:23:09 INFO nova.rpc.common [-] Connected to AMQP server on 10.228.24.60:5672

edit flag offensive delete link more
0

answered 2012-03-20 00:34:15 -0500

swoo gravatar image

The instance is not getting deleted. "nova list" output shows there are still 10 instances.

edit flag offensive delete link more
0

answered 2012-03-20 11:45:42 -0500

johngarbutt gravatar image

Ouch, that looks annoying. Afraid I have not see that set of errors before.

Since you are using DevStack, it might be easier to start again (although it would be nice to know what has cause all your pain): killall screen ./stack.sh

When you try to create an instance (easier using horizon, you can see what is going on better), you can then trace what is happening through the different logs, and you should be able to find the route cause of your issues.

Just want to double check, but I assume there is just a single XenSever with a single DomU running the nova services? And you are using EXT storage for your local storage SR?

edit flag offensive delete link more
0

answered 2012-03-20 22:12:04 -0500

swoo gravatar image

yes, I'm about at the point of starting again.

Yes, one XenServer with a single DomU running the nova services. Local storage.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2012-03-17 00:49:22 -0500

Seen: 291 times

Last updated: Mar 21 '12