Metadata cpu policy dedicated

asked 2017-03-17 03:06:12 -0500

alp317 gravatar image

I have a compute node with 40 vcpus. I assigned 32 to nova scheduler and isolated those vcpus by using isolcpus. Now i am creating VM's using flavor metadata hw:cpu_policy=dedicated and hw:numa_nodes=1 but i can only SPAWN 6 instances of 4 vcpus. So i am using 24 vcpus. I can see in the horizon under hypervisors that i still have 8 vcpus. But i cannot SPAWN the 7th instance using the same flavor.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2017-03-17 05:36:30 -0500

volenbovsky gravatar image


So your 40 vCPUs are implemented as 2 physical CPUs with 10 physical CPUs with HT enabled on Then you assigned 8 vCPUs to something else than Nova (let's say ' to host'). -Are you sure that 32 vCPUs contain 16 vCPUs from one NUMA node and 16 from other NUMA node? -Then once you start launching VMs , do you see that they strictly end up on single NUMA node? -After launching 6th instance, do you see 4 vCPUs from each NUMA node free?

The commands to use are like: -lstopo /cat /proc/cpuinfo /virsh vcpupin/virsh dumpxml from Compute Host -you can take info from Nova database (see some stuff and use JSON tool to make it user-friendly to check that

edit flag offensive delete link more


Yes i verified that instances are pinned to single NUMA Nodes by looking into cpuset tag inside libvirt.xml. And 4 vcpus are free on both nodes. I can see that it created 3 instances on each NUMA Node.

alp317 gravatar imagealp317 ( 2017-03-17 07:39:43 -0500 )edit

Information about NUMA Node 1. "cpuset": [9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39], "pinned_cpus": [33, 35, 37, 39, 9, 11, 13, 15, 17, 19, 29, 31],

alp317 gravatar imagealp317 ( 2017-03-17 07:58:10 -0500 )edit

Can you paste the error and also possibly change Nova logging to debug and possibly see something in nova-scheduler log related to that?

volenbovsky gravatar imagevolenbovsky ( 2017-03-17 10:55:23 -0500 )edit

I finally figured it out. Nova's default thread policy is require. That means it will only use vCPUs that also have their sibling present in the nova vcpu list. As i was dedicating first 8 (0..7) to host their siblings were not used by nova, hence the missing 8 vCPUs. Solution was to use sibling set

alp317 gravatar imagealp317 ( 2017-08-07 00:56:17 -0500 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower


Asked: 2017-03-17 03:06:12 -0500

Seen: 385 times

Last updated: Mar 17 '17