Ask Your Question
0

NUMA awareness during instance placement [closed]

asked 2016-01-19 16:22:04 -0600

kaustubh gravatar image

updated 2016-01-21 18:30:43 -0600

Does OpenStack Kilo (KVM with libvirt) support launching instances on specific host NUMA nodes? I had a look at flavor extra_specs (http://docs.openstack.org/admin-guide-cloud/compute-flavors.html (http://docs.openstack.org/admin-guide...)), which only lists vCPU to physical CPU mapping. Is this list comprehensive?

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by kaustubh
close date 2016-07-08 12:54:25.061119

2 answers

Sort by ยป oldest newest most voted
2

answered 2016-01-21 18:30:11 -0600

kaustubh gravatar image

updated 2016-07-08 12:54:04 -0600

I was able to do this with flavor extra_specs. If anyone else wanted to do something similar:

To launch a 3vCPU VM on NUMA 0 and another on NUMA 1, with strict pinning, create two flavors with the following flavor-keys,

hw:numa_nodes=1             #Expose one NUMA node to VM
hw:numa_cpus.0=0,1,2        #Pin all vCPUs on NUMA 0
hw:cpu_policy=dedicated     #Disallow pinned CPUs to float within NUMA node
hw:mem_page_size=2048       #2M hugepages
hw:dedicated=dedicated
hw:numa_mem.0=4096          #4G RAM on NUMA 0
hw:numa_mempolicy=strict    #Allocate memory from same NUMA node

And similar for NUMA 1,

hw:numa_nodes=1             #Expose one NUMA node to VM
hw:numa_cpus.1=0,1,2        #Pin all vCPUs on NUMA 1
hw:cpu_policy=dedicated     #Disallow pinned CPUs to float within NUMA node
hw:mem_page_size=2048       #2M hugepages
hw:dedicated=dedicated
hw:numa_mem.1=4096          #4G RAM on NUMA 1
hw:numa_mempolicy=strict    #Allocate memory from same NUMA node

Ensure that the following are set in /etc/nova/nova.conf:

cpu_allocation_ratio = 1.0
ram_allocation_ratio = 1.0

And add CoreFilter, RamFilter, NUMATopologyFilter to scheduler filter list.
Verify on compute node after launching the VM,

root@dcmid:/home/kkelkar# virsh dumpxml instance-0000005b | grep vcpu
        <nova:vcpus>3</nova:vcpus>
  <vcpu placement='static'>3</vcpu>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='2'/>
    <vcpupin vcpu='2' cpuset='4'/>

References:

  1. numa-placement-spec
  2. cpu-pinning-spec

Update: Adding a note to clarify, NUMA 0 and 1 refers to the NUMA node within the guest and may or may not correspond to NUMA 0 and 1 respectively on the host depending on available resources.

edit flag offensive delete link more
1

answered 2016-01-19 22:57:51 -0600

littleblueallie gravatar image

Yes! You can set which cores you'd like to be reserved for guest vCPUs by setting the vcpu_pin_set=<core #s> in your /etc/nova.conf file on your compute host.

example:

vcpu_pin_set=16-23,24-31

I don't think I can post links yet (i'm new here!) because of my karma, but if you look at the nova.conf configuration reference for Kilo you can view the option details there! Make sure to restart the nova-compute service after setting the value :)

edit flag offensive delete link more

Comments

I was hoping to have more granularity. For example, launch VM1 on NUMA 0 pinned to physical CPUs 0,1; launch VM2 on NUMA 1 pinned to physical CPUs 2,3. etc.

kaustubh gravatar imagekaustubh ( 2016-01-21 13:05:45 -0600 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2016-01-19 16:22:04 -0600

Seen: 1,445 times

Last updated: Jul 08 '16