Can't create instances larger 1TB Memory

asked 2018-10-17 08:21:29 -0600

awalende gravatar image


*CPU: Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz *OS: Ubuntu 18.04 AMD64 *QEMU: 1:2.11+dfsg-1ubuntu7.6 (Ubuntu Bionic Package) *Openstack: Openstack Queens (Ubuntu Bionic Package) *Libvirt-daemon: 4.0.0-1ubuntu8.5 *Seabios: 1.10.2-1ubuntu1

The Problem: We are not able to start instances, which have a memory size over 1 TB. After starting the instance, they shortly lock up. Starting guests with a lower amount of RAM works perfectly. We dealt with the same problem in the past with an older Qemu Version (2.5) by patching some source files according to this patch:!!qemu-kvm.git/34b32196890e2c41b0aee042e600ba422f29db17/SOURCES!kvm-fix-guest-physical-bits-to-match-host-to-go-beyond-1.patch (!!qem...)

Since we are using the most recent qemu version from the ubuntu bionic repositorys, we should'nt need any patching anymore. However, guests still show a low phys-bits value:

address sizes   : 40 bits physical, 48 bits virtual

I asked the same question to the folks of qemu, they gave me a solution that requires to set some additional flags to the qemu call: (Qemu Thread).

Sadly, I can't try this out since OpenStack does not give the option to pass custom options globally for each future guest. How am I able to allow Nova to start large memory instances in this case?

edit retag flag offensive close merge delete


I have the same problem. Did you ever find a fix or work-around? thx

luowei gravatar imageluowei ( 2018-12-05 13:28:46 -0600 )edit

1 answer

Sort by ยป oldest newest most voted

answered 2018-12-06 13:02:29 -0600

luowei gravatar image

For Queens built on Ubuntu 18.04, here is a hack that worked for me. After applying, all VMs will have 46 hardware physical bits needed for >1TB RAM. I was able to create 2TB RAM VMs (tested with Windows 10, Ubuntu, CentOS,and cirros ;) VMs).

For big memory VMs, using Queens built on Unbuntu 18.04
On the compute nodes:

1) check that there is a "hw physical bits" type
   # kvm -M ? | grep hpb
   pc-i440fx-bionic-hpb Ubuntu 18.04 PC (i440FX + PIIX, +host-phys-bits=true, 1996)
   pc-q35-bionic-hpb    Ubuntu 18.04 PC (Q35 + ICH9, +host-phys-bits=true, 2009)

2) # cd /usr/lib/python2.7/dist-packages/nova/virt/libvirt

3) # cp
4) Add the hack below to
   if self.os_mach_type is not None:
       type_node.set("machine", self.os_mach_type)
   ### begin hack
       type_node.set("machine", "pc-i440fx-bionic-hpb")
   ### end hack
5) # service nova-compute restart
edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools


Asked: 2018-10-17 08:21:29 -0600

Seen: 183 times

Last updated: Dec 06 '18