Ask Your Question
1

Can't create instances larger 1TB Memory

asked 2018-10-17 08:21:29 -0600

awalende gravatar image

Specs:

*CPU: Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz *OS: Ubuntu 18.04 AMD64 *QEMU: 1:2.11+dfsg-1ubuntu7.6 (Ubuntu Bionic Package) *Openstack: Openstack Queens (Ubuntu Bionic Package) *Libvirt-daemon: 4.0.0-1ubuntu8.5 *Seabios: 1.10.2-1ubuntu1

The Problem: We are not able to start instances, which have a memory size over 1 TB. After starting the instance, they shortly lock up. Starting guests with a lower amount of RAM works perfectly. We dealt with the same problem in the past with an older Qemu Version (2.5) by patching some source files according to this patch:

https://git.centos.org/blob/rpms!!qemu-kvm.git/34b32196890e2c41b0aee042e600ba422f29db17/SOURCES!kvm-fix-guest-physical-bits-to-match-host-to-go-beyond-1.patch (https://git.centos.org/blob/rpms!!qem...)

Since we are using the most recent qemu version from the ubuntu bionic repositorys, we should'nt need any patching anymore. However, guests still show a low phys-bits value:

address sizes   : 40 bits physical, 48 bits virtual

I asked the same question to the folks of qemu, they gave me a solution that requires to set some additional flags to the qemu call: https://bugs.launchpad.net/qemu/+bug/1798057 (Qemu Thread).

Sadly, I can't try this out since OpenStack does not give the option to pass custom options globally for each future guest. How am I able to allow Nova to start large memory instances in this case?

edit retag flag offensive close merge delete

Comments

I have the same problem. Did you ever find a fix or work-around? thx

luowei gravatar imageluowei ( 2018-12-05 13:28:46 -0600 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2018-12-06 13:02:29 -0600

luowei gravatar image

For Queens built on Ubuntu 18.04, here is a hack that worked for me. After applying, all VMs will have 46 hardware physical bits needed for >1TB RAM. I was able to create 2TB RAM VMs (tested with Windows 10, Ubuntu, CentOS,and cirros ;) VMs).

For big memory VMs, using Queens built on Unbuntu 18.04
-------------------------------------------------------
On the compute nodes:

1) check that there is a "hw physical bits" type
   # kvm -M ? | grep hpb
   pc-i440fx-bionic-hpb Ubuntu 18.04 PC (i440FX + PIIX, +host-phys-bits=true, 1996)
   pc-q35-bionic-hpb    Ubuntu 18.04 PC (Q35 + ICH9, +host-phys-bits=true, 2009)

2) # cd /usr/lib/python2.7/dist-packages/nova/virt/libvirt

3) # cp config.py config.py.sav
4) Add the hack below to config.py
   if self.os_mach_type is not None:
       type_node.set("machine", self.os_mach_type)
   ### begin hack
   else:
       type_node.set("machine", "pc-i440fx-bionic-hpb")
   ### end hack
   os.append(type_node)
5) # service nova-compute restart
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2018-10-17 08:21:29 -0600

Seen: 32 times

Last updated: Dec 06