Instances with >1TB RAM?

asked 2015-07-01 21:46:56 -0600

linhgb gravatar image

updated 2015-07-01 21:47:41 -0600

  • Openstack version: Juno
  • Hypervisor: KVM
  • Hosts are on Ubuntu 14.04 LTS ''trusty', 3TB of RAM, 64 VCPUs
  • Kernel: 3.13.0-52-generic #86-Ubuntu SMP

I'm testing flavors that have large memory and have succeeded with 512GB and 1024GB flavors. However, any flavor over 1024GB of RAM results in a kernel panic at boot (see below).

QEMU has had initial support for >1TB guest on x86 since v1.7 (we're on 2.0): http:// wiki.qemu.org/ChangeLog/1.7 - I believe that it is this patch: http:// comments.gmane.org/gmane.comp.emulators.qemu/222495

However, there is no BIOS support yet. There was a proposal but it was considered too flakey. Discussion thread here: https:// lists.gnu.org/archive/html/qemu-devel/2013-10/msg02418.html

I've read through the changelogs since then for SeaBIOS and bochs but none has anything about > 1TB guests.

However, I found that apparently RHEL has support for up to 4TB guests on KVM: https:// access.redhat.com/articles/rhel-kvm-limits but I don't have the setup to verify this claim.

Has anyone had success with guests on >1TB of memory? What is your setup?

 3.240000] Hardware name: OpenStack Foundation OpenStack Nova, BIOS Bochs 01/01/2011
[    3.240000] task: ffff896688d20000 ti: ffff896688c7c000 task.ti: ffff896688c7c000
[    3.240000] RIP: 0010:[<ffffffff814403d8>]  [<ffffffff814403d8>] acpi_ex_system_memory_space_handler+0x16e/0x1c2
[    3.240000] RSP: 0000:ffff896688c7f8e0  EFLAGS: 00010246
[    3.240000] RAX: ffffc900c4632000 RBX: ffffc900c4632000 RCX: ffff896688c7fa20
[    3.240000] RDX: 0000000001fd0fff RSI: 0000000000001000 RDI: ffffffff81ca61d0
[    3.240000] RBP: ffff896688c7f910 R08: 0000000000000000 R09: 0000000000001000
[    3.240000] R10: ffff8968bc003900 R11: ffffffff811a684f R12: 0000000000000020
[    3.240000] R13: 0000000000000000 R14: 0000000000000400 R15: ffff8966858849c0
[    3.240000] FS:  0000000000000000(0000) GS:ffff8968bc400000(0000) knlGS:0000000000000000
[    3.240000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    3.240000] CR2: ffffc900c4632000 CR3: 0000000001c12000 CR4: 00000000001406f0
[    3.240000] Stack:
[    3.240000]  ffff896688c7fa20 ffffffff8144026a ffff8968bc0314c8 00000000fed00000
[    3.240000]  0000000000000000 ffff8968bc00f1f8 ffff896688c7f978 ffffffff81439ef6
[    3.240000]  ffff8968bc028948 ffff8968bc00f3a8 ffff896688c7fa20 0000000088c7f998
[    3.240000] Call Trace:
[    3.240000]  [<ffffffff8144026a>] ? acpi_ex_do_logical_op+0x16f/0x16f
[    3.240000]  [<ffffffff81439ef6>] acpi_ev_address_space_dispatch+0x1ef/0x23c
[    3.240000]  [<ffffffff8143d577>] acpi_ex_access_region+0x1e7/0x277
[    3.240000]  [<ffffffff8143d908>] acpi_ex_field_datum_io+0xf9/0x174
[    3.240000]  [<ffffffff8143dafe>] acpi_ex_extract_from_field+0xa5/0x20a
[    3.240000]  [<ffffffff814396b0>] ? acpi_ev_acquire_global_lock+0xd6/0xdc
[    3.240000]  [<ffffffff8143d105>] acpi_ex_read_data_from_field+0x15c/0x18b
[    3.240000]  [<ffffffff81440691>] acpi_ex_resolve_node_to_value+0x190/0x21c
[    3.240000]  [<ffffffff8144091f>] acpi_ex_resolve_to_value+0x202/0x209
[    3.240000]  [<ffffffff814368c0>] acpi_ds_evaluate_name_path+0x7d/0xf6
[    3.240000]  [<ffffffff81436c68>] acpi_ds_exec_end_op+0x94/0x3ec
[    3.240000]  [<ffffffff814485e3>] acpi_ps_parse_loop+0x523/0x575
[    3.240000]  [<ffffffff8144908b>] acpi_ps_parse_aml+0x98/0x26f
[    3.240000]  [<ffffffff814498bd>] acpi_ps_execute_method+0x1be/0x262
[    3.240000]  [<ffffffff8144424f>] acpi_ns_evaluate+0x1c1/0x24d
[    3.240000]  [<ffffffff81446aec>] acpi_evaluate_object+0x13a/0x236
[    3.240000]  [<ffffffff810bff37>] ? down_timeout+0x37/0x60
[    3.240000]  [<ffffffff81426a1b>] acpi_evaluate_integer+0x34/0x52
[    3.240000]  [<ffffffff81426588>] ? acpi_os_signal_semaphore+0x21/0x2d
[    3.240000]  [<ffffffff8144f520>] ? acpi_ut_release_mutex+0x5f/0x63
[    3.240000]  [<ffffffff814294b2>] acpi_bus_get_status_handle+0x1e/0x39
[    3.240000]  [<ffffffff8142be75>] acpi_bus_check_add+0x6a/0x169
[    3.240000]  [<ffffffff814468d5>] acpi_ns_walk_namespace+0xdf/0x18f
[    3.240000]  [<ffffffff8142be0b>] ? acpi_add_single_object+0x4e7/0x4e7
[    3.240000]  [<ffffffff8142be0b>] ? acpi_add_single_object+0x4e7/0x4e7
[    3.240000]  [<ffffffff81446dc1>] acpi_walk_namespace+0x9b/0xcf
[    3.240000]  [<ffffffff81d872f1>] ? acpi_sleep_proc_init+0x2a/0x2a
[    3.240000]  [<ffffffff8142c199>] acpi_bus_scan+0x48/0x67
[    3.240000]  [<ffffffff81d87709>] acpi_scan_init+0x63/0x198
[    3.240000]  [<ffffffff81d9c353 ...
(more)
edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2015-07-02 08:01:20 -0600

noig3 gravatar image

I used RHEO to deploy 2tb instances after deploying hundreds of vanilla Essex - icehouse with 300gb ram.

edit flag offensive delete link more

Comments

Thanks mate. Could you tell me what QEMU version you were on, for the RHEO setup? Also, what BIOS (e.g versions of SeaBIOS and BOCHs)?

linhgb gravatar imagelinhgb ( 2015-07-02 20:28:02 -0600 )edit

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2015-07-01 21:46:56 -0600

Seen: 312 times

Last updated: Jul 02 '15