Ask Your Question
0

Rackspace compatibile cloud - xen with libvirt...

asked 2016-05-26 07:22:26 -0500

Michal gravatar image

updated 2016-05-26 07:24:07 -0500

Hi,

My setup:

I am trying to build Rackspace compatible cloud so I can export/import images without any type of conversion , just export/download and import via glance.

I am using Mitaka release. Controller is Centos 7 and compute nodes are on Ubuntu 14.04 (xen via libvirt). I am using linuxbridge driver inside my neutron configuration, DHCP agent is not present. I am using Puppet for deployment so my config is align.

I don't want to use DHCP in the same way as Rackspace.

Instead I would like to use cloud-init or, like in Rackspace, nova-agent to inject the network config.

I know that all other data like users, SSH keys, user files goes via metadata service (EC2, config_drive).

I have found the following article about how the network configuration is injected and how Rackspace is relaying on xenstore: http://www.syntheticworks.com/rackspace-cloud/linux-rackspace-cloud/all-about-nova-agent-linux/ (http://www.syntheticworks.com/rackspa...)

My findings/questions bellow.



Short version:

  • does the xenstore-ls vm-data from the VM itself works on Xen with libvirt?
  • why I cannot do flat network injection when using Xen 4.6 on CentOS 7.2?
  • how the the Windows VM can be launched on Xen with libvirt?


Long version:

For my tests I have exported CentOS6 image from Rackspace and imported this image via glance client.


Xen-agent/nova-agent way:

a) Compute using Ubuntu 14.04 Xen via libvirt and network flat inject and config drive and Linux VM:

I am able to export/import Rackspace Linux image without modification, flat injection is working however the network interface order is not right when 2+ networks. It works but the order is random. I have noticed that Rackspace is having 0000-0000.... ID for the public network so doing such modification might solve the order problem. Can someone shed more light how NIC order is build/chosen? Based on ID?


b) Compute using Ubuntu 14.04 Xen via libvirt and network flat inject and config drive and Windows VM (I did a tricky export on VM from Rackspace):

Windows VM cannot start, vm_mode flag setup to HVM inside the image property. I am getting libvirt error, part of /var/log/nova/nova-compute.log is here:

http://paste.openstack.org/show/505611/

and errors from /var/log/libvirt/libvirtd.log file:

2016-05-26 11:01:49.561+0000: 7632: error : virNetDevSendEthtoolIoctl:3191 : ethtool ioctl error: No such device
2016-05-26 11:01:49.581+0000: 7632: error : virNetDevSendEthtoolIoctl:3191 : ethtool ioctl error: No such device
2016-05-26 11:01:49.597+0000: 7632: error : virNetDevSendEthtoolIoctl:3191 : ethtool ioctl error: No such device
2016-05-26 11:01:50.004+0000: 7632: debug : virEventPollRunOnce:644 : Poll got error event 4
2016-05-26 11:01:52.450+0000: 7636: error : libxlDomainStart:1073 : internal error: libxenlight failed to create new domain 'instance-00000045'

It can start from the same image when running it with

xl create windows_vm

content of this file bellow:

 firmware_override = "hvmloader"
 builder='hvm'
 memory = 4096
 name = "windows"
 vcpus=2
 vif = [ 'type=ioemu, bridge=brq7e22475a-fe' ]
 disk = [ 'file:/var ...
(more)
edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
0

answered 2016-06-10 09:48:11 -0500

Bob Ball gravatar image

updated 2016-06-10 09:48:47 -0500

does the xenstore-ls vm-data from the VM itself works on Xen with libvirt?

No; vm-data is set by XenAPI when it boots the VM. If you're using libvirt then you're not using XenAPI. Rackspace's public cloud is based on XenServer, using the XenAPI nova driver.

why I cannot do flat network injection when using Xen 4.6 on CentOS 7.2?

I'm not certain, but this may be related to the different usage of the images. Rackspace use an agent (as you know) to configure the networking, but the information needed by the agent is only injected through the XenAPI code paths. NIC ordering is indeed an interesting subject, and one that's actively being worked on in OpenStack - see https://git.openstack.org/cgit/openst... for the plan for Newton to make this much simpler.

how the the Windows VM can be launched on Xen with libvirt?

While the images _should_ work on Xen, you're likely to need a different set of metadata. It's very possible that the flags needed to correctly boot an HVM Windows image may not be passed through the libvirt+xen driver.

I'm not an expert on what metadata would be needed to boot the Windows VMs under libvirt+xen though...

edit flag offensive delete link more
0

answered 2016-06-13 16:04:43 -0500

Michal gravatar image

updated 2016-06-13 16:15:26 -0500

Hi Bob,

Thank you for you answer. I really appreciate it.

I did the XenServer 7 installation based on https://www.citrix.com/blogs/2015/11/30/integrating-xenserver-rdo-and-neutron/ (https://www.citrix.com/blogs/2015/11/...) and http://openstack-xenserver.readthedocs.io/en/latest/ (http://openstack-xenserver.readthedoc...)

As in my original plan - no DHCP service deployed.

I can start the instance but I am having some issues with network and password injection (image has been taken from Rackspace,xe-tools + nova agent installed - didn't touch it, version 6.1, tgz format of vhd, metadata updated).

If I login via XenCenter to the VM with old password and setup the IP manually connectivity works.

Seems that nova-agent is not fetching the configuration from xenstore. I can do "xenstore-ls vm-data" on the VM and I can see network details there like in Rackspace instance. nova-agent doesn't give any output in the logs.

Question: what should be the correct settings inside of nova.conf on controller and compute to get nova-agent working correctly.

My nova.conf from compute-vm:

[DEFAULT]
use_ipv6=False
notify_on_state_change=vm_and_task_state
notify_api_faults=False
state_path=/var/lib/nova
report_interval=10
enabled_apis=osapi_compute,metadata
osapi_compute_listen=0.0.0.0
osapi_compute_listen_port=8774
osapi_compute_workers=2
metadata_listen=0.0.0.0
metadata_listen_port=8775
metadata_workers=2
compute_manager=nova.compute.manager.ComputeManager
service_down_time=60
instance_usage_audit_period=hour
rootwrap_config=/etc/nova/rootwrap.conf
auth_strategy=keystone
use_forwarded_for=False
fping_path=/usr/sbin/fping
enable_instance_password=True
instance_usage_audit=True
heal_instance_info_cache_interval=60
reserved_host_memory_mb=512
allow_resize_to_same_host=False
compute_driver=xenapi.XenAPIDriver
vif_plugging_is_fatal=True
vif_plugging_timeout=300
firewall_driver=nova.virt.firewall.NoopFirewallDriver
force_raw_images=True
use_neutron=True
default_floating_pool=public
flat_network_dns=8.8.8.8
flat_injected=True
dhcp_domain=
config_drive_format=iso9660
force_config_drive=True
mkisofs_cmd=genisoimage
debug=True
log_dir=/var/log/nova
rpc_backend=rabbit
image_service=nova.image.glance.GlanceImageService
config_drive_cdrom=True
osapi_volume_listen=0.0.0.0
inject_password=True
[api_database]
[barbican]
[cache]
[cells]
[cinder]
catalog_info=volumev2:cinderv2:publicURL
[conductor]
[cors]
[cors.subdomain]
[database]
[ephemeral_storage_encryption]
[glance]
api_servers=http://172.19.0.10:9292
[guestfs]
[hyperv]
[image_file_url]
[ironic]
[keymgr]
[keystone_authtoken]
auth_uri=http://172.19.0.10:5000
admin_password=3FVxysdJeqBSkU
admin_user=nova
admin_tenant_name=services
identity_uri=http://172.19.0.10:35357
[libvirt]
[matchmaker_redis]
[metrics]
[neutron]
service_metadata_proxy=True
metadata_proxy_shared_secret =bHPm2mp5QpYVCv
url=http://172.19.0.10:9696
region_name=WRO
ovs_bridge=xapi1
extension_sync_interval=600
auth_url=http://172.19.0.10:35357/v3
auth_type=v3password
password=VABCQfMGGx2tpE
project_domain_name=Default
project_name=services
timeout=30
user_domain_name=Default
username=neutron
[osapi_v21]
[oslo_concurrency]
lock_path=/var/lib/nova/tmp
[oslo_messaging_amqp]
[oslo_messaging_notifications]
driver =messagingv2
[oslo_messaging_rabbit]
rabbit_host=172.19.0.10
rabbit_port=5672
rabbit_use_ssl=False
rabbit_userid=nova
rabbit_password=fBdAhR7bQHcZw9
[oslo_middleware]
[oslo_policy]
policy_file=/etc/nova/policy.json
[rdp]
[serial_console]
[spice]
agent_enabled=true
[ssl]
[trusted_computing]
[upgrade_levels]
[vmware]
[vnc]
enabled=True
keymap=en-us
vncserver_proxyclient_address=
novncproxy_base_url=http://172.19.0.10:6080/vnc_auto.html
[workarounds]
[xenserver]
ovs_integration_bridge=xapi1
agent_resetnetwork_timeout=60
use_agent_default=True
connection_url=http://169.254.0.1
connection_username=root
connection_password=admin
vif_driver=nova.virt.xenapi.vif.XenAPIOpenVswitchDriver
ovs_int_bridge=xapi1
[osapi_v3]
enabled=True
[wsgi]
api_paste_config=api-paste.ini
edit flag offensive delete link more

Comments

Sorry for not seeing this (I didn't get a notification); Can I suggest you send this question to the openstack users mailing list as I know there are some Rackspace developers on there who should be able to help.

Bob Ball gravatar imageBob Ball ( 2016-07-05 05:14:25 -0500 )edit

Hi Bob,

I did manage to do it :) The agent path has to be setup:

agent_path = /usr/sbin/xe-update-guest-attrs

I have now Linux and Windows images working with network and password injection, like at Rackspace! :)

Thank you for all your help here and on xs mailing group.

Kind regards, Michal.

Michal gravatar imageMichal ( 2016-07-05 05:35:33 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2016-05-26 07:22:26 -0500

Seen: 452 times

Last updated: Jun 13 '16