Ask Your Question
0

OpenStack Cloud Instance can't get metadata

asked 2019-05-21 19:55:58 -0500

Mizuka gravatar image

I'm actually setting up a private cloud for my company with OpenStack (Stein). I followed the tutorial from the official website and everything seems to work well... except getting metadata from cloud instance.

Let me explain how is set up my infrastructure :

All OpenStack are installed on KVM host (2xXeon 32 Core, 320Go RAM, 2To HDD, ...)

I set up VM like following :

openstack-controller001 192.168.50.11
openstack-compute001 192.168.50.41
openstack-storage001 192.168.50.61 (for Cinder)
db001 192.168.50.81 (DB is not hosted on the same server as the controller)
ldap001 192.168.50.251 (not using LDAP yet, only DNS and NTP server)

When i launch new instance of Ubuntu or Debian created from cloud images, I'm not able to connect to thoses VM via SSH, my keypair is always refused (error: permission denied). After some investigations, I realised that VM is not uploading SSH private key from the host. It seems the VM is contacting metadata server by using DHCP server IP address of my virtual network instead of metadata proxy server which is the controller if i'm not mistaken ?

[   15.840973] cloud-init[386]: 2019-05-20 05:53:58,124 - url_helper.py[WARNING]: Calling 'http://172.16.10.10/latest/meta-data/instance-id' failed [0/120s]: request error [HTTPConnectionPool(host='172.16.10.10', port=80): Max retries exceeded with url: /latest/meta-data/instance-id (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f098d1c0e10>: Failed to establish a new connection: [Errno 111] Connection refused',))]

172.16.10.10 represents the DHCP server of my Virtual Network (172.16.0.0/16, DHCP Range 172.16.10.10~172.16.20.254). I think there is something wrong with that, despite the config seems correct.

/etc/neutron/neutron.conf (openstack-controller001)

[DEFAULT]
# ...
nova_metadata_host = openstack-controller001
metadata_proxy_shared_secret = XXXXXXXXXXXXXXXXXX

/etc/nova/nova.conf (openstack-compute001)

[neutron]
# ...
service_metadata_proxy = true
metadata_proxy_shared_secret = XXXXXXXXXXXXXXXXXX

Metadata server is running on openstack-controller001 :

    [admin@openstack-controller001 ~]$ systemctl status neutron-metadata-agent
\u25cf neutron-metadata-agent.service - OpenStack Neutron Metadata Agent
   Loaded: loaded (/usr/lib/systemd/system/neutron-metadata-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since \u6708 2019-05-20 14:45:59 JST; 23h ago
 Main PID: 15329 (/usr/bin/python)
   CGroup: /system.slice/neutron-metadata-agent.service
           \u251c\u250015329 /usr/bin/python2 /usr/bin/neutron-metadata-agent --config-...
           \u251c\u250015357 /usr/bin/python2 /usr/bin/neutron-metadata-agent --config-...
           \u2514\u250015358 /usr/bin/python2 /usr/bin/neutron-metadata-agent --config-...

 5\u6708 20 14:45:59 openstack-controller001.adoc.local systemd[1]: Stopped Open...
 5\u6708 20 14:45:59 openstack-controller001.adoc.local systemd[1]: Started Open...
Hint: Some lines were ellipsized, use -l to show in full.

I don't know if my VM can reach 169.254.169.254 server since I can't ssh to it but from network namespace, it does.

[admin@openstack-controller001 ~]$ openstack network list
+--------------------------------------+---------------+--------------------------------------+
| ID                                   | Name          | Subnets                              |
+--------------------------------------+---------------+--------------------------------------+
| 838b6191-33d6-4683-958e-cee434518743 | provider      | d524b6e6-24ad-4a28-9482-b03f4bf22690 |
| c6ed4cfe-43b4-42b7-8fb5-f205a962f9f7 | adoc-net-main | 053681d8-4d5c-4ef4-8be7-f2276d43b26c |
+--------------------------------------+---------------+--------------------------------------+
[admin@openstack-controller001 ~]$ ip netns
qdhcp-838b6191-33d6-4683-958e-cee434518743 (id: 0)
qrouter-3a164855-6c8f-447c-8b0f-49e86d823488 (id: 2)
qdhcp-c6ed4cfe-43b4-42b7-8fb5-f205a962f9f7 (id: 1)

[root@openstack-controller001 admin]# ip netns exec qdhcp-c6ed4cfe-43b4-42b7-8fb5-f205a962f9f7 ping 169.254.169.254
PING 169.254.169 ...
(more)
edit retag flag offensive close merge delete

Comments

According to https://cloudinit.readthedocs.io/en/l..., the metadata address is configured in /etc/cloud/cloud.cfg on the instance. Where does 172 address come from?

Have you tried other images like Centos or Cirros? What is your Ubuntu version?

Bernd Bausch gravatar imageBernd Bausch ( 2019-05-21 22:05:50 -0500 )edit

Hello, it's Debian Cloud Image but I tested with Ubuntu too, errors are not the same but the result is -> no ssh key uploaded. 172.16.0.0/16 is the virtual network the VM are in. The provider network is joigned by a router to this private network. Theses cloud images are daily releases so recent one

Mizuka gravatar imageMizuka ( 2019-05-21 22:20:41 -0500 )edit

Just tried Ubuntu Bionic on my Stein cloud (deployed with Packstack). It uses 169.254.169.254 as expected. cloudinit configuration:

$ grep -r meta /etc/cloud
/etc/cloud/cloud.cfg:#      metadata_urls: [ 'blah.com' ]
/etc/cloud/cloud.cfg: - disable-ec2-metadata

So this is strange.

Bernd Bausch gravatar imageBernd Bausch ( 2019-05-22 01:04:53 -0500 )edit

Also:

$ grep meta /var/log/cloud*
...
cloud-init.log:2019-05-22 05:57:52,734 - DataSourceOpenStack.py[DEBUG]: Using metadata source: 'http://169.254.169.254'
Bernd Bausch gravatar imageBernd Bausch ( 2019-05-22 01:05:22 -0500 )edit

Check your cloud.cfg and the cloud-init logs.

Bernd Bausch gravatar imageBernd Bausch ( 2019-05-22 01:05:47 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2019-05-24 06:01:40 -0500

jsm gravatar image

The DHCP server can assist with providing metadata support on isolated networks. Setting this value to True will cause the DHCP server to append specific host routes to the DHCP request. The metadata service will only be activated when the subnet does not contain any router port. The guest instance must be configured to request host routes via DHCP (Option 121). This option doesn't have any effect when force_metadata is set to True. (boolean value)

enable_isolated_metadata = false

Try to set enable_isolated_metadata = true

edit flag offensive delete link more

Comments

Value is already set to true as the tutorial says, but the result is the same. I put logs below this post about routing tables. My network does have gateway and can ping external addresses.

Mizuka gravatar imageMizuka ( 2019-05-26 23:27:54 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2019-05-21 19:55:58 -0500

Seen: 60 times

Last updated: May 24