Ask Your Question
0

Openstack Stein (OSA)

asked 2020-07-03 07:08:40 -0500

steinuser53 gravatar image

updated 2020-07-03 08:34:49 -0500

Instances do not get an IP address and can't connect to htp://169.254.169.254

Sorry, some more info.

I have installed openstack stein on 3 dell servers (1 controller ad 2 computes) using ansible. I log into the horizon dashboard and create a network, subnet etc (also done from cli) which all goes fine and I can list them from the cli. When I spin up an instance (cirros) it spins up fine and is assigned an IP address (I can see this in horizon and the dhcp and dnsmasq logs) but when I log into the instance it has no IP address and in the logs on horizon it says the following:

checking http://169.254.169.254/2009-04-04/instance-id (http://169.254.169.254/2009-04-04/ins...) failed 1/20: up 0.73. request failed failed 2/20: up 12.75. request failed failed 3/20: up 24.76. request failed

I have checked (from reading other posts) enable_isolated_metadata in dhcp_agent.ini and this is set to true. Also in l3_agent.ini enable_metadata_proxy = True.

If I curl to http://169.254.169.254/ from either compute it fails, but if I replace 169.254.169.254 with the IP address of my controller it works fine. I have also confirmed that all the services are up and running

I have tried almost everything and this is also my first deploy of openstack so I might have overlooked something. Any advice would be appreciated

edit retag flag offensive close merge delete

Comments

What is the output of openstack network agent list? Did you search a little bit on your own? There's plenty of possible solutions, it would be useful to know what you have tried to resolve it yourself and where it failed.

eblock gravatar imageeblock ( 2020-07-03 07:43:39 -0500 )edit

Contradiction: You say "it has no IP address" but "... with the IP address of my controller it works fine". How can you access the controller without having an IP address?

Based on "no IP address", I'd tell you to start by checking the dnsmasq processes that implement DHCP and their logs.

Bernd Bausch gravatar imageBernd Bausch ( 2020-07-03 22:35:45 -0500 )edit

Also check the DHCP namespaces. They should have routing entries and interfaces that connect to tenant networks (sorry, I have no specifics right now).

Also trace packets with tcpdump, based on the diagrams on https://docs.openstack.org/neutron/la....

Bernd Bausch gravatar imageBernd Bausch ( 2020-07-03 22:39:00 -0500 )edit

Hi Bernd, when I say they have no IP address I'm talking about the instances that I spin up. The controller has it's management IP and if I replace 169.254.169.254 with the management IP of my controller then the curl command works fine from both compute nodes.

steinuser53 gravatar imagesteinuser53 ( 2020-07-05 11:33:36 -0500 )edit

Also the dnsmasq logs show that the IP address is given out (also seen on the horizon dashboard) but the instance has no IP address configured when I log into it. I'm not very experienced with it but from all the logs I've been looking at it looks like it's all working fine

steinuser53 gravatar imagesteinuser53 ( 2020-07-05 11:36:07 -0500 )edit

2 answers

Sort by ยป oldest newest most voted
0

answered 2020-07-03 08:49:40 -0500

steinuser53 gravatar image

updated 2020-07-15 06:12:19 -0500

Hi eblock, my apologies for the lack of information, I have now updated my question and thanks for the reply. Yes I've been searching for days now and all the posts where it has been resolved I have checked my config and it was already there.

+--------------------------------------+--------------------+------------------------+-------------------+-------+-------+------------------------
---+
| ID                                   | Agent Type         | Host                   | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+------------------------+-------------------+-------+-------+---------------------------+
| 091349c2-6bae-41d5-92fb-9eb832217788 | Metering agent     | cloud2020l4-controller | None              | :-)   | UP    | neutron-metering-agent    |
| 17aed236-8e91-4dd1-9212-aa751ef6790a | Metadata agent     | cloud2020l4-controller | None              | :-)   | UP    | neutron-metadata-agent    |
| 1d108d6c-42e5-439d-8dbc-ed0b21d61cd5 | L3 agent           | cloud2020l4-controller | nova              | :-)   | UP    | neutron-l3-agent          |
| 4c5c7fa0-7be9-4561-908d-d393068a4af9 | Linux bridge agent | cloud2020l4-compute1   | None              | :-)   | UP    | neutron-linuxbridge-agent |
| 54c79dbb-2807-4183-bfa7-1bf24121ce46 | Linux bridge agent | cloud2020l4-compute2   | None              | :-)   | UP    | neutron-linuxbridge-agent |
| 85aa94d3-b1a8-47bd-9bc2-d5a6352ebfb3 | DHCP agent         | cloud2020l4-controller | nova              | :-)   | UP    | neutron-dhcp-agent        |
| e269d937-30b5-4883-acda-938db426dccd | Linux bridge agent | cloud2020l4-controller | None              | :-)   | UP    | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+------------------------+-------------------+-------+-------+---------------------------+

DHCP

DHCP configuration for ports set([u'6a48a683-c1e3-4eba-848c-53efe1450eae']) is completed. fixed_ips=[{u'subnet_id': u'0a917da2-3446-47a0-b66d-72c572120daa', u'ip_address': u'MY_PRIVATE_IP'}],
tag:tag0,option:classless-static-route,169.254.169.254/32,MY_PRIVATE_NETWORK_FIRST_ALLOCATED_IP,0.0.0.0/0,MY_PRIVATE_NETWORK_GATEWAY tag:tag0,249,169.254.169.254/32,MY_PRIVATE_NETWORK_FIRST_ALLOCATED_IP,0.0.0.0/0,MY_PRIVATE_NETWORK_GATEWAY tag:tag0,option:router,MY_PRIVATE_NETWORK_GATEWAY

Update:

I have compared my install to a fellow colleagues install (he didn't use ansible, we are testing at the moment) in /var/lib/neutron/dhcp/ID/leases he has an entry for every instance he has spun up through horizon, my leases file is empty so maybe this might help in pointing to my issue, I will investigate a bit more in the morning but if anyone has any advice I'd appreciate it.

TCPDUMP (compute br-vlan)

09:25:04.458939 fa:16:3e:06:c4:13 > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 346: vlan 441, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 328)
0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:06:c4:13, length 300, xid 0xf93c270d, secs 121, Flags [none] (0x0000)
  Client-Ethernet-Address fa:16:3e:06:c4:13
  Vendor-rfc1048 Extensions
    Magic Cookie 0x63825363
    DHCP-Message Option 53, length 1: Discover
    Client-ID Option 61, length 7: ether fa:16:3e:06:c4:13
    MSZ Option 57, length 2: 576
    Parameter-Request Option 55, length 9: 
      Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
      Domain-Name, MTU, BR, NTP
      Classless-Static-Route
    Vendor-Class Option 60, length 12: "udhcp 1.29.3"
    Hostname Option 12, length 6: "cirros"

TCPDUMP (controller lxcbr0 & nova-api-container eth0)

09:24:21.715134 fe:05:91:48:23:de > 00:16:3e:0e:fb:39, ethertype IPv4 (0x0800), length 342: (tos 0xc0, ttl 64, id 32644, offset 0, flags [none], proto UDP (17), length 328)
10.0.3.1.67 > 10.0.3.230.68: [bad udp cksum 0x1c2c -> 0x9d6c!] BOOTP/DHCP, Reply, length 300, xid 0x5463e7c8, secs 11797, Flags [none] (0x0000)
  Client-IP 10.0.3.230
  Your-IP 10.0.3.230
  Server-IP 10.0.3.1
  Client-Ethernet-Address 00:16:3e:0e:fb:39
  Vendor-rfc1048 Extensions
    Magic Cookie 0x63825363
    DHCP-Message Option 53, length 1: ACK
    Server-ID Option 54, length 4: 10.0.3.1
    Lease-Time Option 51, length 4: 3600
    RN Option ...
(more)
edit flag offensive delete link more

Comments

You can check out our blog article, maybe the route part is helpful. Check if your image has a route for the metadata IP: netstat -rn.

eblock gravatar imageeblock ( 2020-07-04 03:10:41 -0500 )edit

Hi eblock, when I ran that (before signing up) it shows no routes at all. As I said, I have spent a long time trying to resolve this so I have tried a lot, as a last resort I thought I'd reach out here for some help. Thanks

steinuser53 gravatar imagesteinuser53 ( 2020-07-05 11:38:41 -0500 )edit

I agree with @Bernd, before debugging metadata you should figure out why the instances don't get an address. Seeing the IP in Horizon only means that openstack assigned one, not that the instance actually got one. Is DHCP enabled on the subnet the instance is launched in? Check neutron-dhcp logs.

eblock gravatar imageeblock ( 2020-07-06 02:34:31 -0500 )edit

Hi eblock, yes DHCP is enabled on the subnet. I have checked the dhcp logs and the dnsmasq logs and both look to me like they are handing out the address. I will attach the output below. What I haven't done yet is use tcpdump, I'm not really sure how to do that but I will look it up today.

steinuser53 gravatar imagesteinuser53 ( 2020-07-06 03:45:22 -0500 )edit

Sorry I can't add another answer: DHCP - DHCP configuration for ports set([u'6a48a683-c1e3-4eba-848c-53efe1450eae']) is completed. fixed_ips=[{u'subnet_id': u'0a917da2-3446-47a0-b66d-72c572120daa', u'ip_address': u'MY_PRIVATE_IP'}],

steinuser53 gravatar imagesteinuser53 ( 2020-07-06 03:47:59 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

Stats

Asked: 2020-07-03 07:08:40 -0500

Seen: 97 times

Last updated: Jul 15