Ask Your Question
0

octavia worker not able to connect to amphora

asked 2019-11-04 09:41:17 -0500

generalfuzz gravatar image

updated 2019-11-05 11:13:46 -0500

johnsom gravatar image

I deployed a simple open stack environment on a single bare metal machine using Kolla Ansible with octavia enabled in the globals.yml using the master (train) branch. When I deploy a loadbalancer, the amphora VM is successfully spun up. I can directly SSH into it from the controller node using a network namespace, I can see that HAProxy is running.

However, no matter how I can configure my network, the octavia-worker cannot access the Amphora VMs. The worker log will fill up with:

2019-10-31 03:55:48.307 26 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: ConnectTimeout: HTTPSConnectionPool(host='10.16.100.14', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f8788410b50>, 'Connection to 10.16.100.14 timed out. (connect timeout=10.0)'))
2019-10-31 03:56:03.317 26 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: ConnectTimeout: HTTPSConnectionPool(host='10.16.100.14', port=9443): Max retries exceeded with url: // (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f87884101d0>, 'Connection to 10.16.100.14 timed out. (connect timeout=10.0)'))

I also verified that I am unable to ping/ssh/curl to the amphora vm from inside the octavia-worker docker container.

I created an Ampohra image using disk builder and registered the image with openstack.

openstack image create --container-format bare --disk-format qcow2 --private --file /root/octavia/amphora-x64-haproxy.qcow2 --tag amphora amphora

I created a network for the load bouncer, and associated permissive network rules for the network.

openstack security group create --description 'used by Octavia amphora instance' octavia
 openstack security group rule create --protocol icmp octavia
 openstack security group rule create --protocol tcp --dst-port 5555 --egress octavia
 openstack security group rule create --protocol tcp --dst-port 9443 --ingress octavia

I created the load balancer flavor and flavor profile, along with a VM flavor for the amphora vm

openstack loadbalancer flavorprofile create --name amphora-single-profile --provider amphora --flavor-data '{"loadbalancer_topology": "SINGLE"}'
openstack loadbalancer flavor create --name standalone-lb --flavorprofile amphora-single-profile --description "A non-high availability load balancer for testing." --enable
octavia_amp_flavor_id="$(openstack flavor create --disk 20 --private --ram 4096 --vcpus 2 octavia -f value -c id)"

I updated the globals.yml with the relevant octavia parameters

openstack network list
+--------------------------------------+----------+--------------------------------------+
| ID                                   | Name     | Subnets                              |
+--------------------------------------+----------+--------------------------------------+
| 4a52dd5d-d8c7-4684-aeab-e75c88eaec88 | public1  | 063ec6b3-322a-463e-805d-df5bddf49a18 |
| 588f95d7-9310-4a27-8e83-f6ee5e6738b5 | demo-net | 54449fdf-da42-45b8-ad65-9c35783e1197 |
| 79f55a3b-3bba-4aec-a2a8-5eefaca033d1 | lb-net   | d17fa379-ad28-47cf-8479-8d5a7c46965e |
+--------------------------------------+----------+--------------------------------------+

openstack security group list
+--------------------------------------+-----------------------------------------+----------------------------------+----------------------------------+------+
| ID                                   | Name                                    | Description                      | Project                          | Tags |
+--------------------------------------+-----------------------------------------+----------------------------------+----------------------------------+------+
| 8172cf36-0de2-4afb-82e9-62f5420e3879 | octavia                                 | used by Octavia amphora instance | ad8352fb52eb476585a1f31a8ae5ae78 | []   |
| a488ff33-4849-455d-a0f1-566140b8b9de | default                                 | Default security group           | ad8352fb52eb476585a1f31a8ae5ae78 | []   |
| c997dda4-c6a5-48e0-8835-fe7b394c1894 | default                                 | Default security group           | 2f4d28c2c37b48e887bb325b6ce50b70 | []   |
| cf5f3956-a2c9-4c28-ad6f-93ad5f36fd6e | default                                 | Default security group           |                                  | []   |
+--------------------------------------+-----------------------------------------+----------------------------------+----------------------------------+------+
openstack flavor list
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
| ID                                   | Name      |   RAM | Disk | Ephemeral | VCPUs | Is Public |
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
| 1                                    | m1.tiny   |   512 |    1 |         0 |     1 | True      |
| 2                                    | m1.small  |  2048 |   20 |         0 |     1 | True      |
| 3                                    | m1.medium |  4096 |   40 |         0 |     2 | True      |
| 4                                    | m1.large  |  8192 |   80 |         0 |     4 | True      |
| 5                                    | m1.xlarge | 16384 |  160 |         0 |     8 | True      |
| b7ebbca2-6f10-4bfc-a8de-387cbdc38477 | octavia   |  4096 |   20 |         0 |     2 | True      |
+--------------------------------------+-----------+-------+------+-----------+-------+-----------+
octavia_amp_boot_network_list: "79f55a3b-3bba-4aec-a2a8-5eefaca033d1"
octavia_amp_secgroup_list: "8172cf36-0de2-4afb-82e9-62f5420e3879"
octavia_amp_flavor_id: "b7ebbca2-6f10-4bfc-a8de-387cbdc38477"

Then I re-run the deploy to update the octavia values.

The only way I can ssh into ... (more)

edit retag flag offensive close merge delete

Comments

that resloved or not? im getting same issue with Train.

archiephan gravatar imagearchiephan ( 2019-11-27 02:41:12 -0500 )edit

1 answer

Sort by ยป oldest newest most voted
0

answered 2020-03-03 02:17:12 -0500

ArunVinod gravatar image

Have you checked connecting to port 9443 on amphora instance from the controller?

telent amphora-ip 9443

Usualy, even if the amphora-instance is launched it may take couple of minutes for the services in the amphora to come up and running. The connection to the port 9443 on amphora will only happen once the services are up and during this time this error is expected. But, you have to consider paramater the build_active_retries and build_active_interval in octavia.conf, which should be sufficient to allow the amphora instance to be active.

Ideally once the amphora is up, login and check whether the services are running. Also, check whether any service is running on port 9443.

netstat -plane | grep 9443

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2019-11-04 09:41:17 -0500

Seen: 198 times

Last updated: Nov 04 '19