Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

HTTPS not working to Instance using FloatingIP

I actually have two networking problems, but the more pressing one first: From an outside node, or the Controller node for that matter, I cannot access any HTTPS ports on my instances.

Configuration: 3-node (Controller, Network, Compute) Ubuntu Juno OpenStack. This was installed using the default, manual install with Neutron networking as documented on the docs.openstack.org website. No errors in logs, Cirros instance launches with full SSH access. CentOS7 instance launches with SSH working until I try a "large" output (ls -al of a big directory hangs about 20 lines in [IE> the second network problem]). F5 Networks BIG-IP VE instance launches with SSH working until I try a "large" output. Both the CentOS7 and the BIGIP keep running, and I have full console access at all times. No errors reported in either instance log files. All nodes get all their DHCP assigned IPs. 'Default' security group setup like so:

    # nova secgroup-list-rules default
    +-------------+-----------+---------+-----------+--------------+
    | IP Protocol | From Port | To Port | IP Range  | Source Group |
    +-------------+-----------+---------+-----------+--------------+
    |             |           |         |           | default      | 
    | tcp         | 22        | 22      | 0.0.0.0/0 |              |
    | tcp         | 443       | 443     | 0.0.0.0/0 |              |
    |             |           |         |           | default      | 
    | icmp        | -1        | -1      | 0.0.0.0/0 |              |
    | tcp         | 80        | 80      | 0.0.0.0/0 |              |
    +-------------+-----------+---------+-----------+--------------+

When I do an tcpdump from the BIGIP node, I can see the HTTPS packets coming in, and a response going out. If I create a mirror port on the 'br-int' bridge on the compute node, I just see the responses going out to my client, but no requests coming in?!? That doesn't seem right!

11:25:38.966302 IP6 fe80::bc29:bff:fe04:e7d5 > ip6-allrouters: ICMP6, router solicitation, length 16
11:25:39.231659 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 2501911056:2501911092, ack 4236512516, win 241, options [nop,nop,TS val 81901769 ecr 275729760], length 36
11:25:42.974288 IP6 fe80::bc29:bff:fe04:e7d5 > ip6-allrouters: ICMP6, router solicitation, length 16
11:25:49.236382 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 36:72, ack 53, win 241, options [nop,nop,TS val 81911774 ecr 275739723], length 36
11:25:51.810641 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [S.], seq 1016497018, ack 1398343447, win 14480, options [mss 1460,sackOK,TS val 242030834 ecr 275742177,nop,wscale 7], length 0
11:25:51.813968 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], ack 211, win 122, options [nop,nop,TS val 242030838 ecr 275742190], length 0
11:25:51.832698 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242030857 ecr 275742190], length 1448
11:25:51.832743 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [P.], seq 1449:1654, ack 211, win 122, options [nop,nop,TS val 242030857 ecr 275742190], length 205
11:25:52.036650 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242031061 ecr 275742211], length 1448
11:25:52.444604 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242031469 ecr 275742211], length 1448
11:25:53.260628 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242032285 ecr 275742211], length 1448
11:25:54.252057 ARP, Reply 10.10.10.10 is-at fa:16:3e:eb:b3:d1 (oui Unknown), length 28
11:25:54.892655 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242033917 ecr 275742211], length 1448
11:25:56.812071 ARP, Reply 10.10.10.8 is-at fa:16:3e:72:3e:36 (oui Unknown), length 28
11:25:58.156733 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242037181 ecr 275742211], length 1448
11:25:59.241129 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 72:108, ack 105, win 241, options [nop,nop,TS val 81921779 ecr 275749591], length 36
11:26:04.685109 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242043709 ecr 275742211], length 1448
11:26:09.245127 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 108:144, ack 157, win 241, options [nop,nop,TS val 81931782 ecr 275759549], length 36
11:26:12.254353 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [F.], seq 1654, ack 211, win 122, options [nop,nop,TS val 242051278 ecr 275742211], length 0
11:26:17.740717 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242056765 ecr 275762546], length 1448
11:26:19.248879 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 144:180, ack 209, win 241, options [nop,nop,TS val 81941786 ecr 275769500], length 36
11:26:21.789158 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], ack 212, win 122, options [nop,nop,TS val 242060813 ecr 275772028], length 0
11:26:29.253090 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 180:216, ack 261, win 241, options [nop,nop,TS val 81951791 ecr 275779447], length 36

'10.10.10.8' is the BIGIP, '10.147.95.128' is my web browser node. Since I get SSH connections between the two nodes, I have to assume its not a routing issue.
If I attempt to access the webpage right from the virtual router, still no joy, it hangs and never comes back:

 # ip netns exec qrouter-c939264f-e3ee-45d5-a885-11ad94e04c12 wget https://10.10.10.8
--2015-07-02 12:16:08--  https://10.10.10.8/
Connecting to 10.10.10.8:443... connected.
^C

A 'curl' command locally on the node retrieves the page just fine with no errors.
Any clues about what I'm doing wrong?? Thanks!

HTTPS not working to Instance using FloatingIP

I actually have two networking problems, but the more pressing one first: From an outside node, or the Controller node for that matter, I cannot access any HTTPS ports on my instances.

Configuration: 3-node (Controller, Network, Compute) Ubuntu Juno OpenStack. This was installed using the default, manual install with Neutron networking as documented on the docs.openstack.org website. No errors in logs, Cirros instance launches with full SSH access. CentOS7 instance launches with SSH working until I try a "large" output (ls -al of a big directory hangs about 20 lines in [IE> the second network problem]). F5 Networks BIG-IP VE instance launches with SSH working until I try a "large" output. Both the CentOS7 and the BIGIP keep running, and I have full console access at all times. No errors reported in either instance log files. All nodes get all their DHCP assigned IPs. 'Default' security group setup like so:

    # nova secgroup-list-rules default
    +-------------+-----------+---------+-----------+--------------+
    | IP Protocol | From Port | To Port | IP Range  | Source Group |
    +-------------+-----------+---------+-----------+--------------+
    |             |           |         |           | default      | 
    | tcp         | 22        | 22      | 0.0.0.0/0 |              |
    | tcp         | 443       | 443     | 0.0.0.0/0 |              |
    |             |           |         |           | default      | 
    | icmp        | -1        | -1      | 0.0.0.0/0 |              |
    | tcp         | 80        | 80      | 0.0.0.0/0 |              |
    +-------------+-----------+---------+-----------+--------------+

When I do an tcpdump from the BIGIP node, I can see the HTTPS packets coming in, and a response going out. If I create a mirror port on the 'br-int' bridge on the compute node, I just see the responses going out to my client, but no requests coming in?!? That doesn't seem right!

11:25:38.966302 IP6 fe80::bc29:bff:fe04:e7d5 > ip6-allrouters: ICMP6, router solicitation, length 16
11:25:39.231659 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 2501911056:2501911092, ack 4236512516, win 241, options [nop,nop,TS val 81901769 ecr 275729760], length 36
11:25:42.974288 IP6 fe80::bc29:bff:fe04:e7d5 > ip6-allrouters: ICMP6, router solicitation, length 16
11:25:49.236382 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 36:72, ack 53, win 241, options [nop,nop,TS val 81911774 ecr 275739723], length 36
11:25:51.810641 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [S.], seq 1016497018, ack 1398343447, win 14480, options [mss 1460,sackOK,TS val 242030834 ecr 275742177,nop,wscale 7], length 0
11:25:51.813968 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], ack 211, win 122, options [nop,nop,TS val 242030838 ecr 275742190], length 0
11:25:51.832698 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242030857 ecr 275742190], length 1448
11:25:51.832743 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [P.], seq 1449:1654, ack 211, win 122, options [nop,nop,TS val 242030857 ecr 275742190], length 205
11:25:52.036650 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242031061 ecr 275742211], length 1448
11:25:52.444604 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242031469 ecr 275742211], length 1448
11:25:53.260628 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242032285 ecr 275742211], length 1448
11:25:54.252057 ARP, Reply 10.10.10.10 is-at fa:16:3e:eb:b3:d1 (oui Unknown), length 28
11:25:54.892655 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242033917 ecr 275742211], length 1448
11:25:56.812071 ARP, Reply 10.10.10.8 is-at fa:16:3e:72:3e:36 (oui Unknown), length 28
11:25:58.156733 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242037181 ecr 275742211], length 1448
11:25:59.241129 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 72:108, ack 105, win 241, options [nop,nop,TS val 81921779 ecr 275749591], length 36
11:26:04.685109 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242043709 ecr 275742211], length 1448
11:26:09.245127 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 108:144, ack 157, win 241, options [nop,nop,TS val 81931782 ecr 275759549], length 36
11:26:12.254353 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [F.], seq 1654, ack 211, win 122, options [nop,nop,TS val 242051278 ecr 275742211], length 0
11:26:17.740717 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242056765 ecr 275762546], length 1448
11:26:19.248879 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 144:180, ack 209, win 241, options [nop,nop,TS val 81941786 ecr 275769500], length 36
11:26:21.789158 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], ack 212, win 122, options [nop,nop,TS val 242060813 ecr 275772028], length 0
11:26:29.253090 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 180:216, ack 261, win 241, options [nop,nop,TS val 81951791 ecr 275779447], length 36

'10.10.10.8' is the BIGIP, '10.147.95.128' is my web browser node. Since I get SSH connections between the two nodes, I have to assume its not a routing issue.
If I attempt to access the webpage right from the virtual router, still no joy, it hangs and never comes back:

 # ip netns exec qrouter-c939264f-e3ee-45d5-a885-11ad94e04c12 wget https://10.10.10.8
[url here]
--2015-07-02 12:16:08--  https://10.10.10.8/
[url here]
Connecting to 10.10.10.8:443... connected.
^C

A 'curl' command locally on the node retrieves the page just fine with no errors.
Any clues about what I'm doing wrong?? Thanks!

click to hide/show revision 3
No.3 Revision

HTTPS not working to Instance using FloatingIP

I actually have two networking problems, but the more pressing one first: From an outside node, or the Controller node for that matter, I cannot access any HTTPS ports on my instances.

Configuration: 3-node (Controller, Network, Compute) Ubuntu Juno OpenStack. This was installed using the default, manual install with Neutron networking as documented on the docs.openstack.org website. No errors in logs, Cirros instance launches with full SSH access. CentOS7 instance launches with SSH working until I try a "large" output (ls -al of a big directory hangs about 20 lines in [IE> the second network problem]). F5 Networks BIG-IP VE instance launches with SSH working until I try a "large" output. Both the CentOS7 and the BIGIP keep running, and I have full console access at all times. No errors reported in either instance log files. All nodes get all their DHCP assigned IPs. 'Default' security group setup like so:

    # nova secgroup-list-rules default
    +-------------+-----------+---------+-----------+--------------+
    | IP Protocol | From Port | To Port | IP Range  | Source Group |
    +-------------+-----------+---------+-----------+--------------+
    |             |           |         |           | default      | 
    | tcp         | 22        | 22      | 0.0.0.0/0 |              |
    | tcp         | 443       | 443     | 0.0.0.0/0 |              |
    |             |           |         |           | default      | 
    | icmp        | -1        | -1      | 0.0.0.0/0 |              |
    | tcp         | 80        | 80      | 0.0.0.0/0 |              |
    +-------------+-----------+---------+-----------+--------------+

When I do an tcpdump from the BIGIP node, I can see the HTTPS packets coming in, and a response going out. If I create a mirror port on the 'br-int' bridge on the compute node, I just see the responses going out to my client, but no requests coming in?!? That doesn't seem right!

11:25:38.966302 IP6 fe80::bc29:bff:fe04:e7d5 > ip6-allrouters: ICMP6, router solicitation, length 16
16<br> 11:25:39.231659 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 2501911056:2501911092, ack 4236512516, win 241, options [nop,nop,TS val 81901769 ecr 275729760], length 36
36<br> 11:25:42.974288 IP6 fe80::bc29:bff:fe04:e7d5 > ip6-allrouters: ICMP6, router solicitation, length 16
16<br> 11:25:49.236382 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 36:72, ack 53, win 241, options [nop,nop,TS val 81911774 ecr 275739723], length 36
36<br> 11:25:51.810641 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [S.], seq 1016497018, ack 1398343447, win 14480, options [mss 1460,sackOK,TS val 242030834 ecr 275742177,nop,wscale 7], length 0
0<br> 11:25:51.813968 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], ack 211, win 122, options [nop,nop,TS val 242030838 ecr 275742190], length 0
0<br> 11:25:51.832698 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242030857 ecr 275742190], length 1448
1448<br> 11:25:51.832743 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [P.], seq 1449:1654, ack 211, win 122, options [nop,nop,TS val 242030857 ecr 275742190], length 205
205<br> 11:25:52.036650 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242031061 ecr 275742211], length 1448
1448<br> 11:25:52.444604 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242031469 ecr 275742211], length 1448
1448<br> 11:25:53.260628 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242032285 ecr 275742211], length 1448
1448<br> 11:25:54.252057 ARP, Reply 10.10.10.10 is-at fa:16:3e:eb:b3:d1 (oui Unknown), length 28
28<br> 11:25:54.892655 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242033917 ecr 275742211], length 1448
1448<br> 11:25:56.812071 ARP, Reply 10.10.10.8 is-at fa:16:3e:72:3e:36 (oui Unknown), length 28
28<br> 11:25:58.156733 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242037181 ecr 275742211], length 1448
1448<br> 11:25:59.241129 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 72:108, ack 105, win 241, options [nop,nop,TS val 81921779 ecr 275749591], length 36
36<br> 11:26:04.685109 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242043709 ecr 275742211], length 1448
1448<br> 11:26:09.245127 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 108:144, ack 157, win 241, options [nop,nop,TS val 81931782 ecr 275759549], length 36
36<br> 11:26:12.254353 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [F.], seq 1654, ack 211, win 122, options [nop,nop,TS val 242051278 ecr 275742211], length 0
0<br> 11:26:17.740717 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], seq 1:1449, ack 211, win 122, options [nop,nop,TS val 242056765 ecr 275762546], length 1448
1448<br> 11:26:19.248879 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 144:180, ack 209, win 241, options [nop,nop,TS val 81941786 ecr 275769500], length 36
36<br> 11:26:21.789158 IP 10.10.10.8.https > 10.147.95.128.63978: Flags [.], ack 212, win 122, options [nop,nop,TS val 242060813 ecr 275772028], length 0
0<br> 11:26:29.253090 IP 10.10.10.10.ssh > 10.147.95.128.57127: Flags [P.], seq 180:216, ack 261, win 241, options [nop,nop,TS val 81951791 ecr 275779447], length 36

36<br> '10.10.10.8' is the BIGIP, '10.147.95.128' is my web browser node. Since I get SSH connections between the two nodes, I have to assume its not a routing issue.
issue.<br> If I attempt to access the webpage right from the virtual router, still no joy, it hangs and never comes back:

back:<br>

 # ip netns exec qrouter-c939264f-e3ee-45d5-a885-11ad94e04c12 wget [url here]
--2015-07-02 12:16:08--  [url here]
Connecting to 10.10.10.8:443... connected.
^C

A 'curl' command locally on the node retrieves the page just fine with no errors.
Any clues about what I'm doing wrong?? Thanks!