no service running at port 9697

asked 2015-04-29 02:24:09 -0600

ogzy
53 ●11 ●13 ●15

Here is my problem

$ sudo ip netns exec qrouter-429424ce-6d82-49cc-bc6a-9dc5236ba313 iptables-save | grep 169
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
$ sudo ip netns exec qrouter-429424ce-6d82-49cc-bc6a-9dc5236ba313 netstat -pantu | grep 9697

So somehow i don't see any service running at port 9697 when i check the namespace. I am using juno, wtih neutron + gre settings.

$ ps -efl | grep neutron
5 S root       370     1  0  80   0 - 23678 ep_pol Mar26 ?        00:00:02 /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/1d518823-dd8e-4308-a41c-5338cfe3e6af.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --network_id=1d518823-dd8e-4308-a41c-5338cfe3e6af --state_path=/var/lib/neutron --metadata_port=80 --log-file=neutron-ns-metadata-proxy-1d518823-dd8e-4308-a41c-5338cfe3e6af.log --log-dir=/var/log/neutron
4 S neutron   6054     1  0  80   0 - 29574 ep_pol 10:00 ?        00:00:00 /usr/bin/python /usr/bin/neutron-l3-agent --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/l3_agent.ini --config-file=/etc/neutron/fwaas_driver.ini --log-file=/var/log/neutron/l3-agent.log
0 S open      8473  4669  0  80   0 -  2616 pipe_w 10:21 pts/0    00:00:00 grep --color=auto neutron
4 S neutron   9138     1  0  80   0 - 27906 ep_pol Mar27 ?        00:05:34 /usr/bin/python /usr/bin/neutron-dhcp-agent --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/dhcp_agent.ini --log-file=/var/log/neutron/dhcp-agent.log
5 S nobody    9184     1  0  80   0 -  7050 poll_s Mar27 ?        00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap83f31177-be --except-interface=lo --pid-file=/var/lib/neutron/dhcp/1d518823-dd8e-4308-a41c-5338cfe3e6af/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/1d518823-dd8e-4308-a41c-5338cfe3e6af/host --addn-hosts=/var/lib/neutron/dhcp/1d518823-dd8e-4308-a41c-5338cfe3e6af/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/1d518823-dd8e-4308-a41c-5338cfe3e6af/opts --leasefile-ro --dhcp-range=set:tag0,172.16.4.0,static,86400s --dhcp-lease-max=256 --conf-file=/etc/neutron/dnsmasq-neutron.conf --domain=openstacklocal
5 S nobody    9187     1  0  80   0 -  7050 poll_s Mar27 ?        00:00:00 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap9e0f9bde-3c --except-interface=lo --pid-file=/var/lib/neutron/dhcp/79811367-b27d-4efd-83c9-902bd5c78aed/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/79811367-b27d-4efd-83c9-902bd5c78aed/host --addn-hosts=/var/lib/neutron/dhcp/79811367-b27d-4efd-83c9-902bd5c78aed/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/79811367-b27d-4efd-83c9-902bd5c78aed/opts --leasefile-ro --dhcp-range=set:tag0,172.16.3.0,static,86400s --dhcp-lease-max=256 --conf-file=/etc/neutron/dnsmasq-neutron.conf --domain=openstacklocal
4 S neutron  12591     1  0  80   0 - 28058 ep_pol Mar13 ?        00:04:49 /usr/bin/python /usr/bin/neutron-lbaas-agent --config-file=/etc/neutron/lbaas_agent.ini --config-file=/etc/neutron/neutron.conf --log-file=/var/log/neutron/lbaas-agent.log
4 S neutron  14606     1  0  80   0 - 57192 ep_pol Apr28 ?        00:03:56 /usr/bin/python /usr/bin/neutron-server --config-file /etc/neutron/neutron.conf --log-file /var/log/neutron/server.log --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
4 S neutron  14656     1  0  80   0 - 26708 ep_pol Apr28 ?        00:03:07 /usr/bin/python /usr/bin/neutron-openvswitch-agent --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/plugins/ml2/ml2_conf.ini --log-file=/var/log/neutron/openvswitch-agent.log
4 S root     14814 14656  0  80   0 - 15524 poll_s Apr28 ?        00:00:00 sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ovsdb-client monitor Interface name,ofport --format=json
4 S root     14816 ...

(more)

edit retag flag offensive close merge delete

add a comment

2 answers

Sort by » oldest newest most voted

answered 2015-04-29 03:10:32 -0600

dbaxps

10788 ●57 ●112 ●224 http://bderzhavets.blo...

updated 2015-04-29 11:05:17 -0600

It should look like :-

[root@junoVHS01 ~(keystone_admin)]# ip netns exec qrouter-ecfefef4-4cee-49ff-900c-ac5773031cec netstat -anpt
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:9697            0.0.0.0:*               LISTEN      4487/python         
[root@junoVHS01 ~(keystone_admin)]# ps -ef | grep 4487
root      4487     1  0 11:00 ?        00:00:00 /usr/bin/python /bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/ecfefef4-4cee-49ff-900c-ac5773031cec.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=ecfefef4-4cee-49ff-900c-ac5773031cec --state_path=/var/lib/neutron --metadata_port=9697 --verbose --log-file=neutron-ns-metadata-proxy-ecfefef4-4cee-49ff-900c-ac5773031cec.log --log-dir=/var/log/neutron

Per your report

$ ps -efl | grep neutron
5 S root       370     1  0  80   0 - 23678 ep_pol Mar26 ?        00:00:02 /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/1d518823-dd8e-4308-a41c-5338cfe3e6af.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --network_id=1d518823-dd8e-4308-a41c-5338cfe3e6af --state_path=/var/lib/neutron --metadata_port=80 --log-file=neutron-ns-metadata-proxy-1d518823-dd8e-4308-a41c-5338cfe3e6af.log --log-dir=/var/log/neutron

So, --metadata_port=80 is incorrect . I believe you are missing :-

[root@junoVHS01 neutron(keystone_admin)]# cat l3_agent.ini | grep 9697
# metadata_port = 9697
metadata_port = 9697

The default gateway say 50.0.0.1 exists within a Neutron router namespace on the network node. The Neutron-l3-agent starts a namespace proxy in this namespace and adds some iptables rules to redirect metadata requests to it.

[root@junoVHS01 ~(keystone_admin)]# ip netns exec qrouter-ecfefef4-4cee-49ff-900c-ac5773031cec route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 qg-0289d92f-ca
50.0.0.0        0.0.0.0         255.255.255.0   U     0      0        0 qr-bdc3038d-50
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 qg-0289d92f-ca
[root@junoVHS01 ~(keystone_admin)]# ip netns exec qrouter-ecfefef4-4cee-49ff-900c-ac5773031cec ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qg-0289d92f-ca: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.150  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::f816:3eff:fee8:fb84  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:e8:fb:84  txqueuelen 0  (Ethernet)
        RX packets 22308  bytes 27355095 (26.0 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13510  bytes 1209639 (1.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

qr-bdc3038d-50: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 50.0.0.1  netmask 255.255.255.0  broadcast 50.0.0.255
        inet6 fe80::f816:3eff:fec8:4bf  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:c8:04:bf  txqueuelen 0  (Ethernet)
        RX packets 13465  bytes 1207883 (1.1 MiB)
        RX errors 0  dropped 3  overruns 0  frame 0 ...

(more)

edit flag offensive delete link

add a comment

answered 2015-04-29 10:24:39 -0600

ogzy
53 ●11 ●13 ●15

updated 2015-04-29 10:25:35 -0600

I had checked the l3_agent.ini. The port settings was there. After restarting the compute node i had seen the below processes running

$ ps -efl | grep /usr/bin/neutron-ns-metadata-proxy
5 S root      6548     1  0  80   0 - 23920 ep_pol 15:12 ?        00:00:00 /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/429424ce-6d82-49cc-bc6a-9dc5236ba313.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --router_id=429424ce-6d82-49cc-bc6a-9dc5236ba313 --state_path=/var/lib/neutron --metadata_port=9697 --log-file=neutron-ns-metadata-proxy-429424ce-6d82-49cc-bc6a-9dc5236ba313.log --log-dir=/var/log/neutron
5 S root      7990     1  0  80   0 - 23678 ep_pol 15:13 ?        00:00:00 /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_file=/var/lib/neutron/external/pids/1d518823-dd8e-4308-a41c-5338cfe3e6af.pid --metadata_proxy_socket=/var/lib/neutron/metadata_proxy --network_id=1d518823-dd8e-4308-a41c-5338cfe3e6af --state_path=/var/lib/neutron --metadata_port=80 --log-file=neutron-ns-metadata-proxy-1d518823-dd8e-4308-a41c-5338cfe3e6af.log --log-dir=/var/log/neutron

I don't know why this service was not running. Any idea which service is making the process run?

edit flag offensive delete link

Comments

The Neutron-l3-agent starts a namespace proxy in this namespace and adds some iptables rules to redirect metadata requests to it.

dbaxps ( 2015-04-29 10:57:09 -0600 )edit

Details here http://bderzhavets.blogspot.com/2014/...

dbaxps ( 2015-04-29 10:57:58 -0600 )edit

add a comment

no service running at port 9697

2 answers

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

no service running at port 9697 edit

2 answers

Comments

Your Answer

Get to know Ask OpenStack

Question Tools

Stats

Related questions

Feedback About This Page

OpenStack

Community

Documentation

Branding & Legal

no service running at port 9697