Hello,

I'm trying to set up metering for neutron in Pike. I tested it with a centralized router and it works, but when I try with a distributed router it doesn't record any usage samples. I have one compute node and one network node and I've created an instance with a floating ip.

openstack router show public-router2
+-------------------------+----------------------------------------------------+
| Field                   | Value                                              |
+-------------------------+----------------------------------------------------+
| admin_state_up          | UP                                                 |
| availability_zone_hints |                                                    |
| availability_zones      | nova                                               |
| created_at              | 2018-10-05T12:07:32Z                               |
| description             |                                                    |
| distributed             | True                                               |
| external_gateway_info   | {"network_id": "b96473ce-                          |
|                         | 94f6-464f-a703-5285fb8ff3d3", "enable_snat": true, |
|                         | "external_fixed_ips": [{"subnet_id":               |
|                         | "6c08c3d9-7df1-4bec-b847-19f80b9d1764",            |
|                         | "ip_address": "192.168.252.102"}]}                 |
| flavor_id               | None                                               |
| ha                      | False                                              |
| id                      | 37c1794b-58d1-4d0d-b34b-944ca411b86b               |
| name                    | public-router2                                     |
| project_id              | fe203109e67f4e39b066c9529f9fc35d                   |
| revision_number         | 5                                                  |
| routes                  |                                                    |
| status                  | ACTIVE                                             |
| tags                    |                                                    |
| updated_at              | 2018-10-05T12:09:36Z                               |
+-------------------------+----------------------------------------------------+

openstack network agent list
+-----------+------------+-----------+-------------------+-------+-------+--------------+
| ID        | Agent Type | Host      | Availability Zone | Alive | State | Binary       |
+-----------+------------+-----------+-------------------+-------+-------+--------------+
| 14b9ea75- | L3 agent   | compute1. | nova              | :-)   | UP    | neutron-l3-a |
| 1dc1-4e37 |            | localdoma |                   |       |       | gent         |
| -a2b0-508 |            | in        |                   |       |       |              |
| 3d336916d |            |           |                   |       |       |              |
| 26139ec1- | Metering   | compute1. | None              | :-)   | UP    | neutron-     |
| f4f9-4bb3 | agent      | localdoma |                   |       |       | metering-    |
| -aebb-c35 |            | in        |                   |       |       | agent        |
| 3a36ed79c |            |           |                   |       |       |              |
| 2a54971f- | DHCP agent | network1. | nova              | :-)   | UP    | neutron-     |
| 9849-4ed2 |            | localdoma |                   |       |       | dhcp-agent   |
| -b009-00e |            | in        |                   |       |       |              |
| 45eb4d255 |            |           |                   |       |       |              |
| 443c0b49- | Open       | compute1. | None              | :-)   | UP    | neutron-     |
| 4484-44d2 | vSwitch    | localdoma |                   |       |       | openvswitch- |
| -a704-32a | agent      | in        |                   |       |       | agent        |
| 92ffe6982 |            |           |                   |       |       |              |
| 5d00a219  | L3 agent   | network1. | nova              | :-)   | UP    | neutron-vpn- |
| -abce-    |            | localdoma |                   |       |       | agent        |
| 48ca-     |            | in        |                   |       |       |              |
| ba1e-d962 |            |           |                   |       |       |              |
| 01bd7de3  |            |           |                   |       |       |              |
| bc3458b4  | Open       | network1. | None              | :-)   | UP    | neutron-     |
| -250e-    | vSwitch    | localdoma |                   |       |       | openvswitch- |
| 4adf-90e0 | agent      | in        |                   |       |       | agent        |
| -110a1a7f |            |           |                   |       |       |              |
| 6ccb      |            |           |                   |       |       |              |
| c29f9da8- | Metering   | network1. | None              | :-)   | UP    | neutron-     |
| ca58-4a11 | agent      | localdoma |                   |       |       | metering-    |
| -b500-a25 |            | in        |                   |       |       | agent        |
| 3f820808e |            |           |                   |       |       |              |
| cdce667d- | Metadata   | network1. | None              | :-)   | UP    | neutron-     |
| faa4      | agent      | localdoma |                   |       |       | metadata-    |
| -49ed-    |            | in        |                   |       |       | agent        |
| 83ee-e0e5 |            |           |                   |       |       |              |
| a352d482  |            |           |                   |       |       |              |
| cf5ae104- | Metadata   | compute1. | None              | :-)   | UP    | neutron-     |
| 49d7-4c85 | agent      | localdoma |                   |       |       | metadata-    |
| -a252-cc5 |            | in        |                   |       |       | agent        |
| 9a9a12789 |            |           |                   |       |       |              |
+-----------+------------+-----------+-------------------+-------+-------+--------------+

If I check the node on which my distributed router is running it tells me that it's running on the network node:

neutron l3-agent-list-hosting-router 37c1794b-58d1-4d0d-b34b-944ca411b86b
+--------------------------------------+----------------------+----------------+-------+----------+
| id                                   | host                 | admin_state_up | alive | ha_state |
+--------------------------------------+----------------------+----------------+-------+----------+
| 5d00a219-abce-48ca-ba1e-d96201bd7de3 | network1.localdomain | True           | :-)   |          |
+--------------------------------------+----------------------+----------------+-------+----------+

If I check the iptable rules for the router on the compute and network nodes by running

ip netns exec qrouter-37c1794b-58d1-4d0d-b34b-944ca411b86b iptables -nv -L

I see that compute1 records the traffic while network1 doesn't. Also, I did some debugging and found out that the metering agent on compute1 receives an empty list of routers when querying the routers that it should monitor. Source:

github.com/openstack/neutron/blob/stable/pike/neutron/services/metering/agents/metering_agent.py#L177-L189

github.com/openstack/neutron/blob/stable/pike/neutron/db/metering/metering_rpc.py#L33-L57

I think this is because the router is running on network1. Why is it running on network1 and why does it seem that compute1 does the actual routing?