ok, found that the vxlan remote_ip endpoints can be deleted from the neutron db.

I had something like this:

 Port "vxlan-0b006817"
            Interface "vxlan-0b006817"
                type: vxlan
                options: {df_default="true", in_key=flow, local_ip="11.0.103.23", out_key=flow, remote_ip="11.0.104.23"}

11.0.103.23 should be mapped to 11.0.103.x...instead of the 11.0.104.x On the controller, open mariadb, use the neutron db, then delete the offending endpoints from the ml2_vxlan_endpoints table.

like so:

MariaDB [neutron]> select * from ml2_vxlan_endpoints;
+-------------+----------+
| ip_address  | udp_port |
+-------------+----------+
| 11.0.103.23 |     4789 |
| 11.0.103.24 |     4789 |
| 11.0.104.24 |     4789 |
| 11.0.104.25 |     4789 |
| 11.0.104.26 |     4789 |
| 11.0.107.23 |     4789 |
+-------------+----------+
6 rows in set (0.00 sec)

MariaDB [neutron]> delete from ml2_vxlan_endpoints where ip_address = "11.0.104.24" limit 1;
Query OK, 1 row affected (0.00 sec)

MariaDB [neutron]> delete from ml2_vxlan_endpoints where ip_address = "11.0.104.25" limit 1;
Query OK, 1 row affected (0.00 sec)

MariaDB [neutron]> delete from ml2_vxlan_endpoints where ip_address = "11.0.104.26" limit 1;
Query OK, 1 row affected (0.00 sec)

MariaDB [neutron]> select * from ml2_vxlan_endpoints;
+-------------+----------+
| ip_address  | udp_port |
+-------------+----------+
| 11.0.103.23 |     4789 |
| 11.0.103.24 |     4789 |
| 11.0.107.23 |     4789 |
+-------------+----------+

I wanted to remove the x.x.104.x IPs...and now they are gone.