jomlowe's profile - activity

2018-06-24 19:17:08 -0500 answered a question Ceph and erasure coding

Erasure coding is a per storage pool configuration. For technical reasons out of scope an erasure coded pool must be paired with a replicated pool to do RBD block devices. The replicated pool holds the metadata for the rbd device while the actual data goes to the erasure coded pool. To the best of my knowledge the rbd driver does not support erasure coding as it does not know how to set a data pool. Erasure coding is set at pool creation so you cannot just change in place from replicated to erasure coding. You will also probably want to run luminous or mimic with bluestore osds so you can turn on allow_ec_overwrites to get acceptable performance.

2018-06-24 19:17:08 -0500 answered a question Why is nova cli not using https?

When parsing the endpoint the nova client doesn't seem to always do the right thing with a /v2.1 path, a simple workaround is to add a trailing slash to the endpoint path i.e https://controller:8774/v2.1/ instead of https://controller:8774/v2.1

2018-06-24 19:17:08 -0500 asked a question Why is nova cli not using https?

When using a TLS enabled endpoint for nova, the nova cli tries to connect with http instead of https. The openstack cli does the correct thing and uses https. Why does nova and the python nova client used by heat for example not honor https in the endpoint?

2018-06-24 19:17:08 -0500 answered a question Why is nova ignoring microvesions

Answering my own question here after hammering it out with the nova devs on irc. Older versions used /v2/%(tenant_id)s as the path for the nova endpoint, this locks you into v2 compatibility mode where microversions are ignored. Changing the endpoint path to /v2.1 will enable the v2.1 api with microversions. The queens release of magnum uses a server group with soft anti affinity which requires nova microversion 2.15, this is where I first discovered with the legacy endpoint still present in my deployment. Also note that some nova clients need a trailing slash /v2.1/ to correctly parse http vs https. This should probably get fixed in nova client but using this workaround will get you by until all the clients everywhere are fixed.

2018-06-24 19:17:08 -0500 asked a question Why is nova ignoring microvesions

I have a Pike install that has been upgraded all the way from Kilo and I can't seem to get nova to honor microversions. What is going wrong?

2018-06-24 19:17:08 -0500 commented answer Openstack neutron l3 agent dead

I'm now noticing that my 3 controllers have their own q-l3-plugin.HOST queue and the sum of the consumer counts now matches the consumer count for the main q-l3-plugin queue. So maybe I had some config change that dropped my number of rpc workers leaving phantom consumers gumming up the works.

2018-06-24 19:17:07 -0500 answered a question Openstack neutron l3 agent dead

I had a similar problem, I noticed my q-l3-plugin queue in rabbitmq had an ever increasing number of messages and plenty of consumers 'rabbitmqctl list_queues name consumers messages |grep q-l3-plugin'. I deleted the q-l3-plugin queue and broke the logjam. One interesting thing I noticed I had debug on and until I did this the config wasn't being dumped into the logs for the L3 agent until I cleared this queue. There are a number of stack overflow answers for how to delete, I used the pika one from here: https://stackoverflow.com/questions/5313027/rabbitmq-how-do-i-delete-all-messages-from-a-single-queue (https://stackoverflow.com/questions/5...)