How to dedicate compute hosts to domains?

asked 2017-09-12 02:48:24 -0500

gekaklam gravatar image

Usecase

We want to deploy instances from different domains, to specific compute hosts. Or to phrase it differently:

As a Cloud provider, I want to isolate aggregates at the domain level (i.e. assign which domain can spawn instances on a given aggregate)

Details

In our Openstack Deployment (Mitaka), we have to support 3 different domains (besides the default). We need a way to separate the compute hosts in three groups (aggregates), so that VMs belonging to users of domain A, start in group A, etc. Initially we assume that each compute host will belong only to one group, but that might change.

Domain info:

  • Alice
    • ~200 users (and increasing)
    • 1 project per user (~200 projects)
  • Bob
    • ~5 projects (fixed number)
    • various number of users per project
  • Carol
    • 1-2 projects (fixed number)
    • ~ 5 users / project

What we've tried

We have looked at the https://docs.openstack.org/nova/latest/user/filter-scheduler.html (nova filter scheduler) and on the https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_multitenancy_isolation.py (aggregate_multitenacy_isolation) filter, which is doing what we want but it works on https://www.brad-x.com/2016/01/01/dedicate-compute-hosts-to-projects/ (project level). Given that in one of our domains, we'll have at least 200 projects, we'd prefer to leave this as a last choice.

Modifying the above filter to make the check based on the domain, isn't possible.

The object that the filter receives, and contains the information, is the http://specs.openstack.org/openstack/nova-specs/specs/mitaka/implemented/request-spec-object-mitaka.html# (RequestSpec Object). The information contained in its fields, https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L46 (doesn't include the domain_id).

Possible solutions that we've think of

  1. Write our own filter
    Modify a filter to contain a call to keystone, where it would send the project_id, and get back it's domain. But this feels more like a hack than a proper solution. And it might require storing the admin credentials to the node that the filters are running (controller?), which we'd like to avoid.

  2. Make the separation at a different level (project/flavor/image)
    Besides the isolation per project, we could also isolate the hosts, by providing different images / flavors to the different users. There are available filters for that (https://github.com/openstack/nova/blob/master/nova/scheduler/filters/image_props_filter.py (image_props_filter), https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_instance_extra_specs.py (aggregate_instance_extra_specs)). But again, due to the high number of projects, this would not scale well.

  3. Modify the RequestSpec object
    Finally, we modify the RequestSpec object to include the domain_id field. Then modify the aggregate_multitenacy_isolation filter, to work on domain level. This would be the most elegant solution. However, (besides not knowing how to do that), we don't know what kind of implication that will have and how to package / deploy it.

Is anyone having the same usecase? How would you go about doing it?

It's interesting though, since we though that ... (more)

edit retag flag offensive close merge delete