Ask Your Question
4

How to start a GPU instance? [closed]

asked 2015-03-31 09:36:38 -0500

bishoy gravatar image

Hi all, I have been doing some research about starting instances with GPU capabilities and till now I have found info about Flavors extra options and aggregation filters but I still don't know how to configure Nova to communicate with Tesla GPUs. It still not clear to me how to integrate that and do the PCI passthrough, the docs aren't clear enough. Any one had some luck with that?

Note. I don't want to use LXC as it a lot of the nova features will not be present which is not good.

edit retag flag offensive reopen merge delete

Closed for the following reason the question is answered, right answer was accepted by bishoy
close date 2015-08-07 08:23:57.662705

1 answer

Sort by ยป oldest newest most voted
2

answered 2015-08-07 08:23:29 -0500

bishoy gravatar image

Start with steps 1 - 5 on this page: http://www.linux-kvm.org/page/How_to_...

However, if you have an Intel CPU, then the grep commands are wrong. Instead, use:

dmesg | grep -iE "dmar|iommu"

You should see, among the other stuff: Intel-IOMMU: enabled

Then follow the instructions here: https://wiki.openstack.org/wiki/Pci_p.... All of the configuration settings go in the [DEFAULT] section of /etc/nova/nova.conf. You can use "lspci -nn | grep -i nvidia" to get vendor and product IDs for the Tesla. The vendor ID is 10de. The product ID depends on your Tesla. For an M2070, it is 06d2. You probably have something else though.

So the relevant settings in your nova.conf should look like:

[DEFAULT]
pci_passthrough_whitelist = [{ "vendor_id": "10de", "product_id": "06d2" }]
pci_alias = { "name": "gpu", "vendor_id": "10de", "product_id": "06d2" }

scheduler_driver = nova.scheduler.filter_scheduler.FilterScheduler
scheduler_available_filters = nova.scheduler.filters.all_filters
scheduler_available_filters = nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter
scheduler_default_filters = RamFilter,ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,PciPassthroughFilter

If you are using DevStack, you will need to escape the quotes for these PCI settings in local.conf, or else they will be stripped out of the resulting nova.conf. If you do end up with an incorrect nova.conf and you edit it by hand to fix things, remember that multiple services depend on it and you should restart them all: not just nova-compute.

The relevant local.conf snippet looks like:

[[post-config|$NOVA_CONF]]
[DEFAULT]
pci_passthrough_whitelist=[{ \\"vendor_id\\": \\"10de\\", \\"product_id\\": \\"06d2\\" }]
pci_alias={ \\"name\\": \\"gpu\\", \\"vendor_id\\": \\"10de\\", \\"product_id\\": \\"06d2\\" }
scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
scheduler_available_filters=nova.scheduler.filters.all_filters
scheduler_available_filters=nova.scheduler.filters.pci_passthrough_filter.PciPassthroughFilter
scheduler_default_filters=RamFilter,ComputeFilter,AvailabilityZoneFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,PciPassthroughFilter

Actually, you can have more than one alias, and you can also get more specific about devices in the whitelist (the whitelist says what devices are allowed to be passed through to instances). If that parses out correctly then you should be able to see this stuff in the nova database:

mysql -u root -p nova
Enter password: 

mysql> select * from pci_devices;
+---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+---------------+------------+-----------+
| created_at          | updated_at          | deleted_at | deleted | id | compute_node_id | address      | product_id | vendor_id | dev_type | dev_id           | label           | status    | extra_info | instance_uuid | request_id | numa_node |
+---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+---------------+------------+-----------+
| 2015-04-15 07:33:22 | 2015-04-15 08:08:30 | NULL       |       0 |  1 |               1 | 0000:1b:00.0 | 06d2       | 10de      | type-PCI | pci_0000_1b_00_0 | label_10de_06d2 | available | {}         | NULL          | NULL       |      NULL |
| 2015-04-15 07:33:22 | 2015-04-15 08:08:30 | NULL       |       0 |  2 |               1 | 0000:1c:00.0 | 06d2       | 10de      | type-PCI | pci_0000_1c_00_0 | label_10de_06d2 | available | {}         | NULL          | NULL       |      NULL |
+---------------------+---------------------+------------+---------+----+-----------------+--------------+------------+-----------+----------+------------------+-----------------+-----------+------------+---------------+------------+-----------+
2 rows in set (0.00 sec)

mysql> select * from instance_type_extra_specs;
+---------------------+------------+------------+----+------------------+-----------------------+-------+---------+
| created_at          | updated_at | deleted_at | id | instance_type_id | key                   | value | deleted |
+---------------------+------------+------------+----+------------------+-----------------------+-------+---------+
| 2015-04-15 07:36:55 | NULL       | NULL       |  1 |                8 | pci_passthrough:alias | gpu:1 |       0 |
+---------------------+------------+------------+----+------------------+-----------------------+-------+---------+
1 row in set (0.00 sec)

Given the alias, you can then set up a flavor with:

nova flavor-create 1-gpu 100 4096 40 1
nova flavor-key 1-gpu set "pci_passthrough:alias"="gpu:1"

Then to boot an instance with one Tesla GPU, you specify the m1.small.1-gpu flavor.

If the instance fails to start and you see this in your nova-compute log file:

libvirtError: unsupported configuration: host doesn't support ...
(more)
edit flag offensive delete link more

Get to know Ask OpenStack

Resources for moderators

Question Tools

2 followers

Stats

Asked: 2015-03-31 09:36:38 -0500

Seen: 2,172 times

Last updated: Aug 07 '15