Ask Your Question
0

Using openstack for training machine-learning models in the cloud?

asked 2018-02-08 12:08:37 -0600

Median gravatar image

Hello everyone. I am very new to anything related to cloud in general, so I apologize if this question is newbish or does not comply with the forum rules.

I am working on a project that requires heavy machine learning models to be trained. There are some features that we would like to implement that we think cloud computing/storage could provide us, and I would like to know if OpenStack could be the way to go.

1 - Training these machine learning models could take days or weeks on a standard machine. Is it possible to use openstack to take advantadge of remote CPU/GPU resources to make this process faster?

2 - At times, it may be necessary to provide software updates for the users, and these updates may require downloading some models (.txt / .csv / .xml files) into user's products. Can this be achieved using openstack?

3 - Can openstack be used to create and mantain a storage cloud for an image database (.png / .jpg / .bmp), and have this database be accessed by our software for retrieval and processing?

If OpenStack is the way to go for these kind of features, can you provide some resources for me to start get acquainted to how these things are done in practice? I've been trying to read some resources about this topic, but everything just sounds so abstact that I'm feeling a bit lost.

Thanks in advance.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2018-02-20 08:06:43 -0600

Stephen C gravatar image

Is it possible to use OpenStack to take advantage of remote CPU/GPU resources to make this process faster?

Yes.

  1. OpenStack compute nodes with GPU hardware can be configured so that an instance can use the GPU. Compute nodes with multiple GPUs can be problematic though.

  2. There are OpenStack cloud providers who offer GPU-enabled instances. (According to Google)

  3. If you rented a GPU-enabled instance from a provider, you should be able to set up something so that you could send it requests remotely to do things. Obviously, if the requests are going to take a long time, then you need to design your system for sending requests / getting replies to handle this.


At times, it may be necessary to provide software updates for the users, and these updates may require downloading some models (.txt / .csv / .xml files) into user's products. Can this be achieved using openstack?

It is not clear what you mean here.

In general, an OpenStack instance with a public IP address can make files available to be downloaded; e.g. by running a Web server or FTP server. Your software running on the users machine could be implemented implemented to connect to a cloud server and download updates. (Provided that the user's machine has internet access.)


Can OpenStack be used to create and maintain a storage cloud for an image database (.png / .jpg / .bmp), and have this database be accessed by our software for retrieval and processing?

Yes. This is essential the same as the previous question. Web servers etcetera are generally agnostic to the type of documents they are serving.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Get to know Ask OpenStack

Resources for moderators

Question Tools

1 follower

Stats

Asked: 2018-02-08 12:07:19 -0600

Seen: 59 times

Last updated: Feb 08