Problem: We use local storage for instances. Sometimes an instance can reach 1TB root disk. When migrating such instances between compute nodes it can take many hours.

We use 2 x 10Gb/sec bonded network, enterprise SSDs in RAID 10.

Solution: After testing several solutions (including manual netcat, HPN-SSH), we have found 'bbcp' (http://pcbunn.cithep.caltech.edu/bbcp...) - using bbcp with the default of 4 streams, we can obtain 1.1GB/sec of data transfer (saturating a 10Gb/sec link and most likely the SSD write speed on the target host). We have successfully migrated a 1TB file in 16 minutes.

bbcp could be used as a drop in replacement for scp for migrations, as it runs over SSH, and 'just works'.

Question: How would one have OpenStack's migration feature use bbcp rather than scp? Solutions such as:

• A configuration parameter to provide the migration vehicle of bbcp rather than scp.
• Editing the source code somewhere to call bbcp rather than scp.
• Possibly aliasing scp on the command line to call bbcp instead (although i wanted to avoid this and use it for migrations only).

If anyone has any insight on the above it would be greatly appreciated.