Algorithms, methods and tools to migrate high-performance computing science and engineering applications to the cloud

Abstract

In the cloud computing scenario, the user may choose among several hardware configurations and prices to configure his/her high performing cluster of computers. This opens the opportunity for several optimizations, such as avoiding long waits on job queues and creating specialized clusters for each application. However, migrating the code to the cloud, selecting the most cost-effective set of resources for each application and dealing with performance fluctuations on virtual network infrastructures are still challenges that must be tackled [1].To mitigate these network performance problems, some providers are offering specialized HPC services on the cloud. These services provide guarantees on network performance, nonetheless, they are more expensive and usually limited to a maximum number of machines that can be rented. While some high-performance applications may benefit from these extra guarantees, others are tolerant of network performance variations. Hence, selecting the most cost-effective service for each application may be a challenge itself.High-performance science and engineering applications may depend on multiple software packages, including specialized libraries. However, in many cases, installing and configuring these packages on new systems may be challenging. In this sense, it is crucial to investigate technologies to ease the migration of high-performance programs from workstations and local clusters to the cloud.The main goal here is to investigate techniques and methods to ease the migration of existing high-performance computing applications to the cloud and to optimize the execution of high-performance applications on cloud resources. To facilitate code migration, we plan to investigate the use of container technologies [2].[1] Netto, M. A. S., Calheiros, R. N., Rodrigues, E. R., Cunha, R. L. F., Buyya, R. HPC Cloud for Scientific and Business Applications: Taxonomy, Vision, and Research Challenges. ACM Computing Surveys 51 (2018).[2] Pahl, C., Brogi, A., Soldani, J., and Jamshidi, P. Cloud Container Technologies: a State- of-the-Art Review. IEEE Transactions on Cloud Computing (2017).