Cloud Auto-Scaling Control Engine Based on Machine Learning

Abstract: With the development of modern data centers and networks, many service providers have moved most of their computing functions to the cloud.  Considering the limitation of network bandwidth and hardware or virtual resources, how to manage different virtual resources in a cloud environment so as to achieve better resource allocation is a big problem.  Although some cloud infrastructures provide simple default auto-scaling and orchestration mechanisms, such as OpenStack Heat service, they usually only depend on a single parameter, such as CPU utilization and cannot respond to the network changes in a timely manner.<p> This thesis investigates different auto-scaling mechanisms and designs an on-line control engine that cooperates with different OpenStack service APIs based on various network resource data.  Two auto-scaling engines, Heat orchestration based engine and machine learning based online control engine, have been developed and compared for different client requests patterns.  Two machine learning methods, neural network, and linear regression have been considered to generate a control signal based on real-time network data.  This thesis also shows the network’s non-linear behaviors for heavy traffic and proposes a scaling policy based on deep network analysis.<p> The results show that for offline training, the neural network and linear regression provide 81.5% and 84.8% accuracy respectively.  However, for online testing with different client request patterns, the neural network results are different than we expected, while linear regression provided us with much better results.  The model comparison showed that these two auto-scaling mechanisms have similar behavior for a SMOOTH-load Pattern.  However, for the SPIKEY-load Pattern, the linear regression based online control engine responded faster to network changes while heat orchestration service shows some delay.  Compared with the proposed scaling policy with fewer web servers in use and acceptable response latency, both of the two auto-scaling models waste network resources.