Elastic cloud storage control for non-uniform workloads

Abstract: Storage systems are a critical component of the 3-tier web applications increasingly deployed to cloud computing platforms. Elasticity is the cloud’s most marketable attribute, and in order to achieve this for storage systems automatic control is required to achieve self-management. This work investigates partition-demand aware control, demonstrating experimentally that this information can be turned to control able to consider the structure of the workload it is observing, rather than assuming it evenly distributed across the keyspace. This enables fine-grained load-balancing, leading to a reduction in cloud infrastructure rented by a self-managing storage system. Experimental results with the (Dynamo based) Voldemort key-value store demonstrate the functionality of this control mechanism for pathological examples, and lead the way for future work or integration with more sophisticated storage control systems.