Medium Data on Big Data Predicting Disk Failures in CERNs NetApp-based Data Storage System

Abstract: I describe in this report an experimental system for using classification and regression trees to generate predictions of disk failures in a NetApp-based storage system at the European Organisation for Nuclear Research (CERN) based on a mixture of SMART data, system logs, and low-level system performance dataparticular to NetApp's storage solutions. Additionally, I make an attempt at profiling the system's built-in failure prediction method, and compiling statistics on historical complete-disk failures as well as bad blocks developed. Finally, I experiment with various parameters for producing classification trees and end up with two candidate models which have a true-positive rate of 86% with a false-alarm rate of 4% or atrue-positive rate of 71% and a false-alarm rate of 0.9% respectively, illustrating that classification trees might be a viable method for predicting real-life disk failures in CERNs storage systems.