Scaling out parallel data stream access to a relational database

Abstract: The rise of new applications requiring processing high volumes of continuous and real time data has created the demand for data stream management systems (DSMSs). Applications  using DSMSs often need to access historical data saved on disk to analyze, mine, and process streaming data. The historical data is persistent and often is very voluminous and is therefore  usually stored in a relational database. If this relational database is continuously accessed from the DSMS it will become a bottleneck of the system. The linear road benchmark (LRB) is a simulated data stream benchmark, which includes access to historical data stored on disk. If the query load per unit of time becomes too large the relational database access will become a bottleneck for the DSMS processing. In this thesis we scale out a relational DBMS storing the historical LRB data in order to eliminate this bottleneck. Experiments were done on LRB with access to a scaled out MySQL databases running on a single NUMA machine with 64 cores. Scaling out the relational database access is shown to make it possible to run LRB with L rating of 512 and beyond for only daily expenditure queries and L rating of 150 for all the LRB implemented in the DSMS SCSQ.