Data Retrieval Strategy for Modern Database Models in a Serverless Architecture

Abstract: The arising presence of social media platforms shapes modern system architectures to handle performance and scale. As user volumes increase, the need to keep data consistent and available at storage locations across the world adds complexity in distributed systems. The startup Leader Island has developed a communication platform for companies and organizations, using Amazon Web Services (AWS) as cloud provider to overcome operational concerns. Within their platform, users share various content and interact with each other. Data retrieval is an essential component in the platform, as the users should get various feeds and be able to search for specific content. For this functionality, Leader Island uses a combination of AWS Elasticsearch Service for data retrieval and Amazon DynamoDB for persistent storage. However, this setup has posed challenges within data models, mappings, and retrieval strategies. This project aims to find best practices for data models and mappings within both instances and to investigate in new data retrieval strategies to optimize the latencies of data retrieval. For this, three designs were configured with op- posing models and mappings. The results of each design were collected and measured against each other. Mainly, a design where data retrieval was distributed over both Elasticsearch and DynamoDB, incorporating DynamoDB best practices and a reduced data volume propagated to Elasticsearch, out-performed the initial platform design by a factor of 1.9 concerning platform latencies.