Essays about: "Kafka"

Showing result 1 - 5 of 13 essays containing the word Kafka.

  1. 1. Hudi on Hops : Incremental Processing and Fast Data Ingestion for Hops

    University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

    Author : Netsanet Gebretsadkan Kidane; [2019]
    Keywords : Hudi; Hadoop; Hops; Upsert; SQL; Spark; Kafka; Hudi; Hadoop; Hops; Upsert; SQL; Spark; Kafka;

    Abstract : In the era of big data, data is flooding from numerous data sources and many companies have been utilizing different types of tools to load and process data from various sources in a data lake. The major challenges where different companies are facing these days are how to update data into an existing dataset without having to read the entire dataset and overwriting it to accommodate the changes which have a negative impact on the performance. READ MORE

  2. 2. Intelligent Resource Management for Large-scale Data Stream Processing

    University essay from Uppsala universitet/Institutionen för informationsteknologi

    Author : Oliver Stein; [2019]
    Keywords : ;

    Abstract : With the increasing trend of using cloud computing resources, the efficient utilization of these resources becomes more and more important. Working with data stream processing is a paradigm gaining in popularity, with tools such as Apache Spark Streaming or Kafka widely available, and companies are shifting towards real-time monitoring of data such as sensor networks, financial data or anomaly detection. READ MORE

  3. 3. Geo-distributed multi-layer stream aggregation

    University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

    Author : Pietro Cannalire; [2018]
    Keywords : stream processing; geo-distributed; architecture; algorithms; windowing; data synopses; Apache Spark Structured Streaming; Apache Kafka; Misra-Gries algorithm; flödesbehandling; geo-distribuerade; arkitekturen; algoritmerna; windowing; data synopses; Apache Spark Structured Streaming; Apache Kafka; Misra-Gries-algoritmen;

    Abstract : The standard processing architectures are enough to satisfy a lot of applications by employing already existing stream processing frameworks which are able to manage distributed data processing. In some specific cases, having geographically distributed data sources requires to distribute even more the processing over a large area by employing a geographically distributed architecture. READ MORE

  4. 4. Collecting Information from a decentralized microservice architecture

    University essay from Linköpings universitet/Institutionen för datavetenskap; Linköpings universitet/Institutionen för datavetenskap

    Author : Carl Ekbjörn; Daniel Sonesson; [2018]
    Keywords : Message-oriented middleware; Apache kafka; RabbitMQ; microservices; benchmark;

    Abstract : As a system grows in size, it is common that it is transformed into a microservice architecture. In order to be able monitor this new architecture there is a need to collect information from the microservices. The software company IDA Infront is transitioning their product iipax to a microservice architecture and is faced with this problem. READ MORE

  5. 5. Lifecycle management and smart manufacturing: Modelling and implementation to utilize the digital twin

    University essay from KTH/Industriell produktion; KTH/Industriell produktion

    Author : Chengxue Huang; Hampus Wranér; [2018]
    Keywords : PLCS; Smart manufacturing; Digital Twin; Standards; PLCS; Smart tillverkning; Digital tvilling; Standarder;

    Abstract : Smart manufacturing – smart factories creating smart products – is a topic which has arisen in the academic as well as business community. This thesis covers smart manufacturing in the context of lifecycle management. READ MORE