Comprehensive Analysis of Performance Metric on In-Order Data Stream Process for Sliding Window Aggregation on Business Intelligence

Abstract

A data stream process is usually referred as a event stream process which uses data streaming platform to automatically integrate data from various sources, manage, organize and act upon the data on the fly as it is generated. As the world undergoes digital transformation, organizations leverage data streaming platforms to create new business opportunities. The platforms help to strengthen the competitive advantage and make the organizations’ existing operation more efficient. In real-time applications, data is processed with an unbounded data stream which provides immediate insights for making any decision in analytical applications like marketing, finance, sales and more. Windowing is one of the most efficient processing methods to process data streams where unbounded stream of data or event is split into finite sets or windows based on specific criteria such as time, count of data elements or a condition. For making analytical decisions in real-time, it is a great challenge to handle data streams with efficient performance in an accurate manner. In this work, Sliding Window Aggregation (SWAG) algorithms are analyzed with synthetic datasets experimentally for a data stream process to measure the performance based on throughput and latency. Two-Stacks, Two- Stacks Lite, DABA, and DABA Lite algorithms are analyzed in this work on the synthetic dataset. Among these algorithms, DABA Lite performs well in terms of throughput and latency.