Frugal Topology Construction for Stream Aggregation in the Cloud

Aggregation of streamed data is key to the expansion of the Internet of Things. This paper addresses the problem of designing a topology for reliably aggregating data flows from many devices arriving at a datacenter. Reliability here means ensuring operation without data loss. We seek a frugal solution that prevents wasteful resource consumption (over-provisioning). This problem is salient when building an aggregation service out of components (here aggregation nodes) that exhibit hard constraints on the amount of information they can handle per unit of time. We first formalize the problem and provide an analysis of the relation between monitored devices (plus information they send), and the operations performed at aggregation nodes, in terms of data rates. Building on this rate analysis, we devise a novel algorithm, which we call CSA, that basically outputs an aggregation topology capable of handling those incoming data rates, preventing thereby empirical trial-and-error design. We analyze the algorithm, before validating it on the Amazon Kinesis platform, using a device dataset from a European telco operator.

Published in:
Ieee Infocom 2016 - The 35Th Annual Ieee International Conference On Computer Communications
Presented at:
35th Annual IEEE International Conference on Computer Communications, San Francisco, CA, USA
New York, Ieee

Note: The status of this file is: Anyone

 Record created 2016-12-15, last modified 2020-07-29

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)