Agile Views in a Dynamic Data Management System

This paper calls for a new breed of lightweight systems - dynamic data management systems (DDMS). In a nutshell, a DDMS manages large dynamic data structures with agile, frequently fresh views, and provides a facility for monitoring these views and triggering application-level events. We motivate DDMS with applications in large-scale data analytics, database monitoring, and high-frequency algorithmic trading. We compare DDMS to more traditional data management systems architectures. We present the DBToaster project, which is an ongoing effort to develop a prototype DDMS system. We describe its architecture design, techniques for high-frequency incremental view maintenance, storage, scaling up by parallelization, and various key challenges to overcome to make DDMS a reality.


Published in:
CIDR 2011, Fifth Biennial Conference on Innovative Data Systems Research, 284-295
Presented at:
CIDR 2011, Fifth Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 9-12, 2011
Year:
2011
Laboratories:




 Record created 2011-05-17, last modified 2018-03-17

Publisher's version:
Download fulltext
PDF

Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)