Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Efficient Indexing and Query Processing of Model-View Sensor Data in the Cloud
 
research article

Efficient Indexing and Query Processing of Model-View Sensor Data in the Cloud

Guo, Tian  
•
G.Papaioannou, Thanasis
•
Aberer, Karl  
2014
Journal of Big Data Research

As the number of sensors that pervade our lives increases (e.g., environmental sensors, phone sensors, etc.), the efficient management of massive amount of sensor data is becoming increasingly important. The infinite nature of sensor data poses a serious challenge for query processing even in a cloud infrastructure. Traditional raw sensor data management systems based on relational databases lack scalability to accommodate large-scale sensor data efficiently. Thus, distributed key-value stores in the cloud are becoming a prime tool to manage sensor data. Model-view sensor data management, which stores the sensor data in the form of modeled segments, brings the additional advantages of data compression and value interpolation. However, currently there are no techniques for indexing and/or query optimization of the model-view sensor data in the cloud; full table scan is needed for query processing in the worst case. In this paper, we propose an innovative index for modeled segments in key-value stores, namely KVI-index. KVI-index consists of two interval indices on the time and sensor value dimensions respectively, each of which has an in-memory search tree and a secondary list materialized in the key-value store. Then, we introduce a KVI-index–Scan–MapReduce hybrid approach to perform efficient query processing upon modeled data streams. As proved by a series of experiments at a private cloud infrastructure, our approach outperforms in query-response time and index-updating efficiency both Hadoop-based parallel processing of the raw sensor data and multiple alternative indexing approaches of model-view data.

  • Files
  • Details
  • Metrics
Type
research article
DOI
10.1016/j.bdr.2014.07.005
Author(s)
Guo, Tian  
G.Papaioannou, Thanasis
Aberer, Karl  
Date Issued

2014

Published in
Journal of Big Data Research
Volume

1

Start page

52

End page

65

Subjects

Big data

•

Index

•

Key-value stores

•

MapReduce

•

Approximation Query optimization

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
LSIR  
Available on Infoscience
March 13, 2015
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/112425
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés