Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. NoDB: Efficient Query Execution on Raw Data Files
 
conference paper

NoDB: Efficient Query Execution on Raw Data Files

Alagiannis, Ioannis  
•
Borovica, Renata  
•
Branco, Miguel
Show more
2012
Proceedings of the ACM SIGMOD International Conference on Management of Data
ACM SIGMOD International Conference on Management of Data

As data collections become larger and larger, data loading evolves to a major bottleneck. Many applications already avoid using database systems, e.g., scientific data analysis and social networks, due to the complexity and the increased data-to-query time. For such applications data collections keep growing fast, even on a daily basis, and we are already in the era of data deluge where we have much more data than what we can move, store, let alone analyze. Our contribution in this paper is the design and roadmap of a new paradigm in database systems, called NoDB, which do not require data loading while still maintaining the whole feature set of a modern database system. In particular, we show how to make raw data files a first-class citizen, fully integrated with the query engine. Through our design and lessons learned by implementing the NoDB philosophy over a modern DBMS, we discuss the fundamental limitations as well as the strong opportunities that such a research path brings. We identify performance bottlenecks specific for in situ processing, namely the repeated parsing and tokenizing overhead and the expensive data type conversion costs. To address these problems, we introduce an adaptive indexing mechanism that maintains positional information to provide efficient access to raw data files, together with a flexible caching structure. Our implementation over PostgreSQL, called PostgresRaw, is able to avoid the loading cost completely, while matching the query performance of plain PostgreSQL and even outperforming it in many cases. We conclude that NoDB systems are feasible to design and implement over modern database architectures, bringing an unprecedented positive effect in usability and performance.

  • Files
  • Details
  • Metrics
Type
conference paper
DOI
10.1145/2213836.2213864
Author(s)
Alagiannis, Ioannis  
Borovica, Renata  
Branco, Miguel
Idreos, Stratos
Ailamaki, Anastasia  
Date Issued

2012

Published in
Proceedings of the ACM SIGMOD International Conference on Management of Data
Start page

241

End page

252

Subjects

Adaptive loading

•

In situ querying

•

Positional map

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
DIAS  
Event nameEvent placeEvent date
ACM SIGMOD International Conference on Management of Data

Scottsdale, Arizona, USA

May 20–24, 2012

Available on Infoscience
March 20, 2012
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/78876
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés