Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. The IceProd framework: Distributed data processing for the IceCube neutrino observatory
 
research article

The IceProd framework: Distributed data processing for the IceCube neutrino observatory

Aartsen, M. G.
•
Abbasi, R.
•
Ackermann, M.
Show more
2015
Journal of Parallel and Distributed Computing

IceCube is a one-gigaton instrument located at the geographic South Pole, designed to detect cosmic neutrinos, identify the particle nature of dark matter, and study high-energy neutrinos themselves. Simulation of the IceCube detector and processing of data require a significant amount of computational resources. This paper presents the first detailed description of IceProd, a lightweight distributed management system designed to meet these requirements. It is driven by a central database in order to manage mass production of simulations and analysis of data produced by the IceCube detector. IceProd runs as a separate layer on top of other middleware and can take advantage of a variety of computing resources, including grids and batch systems such as CREAM, HTCondor, and PBS. This is accomplished by a set of dedicated daemons that process job submission in a coordinated fashion through the use of middleware plugins that serve to abstract the details of job submission and job management from the framework. (C) 2014 Elsevier Inc. All rights reserved.

  • Details
  • Metrics
Type
research article
DOI
10.1016/j.jpdc.2014.08.001
Web of Science ID

WOS:000346552500017

Author(s)
Aartsen, M. G.
Abbasi, R.
Ackermann, M.
Adams, J.
Aguilar, J. A.
Ahlers, M.
Altmann, D.
Arguelles, C.
Auffenberg, J.
Bai, X.
Show more
Date Issued

2015

Publisher

Academic Press Inc Elsevier Science

Published in
Journal of Parallel and Distributed Computing
Volume

75

Start page

198

End page

211

Subjects

Data management

•

Grid computing

•

Monitoring

•

Distributed computing

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
LPHE  
Available on Infoscience
February 20, 2015
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/111354
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés