Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Journal articles
  4. Data-centric workloads with MPI_Sort
 
research article

Data-centric workloads with MPI_Sort

Zulian, P.
•
Ben Bader, S.
•
Fourestey, G.  
Show more
May 1, 2024
Journal of Parallel and Distributed Computing

Sorting is a fundamental task in computing and plays a central role in information technology. The advent of rack-scale and warehouse-size data processing shaped the architecture of data analysis platforms towards supercomputing. In turn, established techniques on supercomputers have become relevant to a wider range of application domains. This work is concerned with multi-way mergesort with exact splitting on distributed memory architectures. At its core, our approach leverages a novel and parallel algorithm for multi-way selection problems. Remarkably concise, the algorithm relies on MPI_Allgather and MPI_ReduceScatter_block, two collective communication schemes that find hardware support in most high-end networks. A software implementation of our approach is used to process the Terabyte-size Data Challenge 2 signal, released by the SKA radio telescopes organization. On the supercomputer considered herein, our approach outperforms the state of the art by up to 2.6X using 9,216 cores. Our implementation is released as a compact open source library compliant to the MPI programming model. By supporting the most popular elementary key types, and arbitrary fixed-size value types, the library can be straightforwardly integrated into third-party MPI-based software.

  • Details
  • Metrics
Type
research article
DOI
10.1016/j.jpdc.2023.104833
Scopus ID

2-s2.0-85182267781

Author(s)
Zulian, P.
Ben Bader, S.
Fourestey, G.  

École Polytechnique Fédérale de Lausanne

Krause, R.
Rossinelli, D.
Date Issued

2024-05-01

Published in
Journal of Parallel and Distributed Computing
Volume

187

Article Number

104833

Subjects

Distributed sorting

•

Parallel algorithms

•

Supercomputers

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
SCITAS-GE  
Available on Infoscience
January 16, 2025
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/242837
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés