Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Spinner: Scalable Graph Partitioning in the Cloud
 
conference paper

Spinner: Scalable Graph Partitioning in the Cloud

Martella, Claudio
•
Logothetis, Dionysios
•
Loukas, Andreas  
Show more
2017
2017 Ieee 33Rd International Conference On Data Engineering (Icde 2017)
IEEE 33rd International Conference on Data Engineering (ICDE)

In this paper, we present a graph partitioning algorithm to partition graphs with trillions of edges. To achieve such scale, our solution leverages the vertex-centric Pregel abstraction provided by Giraph, a system for large-scale graph analytics. We designed our algorithm to compute partitions with high locality and fair balance, and focused on the characteristics necessary to reach wide adoption by practitioners in production. Our solution can (i) scale to massive graphs and thousands of compute cores, (ii) efficiently adapt partitions to changes to graphs and compute environments, and (iii) seamlessly integrate in existing systems without additional infrastructure. We evaluate our solution on the Facebook and Instagram graphs, as well as on other large-scale, real-world graphs. We show that it is scalable and computes partitionings with quality comparable, and sometimes outperforming, existing solutions. By integrating the computed partitionings in Giraph, we speedup various real-world applications by up to a factor of 5.6 compared to default hash-partitioning.

  • Details
  • Metrics
Type
conference paper
DOI
10.1109/Icde.2017.153
Web of Science ID

WOS:000403398200146

Author(s)
Martella, Claudio
Logothetis, Dionysios
Loukas, Andreas  
Siganos, Georgos
Date Issued

2017

Publisher

Ieee

Publisher place

New York

Published in
2017 Ieee 33Rd International Conference On Data Engineering (Icde 2017)
ISBN of the book

978-1-5090-6543-1

Total of pages

12

Series title/Series vol.

IEEE International Conference on Data Engineering

Start page

1083

End page

1094

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
IEL  
Event nameEvent placeEvent date
IEEE 33rd International Conference on Data Engineering (ICDE)

San Diego, CA

APR 19-22, 2017

Available on Infoscience
July 10, 2017
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/139182
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés