Granular hydrogels are composed of hydrogel-based microparticles, so-called microgels, that are densely packed to form an ink that can be 3D printed, injected or cast into macroscopic structures. They are frequently used as tissue engineering scaffolds because microgels can be made biocompatible and the porosity of the granular hydrogels enables a fast exchange of reagents, waste products, and if properly designed even the infiltration of cells. Most of these granular hydrogels can be shaped into appropriate macroscopic structures, yet, these structures are mechanically rather weak. The poor mechanical properties prevent the use of these structures as load-bearing materials and hence, limit their field of applications. The mechanical properties of granular hydrogels depend on the composition of microgels and the interparticle interactions. In this review, we discuss different strategies to assemble microparticles into granular hydrogels and highlight the influence of inter-particle connections on the stiffness and toughness of the resulting materials. Mechanically strong and tough granular hydrogels have the potential to open up new fields of their use and thereby to contribute to fast advances in these fields. In particular, we envisage them to be well-suited as soft actuators and robots, tissue replacements, and adaptive sensors.