Determining the size and shape of coarse sediment is of paramount importance to many applications (e.g. sediment transport, flow resistance in numerical hydraulic models, estimation of current velocity and direction, habitat classification). One of the current challenges is that to reach statistical significance, one needs to collect large amounts of sediment: the larger the clast, the heavier the mass required to be sampled. Image-based methods have been developed to reduce the sampling cost since the 1970s, but they require significant expertise and processing time. In order to overcome these constrains, we propose a new method based on machine learning (more specifically on the Mask R-CNN instance segmentation algorithm). After training on manually labeled data, the model is able to perform non-supervised detection, classification, and contouring of non-overlapping clasts visible on images, and eventually quantify their size, shape, position and orientation. Thanks to fast and accurate detection, the model performs well with terrestrial images or large high-resolution ortho-mosaics from drone imagery, and thus the ways the morphometric features of coarsest clasts vary spatially and temporally can be evaluated efficiently at low cost. After demonstrating effectiveness on two pebble beaches in Normandy, France, we applied the model to four sites abroad (Italy, Ireland, Switzerland and USA) exhibiting a wider variety of conditions in terms of clast sizes and shapes. Preliminary results show that the model is able to detect clasts on these sites with the same efficiency as in Normandy. The comparison shows how good the model is at determining the sediment’s spatial heterogeneity and its temporal changes. The model has been made available from a public repository. In the future, the tool could be improved in order to provide further information about the most probable geological composition of each clast, or the distribution of fresh and water-worked elements.