Recent advancements in deep learning have found compelling applications in the image-based inspection of infrastructure systems. However, the efficacy of these models hinges significantly on the quality and diversity of the input data used for training. In this study, we rigorously evaluate the performance of a convolutional neural network architecture for image-based crack segmentation and kinematics determination, taking into account both aleatory and epistemic uncertainties. Our experimental setup involves acquiring images of beams through a series of three-point bending tests. We identify aleatory uncertainties such as blur, noise, changes in contrast, and variations in camera positioning as crucial factors affecting the reliability of crack detection. Notably, our investigation reveals that blur and noise exert a substantial influence on the accuracy of crack detection probabilities. Moreover, we demonstrate that augmenting the data set significantly enhances the robustness of crack detection estimations. Furthermore, our analysis underscores the robustness of the model in effectively detecting cracks across diverse camera angles. We employ Monte Carlo dropout to construct probabilistic crack kinematics maps, acknowledging inherent limitations in model architecture and data set diversity. This approach allows us to quantify and visualize epistemic uncertainties associated with crack kinematics estimation.