Current post-earthquake damage assessment methodologies are not only time-consuming but also subjective in nature and difficult to document. Recent advancements in artificial intelligence and technological devices make it possible to accomplish this task automatically, efficiently, and objectively. Our vision for an automated post-earthquake evaluation begins with image data, such as that obtained by an Unmanned Aerial Vehicle, which is then processed to detect damage and generate a Finite Element Method (FEM) model. This thesis aims to realize this vision for free-standing stone masonry buildings. The main objective of the current research is to propose robust and computationally efficient methodologies to automatically generate 3D models for free-standing stone masonry buildings and provide information on damage detected in RGB images. This allows for an effective and more objective post-earthquake damage assessment with straightforward documentation, allowing future correlation of damage information with the mechanical properties of the model. RGB images were used for two purposes, i.e., 3D model generation and damage detection. Related to 3D models, an image-based pipeline was developed to automatically create level of detail (LOD) models, specifically LOD3, using structure-from-motion and semantic segmentation, in order to produce a geometrical representation of a building. In contrast to the existing works, the method does not rely on post-processing of extremely precise 3D models, does not use predefined templates, does not require human manipulation, and provides semantic understanding of the final model's components. Cracks were detected using state-of-the-art deep learning approaches, which were complemented with a TOPO-Loss function that does not require pixel-precise labels and emphasizes the continuity of the crack topology. When assessing the mechanical effect of a crack, not only the crack geometry but also the crack opening in Mode I and II are important input parameters. While this information can be readily obtained in laboratory settings using digital image correlation measurements, such techniques cannot be applied in real settings when reference images before the damage occurred do not exist. It was therefore developed the first approach for estimating these quantities based on RGB images of the damaged structural component only. The crack opening is determined in Mode I and Mode II from images by formulating the problem as a registration problem of the two crack edges retrieved from a binary mask that represents the crack's semantic segmentation. Damage augmented digital twins were proposed, which are digital representations of real assets that include damage information related to damage and its characterization in addition to geometrical information. In order to contribute to numerical modeling, an image-based pipeline that enables the automatic generation of the FEM geometry using two approaches was proposed: 3D solid elements and macro elements. Finally, it is presented an unique framework for automatically generating geometrical digital twins of stone masonry elements with detail down to the stone level intended to be used in numerical simulations of stone masonry elements, specially those that are built to be tested in laboratory. We expect that the research presented in this doctoral thesis will enable, in the future, automatic post-earthquake assessment of stone masonry buildings.