We present a pipeline for the automatic generation of digital twins of free-standing buildings for damage assessment. Our method takes multiple view images of the target building as input and processes them to produce a 3D model in levels of detail (LOD) format containing damage information. The proposed methodology combines photogrammetry, specifically structure from motion (SfM), and deep-learning-based segmentation techniques. First, we run a SfM framework with the building images to generate sparse point clouds and camera poses. The point cloud is post-processed to extract planar primitives, which are then used to create a watertight polygonal surface model. This model is augmented by projecting opening (e.g., doors, windows) and damage information segmented from images using the camera poses recovered by SfM. We hope to contribute to research in construction management and structural health monitoring by providing a simple yet efficient and reliable method of documenting building and damage assessment information.