This paper presents STEEL-3dPointClouds, a dataset of deformed steel beam-columns obtained using high-fidelity physics-based numerical simulations. These simulations trace the inelastic deformations of hot-rolled wide-flange steel beam-columns under different loading protocols covering a range of responses, starting with no strength loss and up to at least 60% loss of load-bearing capacity for each considered steel member. Each of the ~ 323k samples is a unique point extracted from the hysteretic response of the loaded member and consists of the deformed shape (represented as a 3D point cloud) along with the corresponding reserve capacity and stress/strain fields. To exemplify the use of this dataset, machine learning models are implemented to quantify the reserve capacity of deformed steel members solely using point clouds as inputs and to estimate their key deformation characteristics based on geometric properties. Furthermore, the dataset is used to extract deformations at critical response stages to characterize the geometric tolerances for potential member reuse. Dataset is shared to facilitate development of automated inspection methodologies and benchmarking of computer vision tools.