Many technologically important material properties are underpinned by disorder and short-range structural correlations; therefore, elucidating structure-property relationships in functional materials requires understanding both the average and the local structures. The latter information is contained within diffuse scattering but is challenging to exploit, particularly in single-crystal systems. Separation of the diffuse scattering into its constituent components can greatly simplify analysis and allows for quantitative parameters describing the disorder to be extracted directly. Here, a deep-learning method, DSFU-Net, is presented based on the Pix2Pix generative adversarial network, which takes a plane of diffuse scattering as input and factorizes it into the contributions from the molecular form factor and the chemical short-range order. DSFU-Net was trained on 198 421 samples of simulated diffuse scattering data and performed extremely well on the unseen simulated validation dataset in this work. On a real experimental example, DSFU-Net successfully reproduced the two components with a quality sufficient to distinguish between similar structural models based on the form factor and to refine short-range-order parameters, achieving values comparable to other established methods. This new approach could streamline the analysis of diffuse scattering as it requires minimal prior knowledge of the system, allows access to both components in seconds and is able to compensate for small regions with missing data. DSFU-Net is freely available for use and represents a first step towards an automated workflow for the analysis of single-crystal diffuse scattering.