With the aim of understanding the effect of the environment on the star formation history and morphological transformation of galaxies, we present a detailed analysis of the colour, morphology and internal structure of cluster and field galaxies at 0.4 <= z <= 0.8. We use the Hubble Space Telescope data for over 500 galaxies from the ESO Distant Cluster Survey to quantify how the galaxies' light distribution deviate from symmetric smooth profiles. We visually inspect the galaxies' images to identify the likely causes for such deviations. We find that the residual flux fraction (RFF), which measures the fractional contribution to the galaxy light of the residuals left after subtracting a symmetric and smooth model, is very sensitive to the degree of structural disturbance but not the causes of such disturbance. On the other hand, the asymmetry of these residuals (A(res)) is more sensitive to the causes of the disturbance, with merging galaxies having the highest values of A(res). Using these quantitative parameters, we find that, at a fixed morphology, cluster and field galaxies show statistically similar degrees of disturbance. However, there is a higher fraction of symmetric and passive spirals in the cluster than in the field. These galaxies have smoother light distributions than their star-forming counterparts. We also find that while almost all field and cluster S0s appear undisturbed, there is a relatively small population of star-forming S0s in clusters but not in the field. These findings are consistent with relatively gentle environmental processes acting on galaxies infalling on to clusters.