We investigate the environment and line of sight (LoS) of the H0LiCOW (H-0 Lenses in COSMOGRAIL's Wellspring) lens B1608+656 using Subaru Suprime-Cam and the Hubble Space Telescope (HST) to perform a weak lensing analysis. We compare three different methods to reconstruct the mass map of the field, i.e. the standard Kaiser-Squires inversion coupled with inpainting and Gaussian or wavelet filtering, and GLIMPSE, a method based on sparse regularization of the shear field. We find no substantial difference between the 2D mass reconstructions, but we find that the ground-based data are less sensitive to small-scale structures than the space-based observations. Marginalizing over the results obtained with all the reconstruction techniques applied to the two available HST filters F606W and F814W, we estimate the external convergence, kappa(ext), at the position of B1608+656 is kappa(ext) = 0.11(-0.04)(+0.06), where the error bars correspond, respectively, to the 16th and 84th quartiles. This result is compatible with previous estimates using the number counts technique, suggesting that B1608+656 resides in an overdense LoS, but with a completely different technique. Using our mass reconstructions, we also compare the convergence at the position of several groups of galaxies in the field of B1608+656 with the mass measurements using various analytical mass profiles, and find that the weak lensing results favour truncated halo models.