A search in an all-jet final state for new massive resonances decaying to WW, WZ, or ZZ boson pairs using a novel analysis method is presented. The analysis is performed on data corresponding to an integrated luminosity of 77.3 fb$^{-1}$ recorded with the CMS experiment at the LHC at a centre-of-mass energy of 13 TeV. The search is focussed on potential resonances with masses above 1.2 TeV, where the decay products of each W or Z boson are expected to be collimated into a single, large-radius jet. The signal is extracted using a three-dimensional maximum likelihood fit of the two jet masses and the dijet invariant mass, yielding an improvement in sensitivity of up to 30% relative to previous search methods. No excess is observed above the estimated standard model background. In a heavy vector triplet model, spin-1 Z' and W' resonances with masses below 3.5 and 3.8 TeV, respectively, are excluded at 95% confidence level. In a narrow-width bulk graviton model, upper limits on cross sections are set between 27 and 0.2 fb for resonance masses between 1.2 and 5.2 TeV, respectively. The limits presented in this paper are the best to date in the dijet final state.