Computing averages over a target probability density by statistical re-weighting of a set of samples with a different distribution is a strategy which is commonly adopted in fields as diverse as atomistic simulation and finance. Here we present a very general analysis of the accuracy and efficiency of this approach, highlighting some of its weaknesses. We then give an example of how our results can be used, specifically to assess the feasibility of high-order path integral methods. We demonstrate that the most promising of these techniques-which is based on re-weighted sampling-is bound to fail as the size of the system is increased, because of the exponential growth of the statistical uncertainty in the re-weighted average. © 2011 The Royal Society.