Coconut tree plantations are one of the main sources of income in several South Pacific countries. Thus, keeping track of the location of coconut trees is important for monitoring and post-disaster assessment. Although deep learning based object detectors can attain considerably accurate results, it is inevitable that errors will remain in the predictions obtained for a large test set. Since every mistake counts, in this work we propose a methodology to efficiently use the time of human annotators to find and correct a large part of erroneous coconut tree detections. We propose to use a Random forest classifer that finds detection errors to sort the regions of the image (tiles) in decreasing order of likeliness to have detection errors. In our experiments involving UAV images in the Kingdom of Tonga, the user could analyze only 24% of the tiles and correct approximatively 71% of the errors thanks to the sorting.