We present a new hybrid algorithm for local search in distributed combinatorial optimization. This method is a mix between classical local search methods in which nodes take decisions based only on local information, and full inference methods that guarantee completeness. In general, classical inference methods are time and space exponential in a parameter of the graph called the induced width, thus they may be infeasible for dense problems. On the other hand, local search methods suffer from the fact that the nodes take myopic decisions based only on local information available to them, and thus can easily get stuck in local optima. Their advantage is that they require linear memory, and in many cases provide good solutions with a small amount of effort. We propose a hybrid method that combines the advantages of both these approaches. This method is a utility propagation algorithm controlled by a parameter maxDim which specifies the maximal allowable amount of inference. The maximal space requirements are exponential in this parameter. In the dense parts of the problem, where the required amount of inference exceeds this limit, the algorithm executes a local search procedure guided by as much inference as allowed by maxDim. This hybrid can be seen as a large neighborhood search, where exponential neighborhoods are rigorously determined according to problem structure, and polynomial efforts are spent for their complete exploration at each local search step. The algorithm explores independent parts of the problem simultaneously and asynchronously, and then combines the results, all in a distributed fashion. We show the efficiency of this approach with experimental results from the distributed meeting scheduling domain.