In this paper we study the behavior of local descriptor object recognition methods with respect to 3D geometric transformations and image resolution variations. As expected performance decreases with accentuated perspective and decrease in resolution. To improve performance and robustness, we propose a scheme to fuse color and gradient local descriptors. This approach is motivated by the discriminative power of color in man-made object recognition. The problem of color feature extraction is addressed as well as the considerations on the fusion process and steps to train such fusion. We used SOIL-47A database for experiments and shown a 7\% to 10\% relative improvement when compared with state-of-the-art gradient based descriptors.