We introduce an approach to learning convolution filters whose joint output can be fed to a classifier that labels them as belonging to linear structures or not. The filters are learned using sparse synthesis techniques but we show that enforcing constraints is not required at run-time to achieve good classification performance. In practice, this is important as it drastically reduces the computational cost. We show that our approach outperforms the state-of-the-art on difficult, and very different, images of roads, retinal scans, and dendritic networks.