We study the Hamiltonian formulation of the general first order action of general relativity compatible with local Lorentz invariance and background independence. The most general simplectic structure (compatible with diffeomorphism invariance and local Lorentz transformations) is obtained by adding to the Holst action the Pontriagin, Euler and Nieh-Yan invariants with independent coupling constants. We perform a detailed canonical analysis of this general formulation (in the time gauge) exploring the structure of the phase space in terms of connection variables. We explain the relationship of these topological terms, and the effect of large SU(2) gauge transformations in quantum theories of gravity defined in terms of the Ashtekar-Barbero connection.