Using density-functional theory, we investigate the structural, vibrational, and dielectric properties of Hf and Zr oxides and silicates which have drawn considerable attention as alternative high-kappa materials. For the silicates, we consider hafnon HfSiO4 and zircon ZrSiO4; while for the oxides, we study the cubic and tetragonal phases of HfO2 and ZrO2. Special emphasis is put on the analysis of the differences and similarities between Hf and Zr in these materials. In particular, we discuss the Born effective charge tensors, the phonon frequencies at the Gamma point of the Brillouin zone, and the dielectric permittivity tensors. Our study reveals very similar properties of Hf and Zr compounds, which are essentially related to the chemical homology of Hf and Zr.