The six complementarity determining regions (CDRs) of the T cell receptor (TCR) form multiple contacts with cognate peptide and major histocompatibility complex, thus determining antigen specificity. However, the contacts between the CDRs themselves are less understood. We perform a systematic study of all available TCR structures, and identify consistent patterns of intra- and inter-chain CDR contacts. We further show that the sequences of paired TCRαand TCRβare not independent within sets of antigen-specific TCRs, for most epitopes. We quantify this sequence restriction using a mutual information framework. Co-evolution models can achieve somede novoprediction of TCRα/TCRβpairing, without using a training set of known pairs. The conserved pattern of CDR amino acid contacts, and the mutual sequence constraints between antigen-specific sets of T cell receptorαandβchains could play an important role in shaping the antigen-specific T cell repertoire.