Molecules with tailored binding specificities are needed for many purposes such as the development of therapeutics, the detection and purification of biomolecules or in chemical biology for the study and manipulation of biological systems. With phage display technology, polypeptides with binding affinities to targets of interest can be isolated from billions of polypeptide variants with a modest amount of effort, time and cost. The technology was initially used for the generation and screening of peptide and antibody libraries and was later applied to many different protein scaffolds. More recently, chemically and structurally diverse molecule libraries were generated by chemically modifying phage-displayed polypeptides. In this article, the different classes of natural and non-natural structures that can be encoded and screened by phage display are reviewed with a special focus on bicyclic peptides that we routinely generate in our laboratory.