Photoswitchable ligands are powerful tools to control biological processes at high spatial and temporal resolution. Unfortunately, such ligands exist only for a limited number of proteins and their development by rational design is not trivial. We have developed an in vitro evolution strategy to generate light-activatable peptide ligands to targets of choice. In brief, random peptides were encoded by phage display, chemically cyclized with an azobenzene linker, exposed to UV light to switch the azobenzene into cis conformation, and panned against the model target streptavidin. Isolated peptides shared strong consensus sequences, indicating target-specific binding. Several peptides bound with high affinity when cyclized with the azobenzene linker, and their affinity could be modulated by UV light. The presented method is robust and can be applied for the in vitro evolution of photoswitchable ligands to virtually any target.