One of the key properties of lead-halide perovskites employed in solar cells is the defect tolerance of the materials, in particular regarding intrinsic point defects, which mainly form shallow traps. Considering that high luminescence yields and photovoltaic performance are obtained by simple solution processing from commercial chemicals, it is commonly anticipated that the defect tolerance - at least to a considerable degree - extends to grain boundaries and extrinsic defects, i.e. impurities, as well. However, the effect of impurities has hardly been investigated. Here, we intentionally introduce small quantities of bismuth (10 ppm to 2%) in solution to be incorporated in the perovskite films based on mixed cation mixed anion compositions. We observe that Bi impurities in the %-regime reduce charge carrier collection efficiency and, more importantly, that the open-circuit voltage decreases systematically with impurity concentration even in the ppm regime. This strong defect intolerance against Bi impurities comes along with reduced electroluminescence yields and charge carrier lifetimes obtained from transient photoluminescence experiments. Calculations based on molecular dynamics and density functional theory predict delocalized (approximate to 0.16 eV) and localized deep (approximate to 0.51 eV) trap states dependent on the structural arrangement of the surrounding atoms. Structural characterization supports the idea of Bi being present as a homogeneously spread point defect, which substitutes the Pb2+ by Bi3+ as seen from XPS and a reduction of the lattice parameter in XRD. Sensitive measurements of the photocurrent (by FTPS) and surface photovoltage (SPV) confirm the presence of tail states. Photoelectron spectroscopy measurements show evidence of a deep state. These results are consistent with the common idea of shallow traps being responsible for the reduced charge collection efficiency and the decreased fill factor, and deeper traps causing a substantial reduction of the open-circuit voltage. As Bi is only one potential impurity in the precursor salts used in perovskite solar cell fabrication, our findings open-up a research direction focusing on identifying and eliminating impurities that act as recombination centers - a topic that has so far not been fully considered in device optimization studies.