Evidence from activation studies suggests that sound recognition and localization are processed in two distinct cortical networks that are each present in both hemispheres. Sound recognition and/or localization may, however, be disrupted by purely unilateral damage, suggesting that processing within one hemisphere may not be sufficient or may be disturbed by the contralateral lesion. Sound recognition and localization were investigated psychophysically and using fMRI in patients with unilateral right hemisphere lesions. Two patients had a combined deficit in sound recognition and sound localization, two a selective deficit in sound localization, one a selective deficit in sound recognition, and two normal performance in both tasks. The overall level of activation in the intact left hemisphere of the patients was smaller than in normal control subjects, irrespective of whether the patient’s performance in the psychophysical tasks was impaired. Despite this overall decrease in activation strength, patients with normal performance still exhibited activation patterns similar to those of the control subjects in the recognition and localization tasks, indicating that the specialized brain networks subserving sound recognition and sound localization in normal subjects were also activated in the patients with normal performance, albeit to an altogether lesser degree. In patients with deficient performance, on the other hand, the activation patterns during the sound recognition and localization tasks were severely reduced, comprising fewer and partly atypical activation foci compared to the normal subjects. This indicates that impaired psychophysical performance correlates with a breakdown of parallel processing within specialized networks in the contralesional hemisphere.