We recently found only weak correlations between the susceptibility to various visual illusions. However, we observed strong correlations among different variants of an illusion, suggesting that the visual space of illusions includes several illusion-specific factors. Here, we specifically examined how factors for the vertical–horizontal, Müller–Lyer, and Ponzo illusions relate to each other. We measured the susceptibility to each illusion separately and to combinations of two illusions, which we refer to as a merged illusion; for example, we tested the Müller–Lyer illusion and the vertical–horizontal illusion, as well as a merged version of both illusions. We used an adjustment procedure in two experiments with 306 and 98 participants, respectively. Using path analyses, correlations, and exploratory factor analyses, we found that the susceptibility to a merged illusion is well predicted from the susceptibilities to the individual illusions. We suggest that there are illusion-specific factors that, by independent combinations, represent the whole visual structure underlying illusions.