The first amazing effect with #TheDress is that a large number of people perceive it as white and gold even though the true colors of the dress are black and blue. Such apparent changes in color are common in many illusions. What makes #TheDress even more interesting is that there is a bimodal split of the population in the perception of the dress’s colors (white-gold vs. black-blue), contrary to most color illusions where almost all observers perceive the colors in the same illusory way. On top of this, the percept rarely switches in a given individual. What causes this phenomenon? Here, we tested the role of one-shot learning during the first presentation of the image. By hiding large parts of the image, we were able to influence the perceived colors in naïve observers who had never seen #TheDress before. When the image was covered by white occluders, the majority of participants perceived the dress as black and blue. With black occluders, the majority of observers perceived the dress as white and gold. Importantly, the percept did not change when we subsequently presented the full image, arguing for a crucial role of one-shot learning in #TheDress. Next, we tested whether there are differences in the eye movement patterns of whitegold vs. black-blue perceivers. We did not find any obvious pattern. Hence, our results show that one-shot learning during the first percept, but not the first fixation or the subsequent eye movements, plays a crucial role for color perception in #TheDress.