With the popularization of online social networks (OSNs) and smart mobile devices, photo sharing is becoming a part of people’ daily life. An unprecedented number of photos are being uploaded and shared everyday through online social networks or photo hosting services, such as Facebook, Twitter, Instagram, and Flickr. However, such unrestrained online photo or multimedia sharing has raised serious privacy concerns, especially after reports of citizens surveillance by governmental agencies and scandalous leakage of private photos from prominent photo sharing sites or online cloud services. Popular OSNs typically offer privacy protection solutions only in response to the public demand and therefore are often rudimental, complex to use, and provide limited degree of control and protection. Most solutions allow users to control either who can access the shared photos or for how long they can be accessed. In contrast, in this paper, we take a structured privacy by design approach to the problem of online photo privacy protection. We propose a privacy-preserving photo sharing architecture based on a secure JPEG scrambling algorithm capable of protecting the privacy of multiple users involved in a photo. We demonstrate the proposed photo sharing architecture with a prototype application called ProShare that offers JPEG scrambling as the privacy protection tool for selected regions in a photo, secure access to the protected images, and secure photo sharing on Facebook.