Regulating contact forces is crucial for robotic manipulation, but achieving stiffness variation in multi-fingered hands is challenging due to design complexity. This work presents Mod-VSA, a modular variable stiffness actuator for multi-fingered tendon-driven hands. Mod-VSA consists of concatenated units with a novel stiffness modulation mechanism, allowing a single actuator to adjust stiffness across all units mechanically. The mechanism which can vary the stiffness by a factor of ten, is experimentally characterized with a static force model, and each module includes a Hall sensor for tendon force and displacement measurements. To demonstrate its use-case, the Mod-VSA is combined with the ADAPT Hand, an anthropomorphic multi-fingered hand, to perform finger-level force control and grasping tasks which require both low and high stiffness interactions. Through this VSA design, online force regulation capabilities for a multi-fingered hands can be achieved, expanding their possible applications.