Spin qubits hosted in silicon (Si) quantum dots (QD) are attractive due to their exceptionally long coherence times and compatibility with the silicon transistor platform. To achieve electrical control of spins for qubit scalability, recent experiments have utilized gradient magnetic fields from integrated micro-magnets to produce an extrinsic coupling between spin and charge, thereby electrically driving electron spin resonance (ESR). However, spins in silicon QDs experience a complex interplay between spin, charge, and valley degrees of freedom, influenced by the atomic scale details of the confining interface. Here, we report experimental observation of a valley dependent anisotropic spin splitting in a Si QD with an integrated micro-magnet and an external magnetic field. We show by atomistic calculations that the spin-orbit interaction (SOI), which is often ignored in bulk silicon, plays a major role in the measured anisotropy. Moreover, inhomogeneities such as interface steps strongly affect the spin splittings and their valley dependence. This atomic-scale understanding of the intrinsic and extrinsic factors controlling the valley dependent spin properties is a key requirement for successful manipulation of quantum information in Si QDs.