We provide a detailed study of the interplay between the different interactions which appear in the Brownian motion of a micronsized sphere immersed in a viscoelastic fluid measured with optical trapping interferometry. To explore a wide range of viscous, elastic, and optical forces, we analyze two different viscoelastic solutions at various concentrations, which provide a dynamic polymeric structure surrounding the Brownian sphere. Our experiments show that, depending on the fluid, optical forces, even if small, slightly modify the complex modulus at low frequencies. Based on our findings, we propose an alternative methodology to calibrate this kind of experimental set-up when non-Newtonian fluids are used. Understanding the influence of the optical potential is essential for a correct interpretation of the mechanical properties obtained by optically-trapped probe-based studies of biomaterials and living matter. Published by AIP Publishing.