With the growth in the performance of general-purpose computing no longer following Moore's law and Dennard scaling, the use of hardware-specialized systems is common practice. Field-programmable gate arrays (FPGAs) are adopted in cloud and cyber-physical systems thanks to their hardware parallelism and reconfigurability. They are often coupled with central processing units (CPUs) to leverage the software programmability of CPUs and the hardware versatility of FPGAs. While CPUs have been remotely accessible for a relatively longer time, and so, remote exploits targeting them have been investigated, FPGA-based systems have not had the same thorough investigation of the possibility of remote attacks. As a result, heterogeneous FPGA-CPU systems face a variety of security threats that are less understood. The first step in securing systems is to evaluate the extent of the possible risks and potential adversarial effects on the system. The introduction of low-level hardware control through FPGAs into a system exposes the applications and their users to a variety of threats due to the expansion of interfaces that an adversary can leverage for malicious purposes. Specifically, the low-level programmability enables the remote execution of electrical-level attacks, notably fault injection and side-channel analysis. The work in this thesis investigates the possibility for remote FPGA-based undervolting in FPGA-based systems to inject faults affecting the FPGA and extending to other system components, specifically a CPU. The first part of this thesis explores the possibility of remote undervolting-based fault injection in FPGAs. We demonstrate how maliciously crafted FPGA circuits can enable remote fault injection. We analyze the various properties of these malicious circuits that enable the lowering of the on-chip voltage. We then present how an adversary can leverage these attacker circuits to disturb other circuits sharing the FPGA fabric. The initial evaluation of the potential for remote fault injection paves the way for our exploit capitalizing on malicious FPGA circuits to introduce faults to activate a stealthy hardware Trojan. We present a thorough evaluation of the factors leading to the success of this exploit, X-Attack, in both cloud and embedded FPGA systems. In the second part of the thesis, we investigate the possibility of FPGA-induced voltage drops affecting a CPU within the same system. Our work is the first to confirm that FPGA-based undervolting can inject faults into CPU-based software. We present an evaluation of the fault injection on bare-metal applications, and demonstrate how such faults enable differential fault analysis (DFA) against the advanced encryption standard (AES). Finally, we examine the differing effects when the victim applications are running bare metal and when they are executing on a CPU running an operating system (OS). Overall, the work presented in this thesis exposes new electrical-level fault-injection threats when the power distribution network of a system is tightly shared among trusted applications and untrusted FPGA-based applications. Moreover, the FPGA-induced malicious voltage drops are shown to propagate and affect CPU applications running within the same heterogeneous system. This exposure of threats highlights the importance of considering electrical-level security and brings to light a need for more research on the design of secure modern computing systems.