Building a large-scale quantum computer requires the co-optimization of both the quantum bits (qubits) and their control electronics. By operating the CMOS control circuits at cryogenic temperatures (cryo-CMOS), and hence in close proximity to the cryogenic solid-state qubits, a compact quantum-computing system can be achieved, thus promising scalability to the large number of qubits required in a practical application. This work presents a cryo-CMOS microwave signal generator for frequency-multiplexed control of 4 x 32 qubits (32 qubits per RF output). A digitally intensive architecture offering full programmability of phase, amplitude, and frequency of the output microwave pulses and a wideband RF front end operating from 2 to 20 GHz allow targeting both spin qubits and transmons. The controller comprises a qubit-phase-tracking direct digital synthesis (DDS) back end for coherent qubit control and a single-sideband (SSB) RF front end optimized for minimum leakage between the qubit channels. Fabricated in Intel 22-nm FinFET technology, it achieves a 48-dB SNR and 45-dB spurious-free dynamic range (SFDR) in a 1-GHz data bandwidth when operating at 3 K, thus enabling high-fidelity qubit control. By exploiting the on-chip 4096-instruction memory, the capability to translate quantum algorithms to microwave signals has been demonstrated by coherently controlling a spin qubit at both 14 and 18 GHz.