We introduce the $N\times N$ random matrices $$ X_{j,k}=\exp\left(2\pi i \sum_{q=1}^d\ \omega_{j,q} k^q\right) \quad \text{with } \{\omega_{j,q}\}_{\substack{1\leq j\leq N\\ 1\leq q\leq d}} \text{ i.i.d. random variables}, $$ and $d$ a fixed integer. We prove that the distribution of their singular values converges to the local Marchenko-Pastur law at scales $N^{-\theta_d}$ for an explicit, small $\theta_d>0$, as long as $d\geq 18$. To our knowledge, this is the first instance of a random matrix ensemble that is explicitly defined in terms of only $O(N)$ random variables exhibiting a universal local spectral law. Our main technical contribution is to derive concentration bounds for the Stieltjes transform that simultaneously take into account stochastic and oscillatory cancellations. Important ingredients in our proof are strong estimates on the number of solutions to Diophantine equations (in the form of Vinogradov's main conjecture recently proved by Bourgain-Demeter-Guth) and a pigeonhole argument that combines the Ward identity with an algebraic uniqueness condition for Diophantine equations derived from the Newton-Girard identities.