In this work, we study the security of a semi-device-independent Quantum Random Number Generator (QRNG) based on Unambiguous State Discrimination (USD). The original security proof relied on bounding the guessing probability of a classical adversary, which effectively corresponds to estimating the conditional min-entropy, leading to rather loose bounds. Thus, we present new tighter bounds on the conditional entropy directly, based on a converging sequence of optimization problems. We further extend the analysis to quantum adversaries by combining this approach with an SDP hierarchy which relies on the Gram matrix of the input states. Using numerical simulations, we investigate the protocol's robustness to both noise and finite-size effects, considering non-i.i.d. rounds of the protocol, in contrast to the assumptions made in the original proof. Finally, we propose an improvement to the protocol by introducing an input to Bob that determines his measurement, which allows us to certify security even in the quantum adversary model.