In-band full-duplex systems can transmit and receive information simultaneously and on the same frequency band. However, due to the strong self-interference caused by the transmitter to its own receiver, the use of non-linear digital self-interference cancellation is essential. In this work, we describe a hardware architecture for a neural network-based non-linear self-interference (SI) canceller and we compare it with our own hardware implementation of a conventional polynomial based SI canceller. Our results show that, for the same SI cancellation performance, the neural network canceller has an 8.1x smaller area and requires 7.7x less power than the polynomial canceller. Moreover, the neural network canceller can achieve 7 dB more SI cancellation while still being 1.2x smaller than the polynomial canceller and only requiring 1.3x more power. These results show that NN-based methods applied to communications are not only useful from a performance perspective, but can also lead to order-of-magnitude implementation complexity reductions.