Recently, a novel class of Recurrent Neural Networks (RNNs) known as Structured State-space Models (SSMs) has emerged, leveraging dynamical system properties. While most SSM architectures use linear time-invariant systems as the recurrent unit, bilinear systems offer a more expressive alternative. Although existing studies impose structural restrictions on the bilinear systems, stability is not guaranteed, potentially leading to unstable or ill-posed training. This paper introduces a generic bilinear system as the recurrent unit for SSMs. A stability condition based on Integral Quadratic Constraints (IQCs) is derived to ensure the model's stability during and after the training. To this purpose, a free parameterisation of this stability condition is provided, enabling the use of gradient-based optimisation algorithms. Moreover, a Parallel Scan algorithm is provided for forward propagation to enhance the training efficiency. The effectiveness of the proposed architecture is demonstrated by applying it to the non-linear system identification task for an F-16 ground vibration benchmark while incorporating the prior regarding the system stability into the learning process.