Three-dimensional (3D) knee angle measurement is one of the key measures in human gait analysis. Inertial sensor capable of measuring joint motion under unconstrained conditions is a practical tool for clinical evaluation and rehabilitation. An inertial measurement unit (IMU) consisting of accelerometer and gyroscope allows orientation measurement in 3D with an additional sensor (i.e., magnetometer). However, ferromagnetic interference negatively affects the performance of magnetometer and thus reduces measurement accuracy. In this study, a technique based on nonlinear autoregressive neural network with exogenous inputs (NARX) is presented to measure 3D segmental orientation during gait without the use of magnetometer. With IMUs attached to the thigh and shank, 3D knee angles in long-distance treadmill walking were computed and validated against an optical motion analysis system as the gold standard. Pseudo-integrator (PI) was also compared to the reference system for benchmarking. The learning capability of NARX was further assessed with the comparison of complementary filter (CF) to the reference system. The proposed NARX model was shown to outperform PI with biases between -3.5 degrees and -0.2 degrees, and root mean square errors between 4.5 degrees and 2.5 degrees. Results demonstrated the capability of NARX in providing accurate estimates of 3D knee joint angle while avoiding interference as encountered in systems incorporating magnetometer, suggesting that NARX is feasible to computing long-term ambulatory measurements of body segment orientation and 3D joint angles.