With increasing power densities, runtime thermal management is becoming a necessity in today’s systems, especially so for highly integrated Multi-Processor Systems-on-Chip (MPSoCs). In this paper, we propose a neural network (NN) based approach to implement an on-chip thermal simulator to aid such runtime management for MPSoCs. The proposed method combines the advantage of approximating the thermal properties of the chip as a linear system with the ease of fully parallel analog implementation of NNs. We perform a case study with the Niagara UltraSPARC T1 MPSoC for real-life applications, benchmarking our results with an accurate higher order Runge-Kutta (RK4) solver, that is employed in tools such as HotSpot. Within a few gate delays, the proposed NN design can simulate temperatures of the MPSoC 500 ms into the future - corresponding to thousands of iterations of the RK4 solver, with a maximum error of 1-2 K