We propose an all-optical switch based on an asymmetric directional coupler structure with epsilon-near-zero (ENZ) layer. The nonlinear optical properties the of ENZ layer are analyzed by hot-electron dynamics process, and the all-optical operating performance of the switch on the silicon nitride platform is investigated. It is found that the pump-induced refractive index change in ENZ layer gives rise to a transfer of signal light in the optical system. We demonstrate that the proposed switch design features an insertion loss of < 2.7 dB, low crosstalk of < - 18.93 dB, and sub-pico-second response time at the communication wavelength of 1.55 mu m. With ultrafast response, high performance, and simple structure, the device provides new possibilities for all-optical communication and signal processing.