We propose a transform for signals defined on the sphere that reveals their localized directional content in the spatio-spectral domain when used in conjunction with an asymmetric window function. We call this transform the directional spatially localized spherical harmonic transform (directional SLSHT) which extends the SLSHT from the literature whose usefulness is limited to symmetric windows. We present an inversion relation to synthesize the original signal from its directional-SLSHT distribution for an arbitrary window function. As an example of an asymmetric window, the most concentrated band-limited eigenfunction in an elliptical region on the sphere is proposed for directional spatio-spectral analysis and its effectiveness is illustrated on the Mars topographic data-set. Finally, since such typical data-sets on the sphere are of considerable size and the directional SLSHT is intrinsically computationally demanding depending on the band-limits of the signal and window, a fast algorithm for the efficient computation of the transform is developed. The floating point precision numerical accuracy of the fast algorithm is demonstrated and a full numerical complexity analysis is presented.