Translation elongation plays a key role in cellular homeostasis, and dysregulation of this process has been implicated in various diseases and metabolic disorders. Uncovering the causes of intragenic heterogeneity of translation, especially in contexts of different amino acid deprivations, could help increase our understanding of these disorders and pave the way for novel therapeutics. Ribosome profiling provides accurate measurements for the genome-wide ribosome footprints, which could be utilized to investigate these mechanisms. Here we present Riboclette, a conditional deep learning model featuring a dual output head that uses the mRNA sequence input to accurately predict the ribosome footprint profiles across six amino acid deprivation conditions. Exploiting standard interpretability methods, we identify specific codons related to deprived amino acids, poly-basic regions, and negatively charged amino acids as the primary drivers of the stalling response. Moreover, we use Riboclette to extract motif level drivers for ribosome stalling by performing in silico perturbation experiments. The extracted motifs highlight the cumulative contribution of deprived codons in triggering ribosome stalling, which, depending on the condition, have been identified to affect translation up to ten codons downstream. Moreover, motifs precisely explain stalling at different codon positions, allowing for the differentiation between expected determinants of rare stalling events. Our framework offers an accurate and explainable method for understanding the effect of different amino acid deprivation conditions at a codon resolution to help elucidate the impact of intragenic variations on the regulation of translation elongation.