The cortical tracking of stimulus features is a crucial neural requisite of how we process continuous music. We here tested whether cortical tracking of the beat, typically related to rhythm processing, is modulated by pitch predictability and other top-down factors. Participants listened to tonal (high pitch predictability) and atonal (low pitch predictability) music while undergoing electroencephalography. We analyzed their cortical tracking of the acoustic envelope. Cortical envelope tracking was stronger while listening to atonal music, potentially reflecting listeners' violated pitch expectations and increased attention allocation. Envelope tracking was also stronger with more expertise and enjoyment. Furthermore, we showed cortical tracking of pitch surprisal (using IDyOM), which suggests that listeners' expectations match those computed by the IDyOM model, with higher surprisal for atonal music. Behaviorally, we measured participants' ability to finger-tap to the beat of tonal and atonal sequences in two experiments. Finger-tapping performance was better in the tonal condition, indicating a positive effect of pitch predictability on behavioral rhythm processing. Cortical envelope tracking predicted tapping performance for tonal music, as did pitch-surprisal tracking for atonal music, indicating that high and low predictability might impose different processing regimes. Taken together, our results show various ways that top-down factors impact musical rhythm processing.