The recognition of daily human activities is a decisive interface component for more intuitive virtual reality (VR) interactions. In this paper, we describe a hierarchical model of body actions based on fine-grained action primitives. The associated recognition algorithm allows on-the-fly identification of simultaneous actions. Measurements highlight robustness to participants' variability and high detection rates when using the full potential of the action model. An example illustrates an interaction with a virtual character driven by the participant's action recognition