Modular multilevel matrix converter is a type of converter used in medium-voltage ac-ac power conversion. Controlling the energy content of the converter arms is a prerequisite for a proper converter operation, and at the same time a challenging control task. So far, there are several approaches presented in the literature that successfully deal with the challenge. However, almost all of them fail when it comes to the simplicity of implementation, and ease of understanding by control engineers and researchers entering the field. This paper extends the direct arm energy control concept, already introduced for the class of modular multilevel converters, to the modular multilevel matrix converter. Presented control approach is characterized by an intuitive and simple implementation, ability to control the arm energies to arbitrary values, and stable operation under all operating conditions, including grid imbalances. Validity of the control concept was demonstrated using an in-house-developed hardware-in-the-loop simulator of the modular multilevel matrix converter, with control algorithms being deployed to the industrial-grade controller.