This study attempts to make a compact humanoid robot acquire a giant-swing motion without any robotic models by using reinforcement learning; only the interaction with environment is available. Generally, it is widely said that this type of learning method is not appropriated to obtain dynamic motions because Markov property is not necessarily guaranteed during the dynamic task. However, in this study, we try to avoid this problem by embedding the dynamic information in the robotic state space; the applicability of the proposed method is considered using both the real robot and dynamic simulator. This paper, in particular, discusses how the robot with 5-DOF, in which the Q-Learning algorithm is implemented, acquires a giant-swing motion. Further, we describe the reward effects on the Q-Learning. Finally, this paper demonstrates that the application of the Q-Learning enable the robot to perform a very attractive giant-swing motion.