In this paper, we address the problem of the recognition of isolated complex mono- and bi-manual hand gestures. In the proposed system, hand gestures are represented by the 3D trajectories of blobs. Blobs are obtained by tracking colored body parts in real-time using the EM algorithm. In most of the studies on hand gestures, only small vocabularies have been used. In this paper, we study the results obtained on a more complex database of mono- and bi-manual gestures. These results are obtained by using a state-of-the-art sequence processing algorithm, namely Hidden Markov Models (HMMs), implemented within the framework of an open source machine learning library.