Time series classification (TSC) is an important and challenging problem in machine learning. In this work, we tackle the problem of TSC by first applying a Bidirectional Encoder Representations from Transformers (BERT) model, and then applying a convolutional neural network (CNN) for the classification. We report suboptimal results compared to the state-of-the-art, as the model is overfitting early in the training pro- cess. We believe that this issue might be overcome by tuning the hyperparameters more carefully.