This paper presents a video OCR system that automatically extracts closed captions from video frames as keywords (or as we called "cues") for building annotations of sport videos. In this system, text regions that contain closed captions are first identified using support vector machines (SVMs). We then enhance the identified text regions by using two groups of asymmetric filters and recognize them using commercial OCR software package. The resulting captions are recorded as cues in XML format for video annotation and retrieval task.