This paper provides a novel method of detecting key captions containing player information in golf videos. We use the color pattern of captions and its repetition property to determine the key captions. The experimental results show that the proposed method achieves a much higher accuracy than existing methods.