In this paper, we present a walking guidance system for the visually impaired for use at subway stations. This system, which is based on environmental knowledge, automatically detects and recognizes both exit numbers and arrow signs from natural outdoor scenes. The visually impaired can, therefore, utilize the system to find their own way (for example, using exit numbers and the directions provided) through a subway station. The proposed walking guidance system consists mainly of three stages: (a) sign detection using the MCT-based AdaBoost technique, (b) sign recognition using support vector machines and hidden Markov models, and (c) three verification techniques to discriminate between signs and non-signs. The experimental results indicate that our sign recognition system has a high performance with a detection rate of 98%, a recognition rate of 99.5%, and a false-positive error rate of 0.152.