The hybrid neural networks have characteristics such as fast learning times, generality, and simplicity, and are mainly used to classify learning data and to model non-linear systems. The middle layer of a hybrid neural network clusters the learning vectors by grouping homogenous vectors in the same cluster. In the clustering procedure, the homogeneity between learning vectors is represented as the distance between the vectors. Therefore, if the distances between a learning vector and all vectors in a cluster are smaller than a given constant radius, the learning vector is added to the cluster. However, the usage of a constant radius in clustering is the primary source of errors and therefore decreases the recognition success rate. To improve the recognition success rate, we proposed the enhanced hybrid network that organizes the middle layer effectively by using the enhanced ART1 network adjusting the vigilance parameter dynamically according to the similarity between patterns. The results of experiments on a large number of calling card images showed that the proposed algorithm greatly improves the character extraction and recognition compared with conventional recognition algorithms.