This paper proposes a novel method for the recognition of passports based on the CDM(Conditional Dilation Morphology) masking and the ART2-based RBF neural networks. For the extraction of individual codes for recognizing, this paper targets code sequence blocks including individual codes by applying Sobel masking, horizontal smearing and a contour tracking algorithm on the passport image. Individual codes are recovered and extracted from the binarized areas by applying CDM masking and vertical smearing. This paper also proposes an ART2-based hybrid network that adapts the ART2 network for the middle layer. This network is applied to the recognition of individual codes. The experiment results showed that the proposed method has superior in performance in the recognition of passport.