Interpattern association (IPA) model which the interconnection weight matrix(IWM) is constructed by the association between patterns is effective in similar pattern recognitions. But, if the number of reference patterns is increased, the ability of recognition is decreased. Using a hierarchical recognition algorithm which adopts the tree search strategy, we classified reference patterns into sub-groups by similarity. In IPA model, if input includes random noise we make it converge to reference pattern by means of input includes random noise we make it converge to reference pattern by means of increasing the number of pixels of prohibited state in IWM. In relation to reference patterns the pixel of prohibited state made partially prohibited state of no connected state using which is not included common and feature regions by each reference patterns.