Objectives This study is aimed at assessing the reliability of the Pattern identification questionnaire (PIQ) developed by Korea Institute of Oriental Medicine and examining the validity of the PIQ by comparing the pattern identification scores of different groups. Methods We conducted a survey of 258 participants (79 teachers and 179 graduate students at one School of Korean Medicine) using self-reported questionnaire and all the samples were retested. The test-retest reliability was assessed by Kappa coefficient(${kappa}$) and Pearson correlation coefficient. Also we compared the differences in pattern identification scores according to sex, age and occupation. Results 1. One of 116 questions are impossible to calculate; 22 of them (18.97%) scored under 0.4 in ${kappa}$; 90(77.59%) ranged from 0.4 to 0.8 in ${kappa}$; and three questions (3.58%) scored 0.8 or over in ${kappa}$. 2. Pearson correlation coefficients between test score and retest score of all pattern identification items are 0.4 or over. 3. The mean score for pattern identification in women was generally higher than that in men, particularly in patterns of blood-deficiency, blood-stasis, yang-deficiency and kidney disease. 4. The mean score for pattern identification in the graduate student group was generally higher than that in the teacher group. Conclusion In test-retest reliability, the PIQ showed relatively high reliability. The mean pattern identification score showed differences in regards to retaining knowledge about Korean medicine. Therefore, future research involving modification of questionnaire items and confirming the validity of this questionnaire is required.