Objective To investigate the reliability and validity of the Korean version of the Community Assessment of Psychic Experiences-15 item positive scale (CAPE-15) in college students. Methods This study had two stages: initial screening with self-report questionnaires including the CAPE-15, and semi-structured interviews to investigate the instrument’s diagnostic validity. The initial screening involved 1,749 college students. The modified Korean version of Prodromal Questionnaire-16 item (mKPQ-16) was also administered. The criteria for ultra-high risk (UHR) of psychosis in the Comprehensive Assessment of At-Risk Mental States (CAARMS) were the gold standard for diagnosis. Results Twelve of the interviewed subjects met the CAARMS criteria for UHR of psychosis. The area under the receiver operating characteristic curve was highest (0.936) for the CAPE-15 distress score (p<0.001). The use of 6 as the cutoff for the CAPE-15 distress score resulted in the best balance of sensitivity (91.7%) and specificity (85.2%), with a favorable positive predictive value of 32.4%. The coefficients of correlation between the CAPE-15 and mKPQ-16 were significant. Conclusion The Korean version of the CAPE-15 is a good instrument for screening for psychosis risk in collegiate settings. The validation of this scale could contribute to the early identification of psychosis in the Korean community.