The primary goal of the study was to provide some preliminary validity evidence for the Vocabulary Grade Levels Test (Busan Metropolitan City Office of Education, 2009), which is designed to measure the receptive vocabulary knowledge of learners in L2. For the purpose of the current study, 327 participants at the elementary school participated in the study and were asked to take two different vocabulary tests. Namely, a Vocabulary Size Test (Nation, 2001) and a Vocabulary Grade Levels Test. The data were analyzed using correlation in order to discover the relationship between these two types of tests. Following this, the Rasch analysis was conducted to examine the reliability and validity of the measurement in question. The data analysis showed that both grade separation reliability and item separation reliability were high, indicating that the Vocabulary Grade Levels Test well discriminates learners with a wide range of proficiency levels. The findings of the study are discussed, along with further improvements in order to ascertain the validity of this particular vocabulary test.