In this paper, we propose a robust pupil detection method using rank order filter and pixel value difference in facial image. We have detected the potential pupil candidates using rank order filter. Many false pupil candidates found at eyebrow are removed using the fact that the pixel difference is much at the boundary between pupil and sclera. The rest pupil candidates are grouped into pairs. Each pair is verified according to geometric constraints such as the angle and the distance between two candidates. A fitness function is obtained for each pair using the pixel values of two pupil regions, we select a pair with the smallest fitness value as a final pupil. The experiments have been performed for 400 images of the BioID face database. The results show that it achieves more than 90% accuracy, and especially the proposed method improves the detection rate and high accuracy for face with spectacle.