In this paper, we propose the technique for recognizing the human s emotion by using the CCD color image. To do this, we first get the face image by using skin-color from the original color image acquired by the CCD camera. And we propose the method for finding man s feature points(eyebrows, eye, nose, mouse) from the face image and the geometrical method for recognizing human s emotion (surprise, anger, happiness, sadness) from the structural correlation of man s feature feints. The proposed method in this paper recognize the human s emotion by learning the neural network. Finally, we have proven the effectiveness of the Proposed method through the experimentation.