In this paper we propose a method for verifying video in a video telephony system implemented in DM6446 DaVinci Processor. Each frame is categorized either error free frame or error frame depending on the predefined criteria. Human face is chosen as a basic means for authenticating the video frame. Skin color based algorithm is implemented for detecting the face in the video frame. The video frame is classified as error free frame if there is single face object with clear view of facial features (eyes, nose, mouth etc.) and the background of the image frame is not different then the predefined background, otherwise it will be classified as error frame. We also implemented the image histogram based NCC (Normalized Cross Correlation) comparison for video verification to speed up the system. The experimental result shows that the system is able to classify frames with 90.83% of accuracy.