This paper describes realtime visual tracking system of moving object for three dimensional target using EKF(Extended Kalman Filter). We present a new realtime visual tracking using EKF algorithm and image prediction algorithm. We demonstrate the performance of these tracking algorithm through real experiment. The experimental results show the effectiveness of the EKF algorithm and image prediction algorithm for realtime tracking and estimated state value of filter, predicting the position of moving object to minimize an image processing area, and by reducing the effect by quantization noise of image.