This paper focuses on the implementation of an efficient tracking method of a moving object using optimal representative blocks by way of a pan-tilt camera. The key idea is derived from the fact that when the image size of a moving object is shrunk in an image frame according to the distance between the mobile robot camera and the object in motion, the tracking performance of a moving object can be improved by reducing the size of representative blocks according to the object image size. Motion estimations using Edge Detection (ED) and Block-Matching Algorithm (BMA) are regularly employed to track objects by vision sensors. However, these methods often neglect the real-time vision data since these schemes suffer from heavy computational load. In this paper, a representative block able to significantly reduce the amount of data to be computed, is defined and optimized by changing the size of representative blocks according to the size of the object in the image frame in order to improve tracking performance. The proposed algorithm is verified experimentally by using a two degree-of- freedom active camera mounted on a mobile robot.