In recent year s there have been increasing interests in real-time object tracking with image information. This dissertation presents a real-time object tracking method through the object recognition based on neural networks that have robust characteristics under various illuminations. This dissertation proposes a global search and a local search method to track the object in real-time. The global search recognizes a target object among the candidate objects through the entire image search, and the local search recognizes and track only the target object through the block search. This dissertation uses the object color and feature information to achieve fast object recognition. The experiment result shows the usefulness of the proposed method is verified.