This paper deals with the dynamic hand gesture recognition based on computer vision using neural networks. This paper proposes a global search method and a local search method to recognize the hand gesture. The global search recognizes a hand among the hand candidates through the entire image search, and the local search recognizes and tracks only the hand through the block search. Dynamic hand gesture recognition method is based on the skin-color and shape analysis with the invariant moment and direction information. Starting point and ending point of the dynamic hand gesture are obtained from hand shape. Experiments have been conducted for hand extraction, hand recognition and dynamic hand gesture recognition. Experimental results show the validity of the proposed method.