In this paper, we propose the classification and tracking method of the hand region using deformable template and condensation. To do this, first, we extract the hand region by using the fuzzy color filter and HCbCr color model. Second, we extract the edge of hand by applying the Canny edge algorithm. Third, we find the first template by calculating the conditional probability between the extracted edge and the model edge. If the accurate template of the first object is decided, the condensation algorithm tries to track it. Finally, we demonstrate the effectiveness and feasibility of the proposed method through some experiments.