In this paper, we present a position-based robot hand tracking scheme where a pan-tilt camera is controlled such that a robot hand is always shown in the center of an image frame. We calculate the rotation angles of a pan-tilt camera by transforming the coordinate systems. In order to identify the model parameters, we applied two optimization techniques: a nonlinear least square optimizer and a particle swarm optimizer. From the simulation results, it is shown that the considered parameter identification problem is characterized by a highly multimodal landscape; thus, a global optimization technique such as a particle swarm optimization could be a promising tool to identify the model parameters of a robot hand tracking system, whereas the nonlinear least square optimizer often failed to find an optimal solution even when the initial candidate solutions were selected close to the true optimum.