This paper presents a real-time motion restoration system for people who need remedial exercise of musculoskeletal based on Inverse Kinematics. A new approach is suggested to recognize a gesture based on restored human motion which is calculated the 3D positions of intermediate joints using 3D positions of body features estimated from images. For generating the 3D candidate positions of intermediate joints which cannot be extracted from images, we apply an Inverse Kinematics theory to compute the target position of intermediate joints. And we can reduce the number of candidate positions by applying the various physical constraints of body. Finally, we can generate the more accurate final position using the Kalman filter for a motion tracking and the relationship between the previous frame information and the candidate positions. The system provide motion information which are rotation angle and height in real-time, therefore the rehabilitation exercises can be performed based on the information and figured out proper exercise for individual status.