In order to achieve a force controller with high performance, an accurate torque servo is required. However, the precise torque servo for a double vane rotary actuator system has not been developed till now, due to many nonlinear characteristics and system parameter variations. In this paper, the torque servo structure for the double vane rotary actuator system is proposed based on the torque model. Nonlinear equations are set up using dynamics of the double vane rotary hydraulic actuator system. Then, to derive the torque model, the nonlinear equations are linearized using a taylor series expansion. Both effectiveness and performance of the design of torque servo are verified by torque servo experiments and applying the suggested torque model to an impedance controller.