Hydraulic excavators are the representative of field robot and have been used in various fields of construction. Since the excavator operates in the hazardous working environment, operators of excavator are exposed in harmful environment. Therefore, the hydraulic excavator automation and remote operation system has been investigated to protect from the hazardous working environment. In this paper, remote operation excavator system is developed using the mini hydraulic excavator and the tracking control system of each links of excavator is designed. To apply the tracking control system, the adaptive sliding mode control algorithm is proposed. It is found that the performance of the proposed control system is improved through experimental results of using the remote operation excavator system.