This paper present a robust controller design method based on the estimation of velocity disturbance to construct a robust fine seek control system. A loop gain adjustment algorithm is introduced to accurately estimate the velocity disturbance in spite of the uncertainties of fine actuator. A weighting function is optimally selected from a minimum fine seek open-loop gain, calculated by estimating the velocity disturbance. A robust fine seek controller is designed by considering a robust $H_{infty}$ control problem using the weighting function. The proposed controller design method is applied to the fine seek control system of a DVD rewritable drive and is evaluated through the experimental results.