This paper is concerned with the design or an LMI (Linear Matrix Inequality) -based H$infty$ controller for a line of sight (LOS) stabilization system and with its robustness performance. The linearization of the system is necessary to analyze various nonlinear characteristics, but the linearization entails modeling uncertainties which reduce its performance. In addition, the stability of the LOS can be adversely affected by angular velocity disturbances while the vehicle is moving. As the vehicle accelerates, all the factors that are Ignored and simplified for the linearization tend to Inhibit the performance of the system. The robustness in the face of these uncertainties needs to be assured. This paper employs H$infty$ control theory to address these problems and the LMI method to provide a suitable controller with minimal constraints for the system. Even though the system matrix does not have a full rank, the proposed method makes it possible to design a H$infty$ controller and to deal with R and S matrices for reducing the system order. It can be also shown that the proposed robust controller has a better disturbance attenuation and tracking performance. The LMI method is also used to enhance the applicability of the proposed reduced-order H$infty$ controller for the system given. The LMI-based H$infty$ controller has superior disturbance attenuation and reference input tracking performance, compared with that of the conventional controller under real disturbances.