Rigorous control synthesis for an unmanned aerial vehicle necessitates the availability of a good, reasonable model for such a vehicle. The work reported in this paper focuses on the modeling of a rotary unmanned aerial vehicle (RUAV) and the development of a robust controller that can handle parameter uncertainties and disturbances. The parameters of the plant model are obtained through the use of the prediction error method with real flight data. The response of the identified linear model shows a good match with the measured flight data. The $H_{infty}$ control scheme is applied to obtain a robustly stable controller using the identified model. The proposed controller is analyzed using frequency-domain analysis and time-domain simulations. The performance of the proposed $H_{infty}$ controller is better than that of the conventional proportional derivative controller in that the proposed controller has a shorter settling time and less overshoot. Furthermore, the degradation of the proposed controller performance is negligible and stability is maintained when the input gains to the plant are doubled, which demonstrates the good performance and robustness of the controller.