This paper is devoted to the robust $H_{infty}$ consensus control of multi-agent systems with model parameter uncertainties and external disturbances. In particular, switching networks of multiple agents with general linear dynamics are considered, and uncertain communication delays are also taken into account. It shows that a sufficient condition in terms of linear matrix inequalities (LMIs) is derived for the robust consensus performance with a given $H_{infty}$ disturbance attenuation level, and meanwhile the unknown feedback matrix of the proposed distributed state feedback protocol is also determined. A numerical example is included to validate the theoretical results.