On the framework of a linearly perturbed Friedmann-Robertson-Walker spacetime, we derive an expression for the cosmological angular diameter distance affected by scalar and tensor perturbations. Our expression is applicable in linear order to distances in general FRW models. We study the effect of a stocastic gravitaional wave background on the two-point correlation function of the angular diameter distance fluctuations and, on the basis of this we also derive an expression for the power spectrum of the angular diameter distance fluctuations.