Geometric effect of landmarks to the navigation error is investigated in the two-dimensional line-of-sight measurement based vision navigation system. DOP is derived between line-of-sight measurement error and navigation solution error. For cases of three landmarks in an area, variations of the DOP were observed through computer simulations. Vision navigation system experiments were performed for the cases. Simulation and experimental results show that navigation solution errors have similar trend to DOP values of the simulation.