In this paper, we propose a robust estimation of camera parameters from image sequence for high quality video composition. We first establish correspondence of feature points between consecutive image fields. After the establishment, we formulate a nonlinear least-square data fitting problem. When the image sequence contains moving objects, and/or when the correspondence establishment is not successful for some feature points, we get bad observations, outliers. They should be properly eliminated for a good estimation. Thus, we propose an iterative algorithm for rejecting the outliers and fitting the camera parameters alternatively. We show the validity of the proposed method using computer generated data sets and real image sequeces.