In many multiple regression analyses, the “multi-collinearity” problem arises since some independent variables are highly correlated with each other. Practically, the Ridge regression method is often adopted to deal with the problems resulting from multi-collinearity. We propose a better alternative method using iteration to obtain an exact least squares estimator. We prove the solvability of the proposed algorithm mathematically and then compare our method with the traditional one.