Solution-based deposition of $CuIn_xGa_{1-x}Se_2$ (CIGSe) thin films is well known non-vacuum process for the fabrication of CIGSe solar cells. However, due to the usage of organic chemicals in the preparation of CIG precursor solutions, the crystallization of the polycrystalline CIGSe and the performance of CIGSe thin film solar cells were significantly affected by the carbon residues from the organic chemicals. In this work, we have tried to eliminate the carbon residues in the CIG precursor thin films efficiently by using soft-annealing process. By adjusting soft-annealing temperature, it is possible to control the amount of carbon residues in CIG precursor thin films. The reduction of the carbon residues in CIG precursors by high temperature soft-annealing improves the grain size and morphology of polycrystalline CIGSe thin films, which are also closely related with the electrical properties of CIGSe thin film solar cells.