In order to investigate the effect of contained impurities of polyester fiber fabric on the scouring and dyeing property, 3 kinds of polyester fabrics were heat-set treated at $160-200^circ{C}$ and evaluated its scouring and dyeing property by through with soxhlet extraction and K/S value of the dyed fabric derived from the surface reflectance. The impurities such as oiling and sizing agent are adhered more strongly on the polyester fabric surface by heat setting temperature. In scouring test, the removal of the these re-adhered impurities on the polyester fabric is very difficult in comparing with unheat-set treated polyester fabric. It is also confirmed that the remained impurities on the polyester fiber decreased its dyeing property.