We investigated the effects of $O_2$ annealing (i.e., temperature and time) on the characteristics of hafnium silicate ($HfSi_xO_y$) films deposited on a Si substrate by atomic layer deposition process (ALD). We found that the post deposition annealing under oxidizing ambient causes the oxidation of residual Hf metal components, resulting in the improvement of electrical characteristics (e.g., hysteresis window and leakage current are decreased). In addition, we observed the annealing temperature is more important than the annealing time for post deposition annealing. Based on these observations, we suggest that post deposition annealing under oxidizing ambient is necessary to improve the electrical characteristics of $HfSi_xO_y$ films deposited by ALD. However, the annealing temperature has to be carefully controlled to minimize the regrowth of interfacial oxide, which degrades the value of equivalent oxide thickness.