The Ti-Si-N coating layers were synthesized on SKD 11 steel substrate by a DC reactive magnetron co-sputtering technique with separate Ti and Si targets. The high resolution transmission electron microscopy (HRTEM) and X-ray photoelectron spectroscopy (XPS) analyses for the coating layers revealed that microstructure of Ti-Si-N layer was nanocomposite, consisting of nano-sized TiN crystallites surrounded by amorphous $Si_3$$N_4$ phase. The highest hardness value of about 39 GPa was obtained at the Si content of ~11at.%, where the microstructure had fine TiN crystallites (about 5nm in size) dispersed uniformly in amorphous matrix. As the Si content in Ti-Si-N films increased, the TiN crystallites became from aligned to randomly oriented microstructure, finer, and fully penetrated by amorphous phase. Free Si appeared in the layers due to the deficit of nitrogen source at higher Si content. Friction coefficient and wear rate of the Ti-Si-N coating layer significantly decreased with increase of relative humidity. The self-lubricating tribe-layers such as $SiO_2$ or (OH)$Si_2$ seemed to play an important role in the wear behavior of Ti-Si-N film against steel.