In this study, ternary Ti-Mo-N and new quaternary Ti-Mo-Si-N coatings were synthesized on steel substrates(AISI D2) and Si wafers by a hybrid coating system of arc ion plating (AIP) using Ti target and d.c. magnetron sputtering technique using Mo and Si targets in $N_2/Ar$ gaseous mixture. Ternary Ti-Mo-N coatings were substitutional solid-solution of (Ti, Mo)N and showed maximum hardness of approximately 30 GPa at the Mo content of ${sim}10$. %. The Ti-Mo-Si-N coating with the Si content of 8.8 at. % was a composite consisting of fine (Ti, Mo)N crystallites and amorphous $Si_3N_4$ phase. The hardness of the Ti-Mo-Si(8.8 at. %)-N coatings exhibited largely increased hardness value of ${sim}48$ GPa due to the microstructural evolution to the fine composite microstructure and the refinement of (Ti, Mo)N crystallites. The average friction coefficient of the Ti-Mo-Si-N coatings largely decreased with increase of Si content. The microstructures of Ti-Mo-Si-N coatings were investigated with instrumental analyses of XRD, XPS, and HRTEM in this work.