To optimize the combustion system in a HSDI diesel engine, a new analysis technology was developed. The in-cylinder 3-D combustion analysis was carried out by the modified KIVA-3V, and the spray characteristics for the high pressure injection system were analyzed by HYDSIM. The combustion design parameters were optimized by coupling the KIVA-3V and the iSIGHT. The optimization procedure consists of 3 steps. The $1^{st}$ step is the sampling method by the Design of Experiment(DOE), the $2^{nd}$ step is the approximation using the Neural Network method, and the $3^{rd}$ step is the optimization using the Genetic Algorithm. The developed procedures have been approved as very effective and reliable, and the computational results agree well with the experimental data. The analysis results show that the optimized combustion system in a HSDI diesel engine is capable of reducing NOx and Soot emissions simultaneously keeping a same level of the fuel consumption(BSFC).