In this paper, a methodology for optimal design of PID controller using the modified genetic algorithm has been proposed to improve the transient stability at system fault in HVDC transmission system, mathematical model preparation for stability analysis, and supplementary signal control by an optimal PID controller using the modified genetic algorithm(MGA). The propriety was verified through computer simulations regarding transient stability. It means that the application of MGA-PID controller in HVDC transmission system can contribute the propriety to the improvement of the transient stability in HVDC transmission system and the design of MGA-PID controller has been proved indispensible when applied to HVDC transmission system.