In this study the optimization of plate-fin type heat sink with vortex generator for the thermal stability is performed numerically. The optimum solutions in the heat sink are obtained when the temperature rise and the pressure drop are minimized simultaneously. Thermal performance of heat sink is influenced by the heat sink shape such as the base-part fin width, lower-part fin width, and basement thickness. To acquire the optimal design variables automatically, CFD and mathematical optimization are integrated. The flow and thermal fields are predicted using the finite volume method. The optimization is carried out by means of the sequential quadratic programming (SQP) method which is widely used for the constrained nonlinear optimization problem. The results show that the optimal design variables are as follows; B$_1$=2.584 mm, B$_2$=1.741 mm, and t=7.914 mm when the temperature rise is less than 40 K. Comparing with the initial design, the temperature rise is reduced by 4.2 K, while the pressure drop is increased by 9.43 Pa. The relationship between the pressure drop and the temperature rise is also presented to select the heat sink shape for the designers.