A numerical procedure for optimizing the shape of three-dimensional channel with V-shaped ribs extruded on both walls has been carried out to enhance the turbulent heat transfer. The response surface based optimization is used as an optimization technique with Reynolds-averaged Wavier-stoked analysis. Shear stress transport (SST) turbulence model is used as a turbulence closure. Computational results for average heat transfer rate show good agreements with experimental data. The objective function is defined as a linear combination of heat transfer and friction loss-related terms with a weighting factor. Three dimensionless variables such as, rib pitch-to-rib height ratio, rib height-to-channel height ratio, and the attack angle of the rib are chosen as design variables. Nineteen training points obtained by D-optimal designs for three design variables construct a reliable response surface. In the sensitivity analysis, it is found that the objective function is most sensitive to the ratio of rib height-to-channel height ratio. And, optimal values of design variables have been obtained in a range of the weighting factor.