A numerical optimization has been carried out to determine the shape of the three-dimensional channel with oblique ribs attached on both walls to enhance turbulent heat transfer. The response surface based optimization is used as an optimization technique with Reynolds-averaged Navier-Stokes analysis of fluid flow and heat transfer. Shear stress transport (SST) turbulence model is used as a turbulence closure. Numerical results fur heat transfer rate show good agreements with experimental data. four dimensionless variables such as, rib pitch-to-rib height ratio, rib height-to-channel height ratio, streamwise rib distance on opposite wall to rib pitch ratio, and the attack angle of the rib are chosen as design variables. The objective function is defined as a linear combination of heat-transfer and friction-loss related coefficients with a weighting factor. D-optimal method is used to determine the training points as a means of design of experiment. Sensitivity of the objective parameters to each design variable has been analyzed. And, optimal values of the design variables have been obtained in a range of the weighting factor.