Although the Mohr-Coulomb and Hoek-Brown failure criteria have been adopted widely in rock mechanics, they neglect the ${sigma}_2$ effect. The result of true triaxial tests on rock samples, however, reveals that the ${sigma}_2$ effect on strength of rocks is considerable, so that rock failure criteria taking into account the influence of ${sigma}_2$ are necessary for the precise stability evaluation of rock structures. In this study, a new nonlinear 3-D failure criterion has been suggested by combining the Hoek-Brown criterion with the smooth octahedral shape function taken from Jiang & Pietruszczak (1988). The performance of the new criterion was assessed by comparing the strength predictions from both the suggested criterion and the corresponding linear 3-D criterion. The resulting fit of the new criterion to the true triaxial test data for six rock types taken from the literature shows that the criterion fits the experimental data very well. Furthermore, for the data sets having data taken in the low ${sigma}_3$ range, the nonlinear failure criterion works better than the linear criterion.