In the present paper, a coaxial rotating smeared crack model is proposed for mass concrete in three-dimensional space. The model is capable of applying both the constant and variable shear transfer coefficients in the cracking process. The model considers an advanced yield function for concrete failure under both static and dynamic loadings and calculates cracking or crushing of concrete taking into account the fracture energy effects. The model was utilized on Koyna Dam using finite element technique. Dam-water and dam-foundation interactions were considered in dynamic analysis. The behavior of dam was studied for different shear transfer coefficients considering/neglecting fracture energy effects. The results were extracted at crest displacement and crack profile within the dam body. The results show the importance of both shear transfer coefficient and the fracture energy in seismic analysis of concrete dams under high hydrostatic pressure.