A cyclic viscoelastic-viscoplastic constitutive model for clay is incorporated into an effective stress based seismic response analysis to describe viscous effect of clay layer to sand layer during earthquake. The seismic response against main shock of 1995 Hyogoken Nambu Earthquake is analyzed in the present study. Acceleration responses in both clay layer and just upper liquefiable sand layer are damped due to viscous effect of clay. A cyclic viscoelastic-viscoplastic constitutive model for clay was implemented into a FEM code, and $Newmark{eta}$ method was employed for the time discretization in the finite element formulation. Seismic responses were simulated by numerical method with recorded data at Port Island, Kobe, Japan. As results of this study, it was found that a cyclic viscoelastic-viscoplastic constitutive model can give good description of dynamic behavior characteristics including viscoelastic effect.