An experimental method based on flexural wave propagation is proposed for identification of structural damage in rail fastening systems. The vibration of a rail clamped and supported by viscoelastic pads is significantly influenced by dynamic support properties. Formation of a defect in the rail fastening system induces changes in the flexural wave propagation characteristics owning to the discontinuity in the structural properties. In this study, frequency-dependent support stiffness was measured to monitor this change by a transfer function method. The sensitivity of wave propagation on the defect was measured from the potential energy stored in a continuously supported rail. Further, the damage index was defined as a correlation coefficient between the change in the support stiffness and the sensitivity. The defect location was identified from the calculated damage index.