In this paper, the characteristics of contact surface damage due to fretting in a press-fitted shaft below the fretting fatigue limit are proposed by experimental methods. A series of fatigue tests and interrupted fatigue tests of small scale press-fitted specimen were carried out by using rotating bending fatigue test machine. Macroscopic and microscopic characteristics were examined using scanning electron microscope (SEM), optical microscope or profilometer. It is found that fretting fatigue cracks were initiated even under the fretting fatigue limit on the press-fitted shafts by fretting damage. The fatigue cracks of press-fitted shafts were initiated from the edge of contact surface and propagated inward in a semi-elliptical shape. Furthermore, the fretting wear rates at the contact edge are increased rapidly at the initial stage of total fatigue life. After steep increasing, the increase of wear rate is nearly constant under the load condition below the fretting fatigue limit. It is thus suggested that the fretting wear must be considered on the fatigue life evaluation because the fatigue crack nucleation and propagation process is strongly related to the evolution of surface profile by fretting wear in the press-fitted structures.