This paper presents the use of a novel fiber optic accelerometer system to monitor ambient vibration (both wind-induced one and vehicle-induced) of a real bridge structure. This sensor system integrates the $Moir'{e}$ fringe phenomenon with fiber optics to achieve accurate and reliable measurements. A low-cost signal processing unit implements unique algorithms to further enhance the resolution and increase the dynamic bandwidth of the sensors. The fiber optic accelerometer has two major benefits in using this fiber optic accelerometer system for monitoring civil engineering structures. One is its immunity to electromagnetic (EM) interference making it suitable for difficult applications in such environments involving strong EM fields, electrical spark-induced explosion risks, and cabling problems, prohibiting the use of conventional electromagnetic accelerometers. The other is its ability to measure both low- and high-amplitude vibrations with a constantly high resolution without pre-setting a gain level, as usually required in a conventional accelerometer. The second benefit makes the sensor system particularly useful for real-time measurement of both ambient vibration (that is often used for structural health monitoring) and strong motion such as earthquake. Especially, the semi-strong motion and the small ambient one are successfully simulated and measured by using the new fiber optic accelerometer in the experiment of the structural health monitoring of a real bridge.