Vibration-based structural health monitoring is one of non-destructive evaluation (NDE) techniques for civil infrastructures. This paper presents a novel fiber optic accelerometer system to monitor civil engineering structures and a successful application of the novel sensor system for measuring ambient vibration of a real building structure. This sensor system integrates the Moire fringe phenomenon with fiber optics to achieve accurate and reliable measurements. The sensor system is immune to electromagnetic (EM) interference making it suitable for difficult applications in such environments involving strong EM fields, electrical spark-induced explosion risks, and cabling problems, prohibiting the use of conventional electromagnetic accelerometers. A prototype sensor system has been developed, together with a signal processing software. The experimental studies demonstrated the high-performance of the fiber optic sensor system. Especially, the sensor was successfully used for monitoring a real building on UCI (University of California Irvine, USA).