Due to structural complicacy, structural health monitoring for civil engineering needs more accurate and effectual methods of damage identification. This study aims to import multi-source information fusion (MSIF) into structural damage diagnosis to improve the validity of damage detection. Firstly, the essential theory and applied mathematic methods of MSIF are introduced. And then, the structural damage identification method based on multi-mode information fusion is put forward. Later, on the basis of a numerical simulation of a concrete continuous box beam bridge, it is obviously indicated that the improved modal strain energy method based on multi-mode information fusion has nicer sensitivity to structural initial damage and favorable robusticity to noise. Compared with the classical modal strain energy method, this damage identification method needs much less modal information to detect structural initial damage. When the noise intensity is less than or equal to 10%, this method can identify structural initial damage well and truly. In a word, this structural damage identification method based on multi-mode information fusion has better effects of structural damage identification and good practicability to actual structures.